A collection of generative algorithms and techniques implemented in Python.

This work is part of the Xecs TASTI project, nr. 2022005.

You will need:

• python (see pyproject.toml for full version)
• Git
• uv
• a .secrets file with the required secrets and credentials
• load environment variables from .env
• NVIDIA Drivers(mandatory) and CUDA >= 12.1 (mandatory if Docker/Apptainer is not used)
• Weights & Biases account

Clone this repository (requires git ssh keys)

git clone --recursive [email protected]:caetas/GenerativeZoo.git
cd GenerativeZoo

Create the environment and install the dependencies:

uv sync --python3.12

You can activate the environment with:

source .venv/bin/activate

You might be required to run the following command once to setup the automatic activation of the conda environment and the virtualenv:

direnv allow

Feel free to edit the .envrc file if you prefer to activate the environments manually.

You can activate the environment with:

.venv-dev/Scripts/Activate.ps1

Create a .secrets file and add your Weights & Biases API Key:

WANDB_API_KEY = <your-wandb-api-key>

Create the image using the provided Dockerfile

docker build --tag generativezoo .

Or download it from the Hub:

docker pull docker://ocaetas/generativezoo

Then run the script job_docker.sh that will execute main.sh:

cd scripts
bash job_docker.sh

To access the shell, please run:

docker run --rm -it --gpus all --ipc=host --env-file .env -v $(pwd)/:/app/ generativezoo bash

Convert the Docker Image to a .sif file:

apptainer pull generativezoo.sif docker://ocaetas/generativezoo

Then run the script job_apptainer.sh that will execute main.sh:

cd scripts
bash job_apptainer.sh

To access the shell, please run:

apptainer shell --nv --env-file .env --bind $(pwd)/:/app/ generativezoo.sif

Add the flag --nvccli if you are using WSL.

Note: Edit the main.sh script if you want to train a different model.

The listed models are already implemented and fully integrated in the model zoo. Models with 🚀 support Multi-GPU and Mixed Precision training using Accelerate. Models with 📦 support a latent space implementation with the flag --latent.

• Vanilla VAE Paper | Code¹ | Script | Documentation
• Conditional VAE Paper | Code¹ | Script | Documentation
• Hierarchical VAE Paper | Code² | Script | Documentation

• Adversarial VAE Paper | Code¹ | Script | Documentation
• DC-GAN Paper | Code³ | Script | Documentation
• Conditional GAN Paper | Code³ | Script | Documentation
• CycleGAN Paper | Code⁴ | Script | Documentation
• Prescribed GAN Paper | Code⁵ | Script | Documentation
• Wasserstein GAN with Gradient Penalty Paper | Code⁶ | Script | Documentation
• VQ-GAN Paper | Code²⁴ | Script | Documentation

• DDPM^🚀📦 Paper | Code^7,8,9,21 | Script | Documentation
• Conditional DDPM^🚀📦 Paper | Code^10,21 | Script | Documentation
• Diffusion AE Paper | Code¹¹ | Script | Documentation

• SGM^🚀📦 Paper | Code^12,21 | Script | Documentation
• NCSNv2 Paper | Code¹³ | Script | Documentation

• GPT Paper | Code²² | Script | Documentation
• MaskGiT Paper | Code²³ | Script | Documentation
• PixelCNN Paper | Code¹⁴ | Script | Documentation

• Vanilla Flow Paper | Code¹⁴ | Script | Documentation
• RealNVP Paper | Code¹⁵ | Script | Documentation
• Glow Paper | Code¹⁶ | Script | Documentation
• Flow++ Paper | Code¹⁷ | Script | Documentation

• Flow Matching^🚀📦 Paper | Code^18,21 | Script | Documentation
• Conditional Flow Matching^🚀📦 Paper | Code^18,21 | Script | Documentation
• Rectified Flows^🚀📦 Paper | Code¹⁹ | Script | Documentation

• Stable Diffusion + LoRA^🚀 Paper | Code²⁰ | Script | Documentation
• ControlNet^🚀 Paper | Code²⁰ | Script | Documentation
• InstructPix2Pix^🚀 Paper | Code²⁰ | Script | Documentation

Accelerate should be configured for your hardware setup using:

accelerate config

The models that support these features should be launched with:

accelerate launch --multi_gpu --mixed_precision=fp16 --num_processes=2 {script_name.py} {--arg1} {--arg2} ...

The following datasets are ready to be used to train and sample from the provided models. They are automatically downloaded when you try to use them for the first time.

• MNIST Source
• FashionMNIST Source
• ChestMNIST++ Source
• OctMNIST++ Source
• PneumoniaMNIST++ Source
• TissueMNIST++ Source

• CIFAR-10 Source
• CIFAR-100 Source
• SVHN Source
• Places365 Source
• DTD Source
• TinyImageNet Source MANUAL DOWNLOAD REQUIRED Link
• Horse2Zebra Source MANUAL DOWNLOAD REQUIRED Link
• ImageNet-1k Source
• CelebA Source

The code examples are setup to use Weights & Biases as a tool to track your training runs. Please refer to the full documentation if required or follow the following steps:

1. Create an account in Weights & Biases

2. If you have installed the requirements you can skip this step. If not, activate the conda environment and the virtualenv and run:

   pip install wandb

3. Run the following command and insert you API key when prompted:

   wandb login

If you want to fully disable Weights & Biases during training, use the flag --no_wandb.

If you want to turn off syncing with the server but want to retain the local copy, run:

wandb offline 

Full documentation is available here: docs/.

See the Developer guidelines for more information.

Contributions of any kind are welcome. Please read CONTRIBUTING.md for details and the process for submitting pull requests to us.

Please read MODELRULES.md for details on how you should build your models for this repository.

Thank you for improving the security of the project, please see the Security Policy for more information.

This project is licensed under the terms of the CC-BY-4.0 license. See LICENSE for more details.

All the repositories used to generate this code are mentioned in each of the corresponding files and referenced in Implemented Models:

1. PyTorch-VAE
2. nvae
3. conditional-GAN
4. PyTorch-GAN
5. PresGANs
6. wgan-gp
7. minDiffusion
8. DenoisingDiffusionProbabilisticModels
9. ddim
10. Conditional_Diffusion_MNIST
11. Generative Models
12. SGM Tutorial
13. score_sde_pytorch
14. uvadlc_notebooks
15. real-nvp
16. Glow-PyTorch
17. flowplusplus
18. conditional-flow-matching
19. minRF
20. diffusers
21. guided-diffusion
22. nanoGPT
23. Halton-MaskGIT
24. taming-transformers

If you publish work that uses GenerativeZoo, please cite GenerativeZoo as follows:

@misc{GenerativeZoo,
author = {Francisco Caetano},
title = {A collection of generative algorithms and techniques implemented in Python.},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/caetas/GenerativeZoo}},
year = {2024},
}