This repository is created to fine-tune your VAE model of Stable Diffusion model, which you can change input image size, or with a new dataset.

Note: I follow the guidance here, in which some first epochs are trained with (l1 + Lpips), later epochs are trained with (l2 + 0.1*Lpips) loss.

Please download pre-trained SD model here, and put into sd_model folder.

Copy ldm folder from this repo into your current directory.

For training, run the following script:

python train.py \
	--data_dir <YOUR DATA FOLDER>
	--batch_size 2 \
	--num_epochs 20 \
	--lr 2e-5 \
	--val_size 0.1 \
	--precision 16 \
	--image_size <YOUR DESIRED SIZE> \
	--lpips_loss_weight 1.0 \
	--ema_decay 0.99\

If you encounter any error, there are some helpful discussions in the issue tag.