(NeurIPS 2023 Workshop on Diffusion Models) & ICME 2024
- Our code is based on diffusers-0.19.0
- Download the dataset from GoogleDrive.
- create environment.
conda create --name floyd --file environment.yml
conda activate floyd
- If you want to get a prompt of our own images, you can use BLIP_2.ipynb to get the text prompt.
- Reconstruct a image based on IterInv.
python end2end_inv.py \
--input_image 'images/pix2pix-zero/cat/cat_7.png' \
--results_folder 'output/all_imgs_inversion_in_prompt_file' \
--prompt_str 'a cat'
--enable_1 \
--enable_3for2 \
--enable_3 \
# or
bash bashes/bash_inv_1img.sh
- Reconstruct multiple images based on IterInv.
python bashes/ours_inv_multi_prompt.py
- Reconstruct based on DDIM Inversion. Choose stage 1/2/3 to groups what you want.
python bashes/ddim_inv_multi_prompts.py
# including ddim_stage23_inv.py ddim_failure_stage_3.py end2end_ddim_inv.py to chooose.
- Reconstruct based on SDXL.
python SDXL.py
# or in SDXL.ipynb to run it one-by-one step for better development.
- editing with IterInv + DiffEdit.
python end2end_diffedit.py \
--enable_1 --enable_3for2 --enable_3 \
--inpaint_strength 0.4
python evaluation_scores.py
# or single-step debug in evaluation_scores.ipynb
Change the folder to choose what you want to evaluate.
Thanks for the diffusers and DeepFloyd-IF, which helps us to quickly implement our ideas.
Note: this is our draft code release for paper. The cleaned version will be released later.
If our work is useful for your research, you can consider citing:
@article{tang2023iterinv,
title={IterInv: Iterative Inversion for Pixel-Level T2I Models},
author={Tang, Chuanming and Wang, Kai and van de Weijer, Joost},
journal={arXiv preprint arXiv:2310.19540},
year={2023}
}