News About Updated Manuscript 📖
- Rename LAControlNet (a little confusing😆) to IRControlNet.
- We train IRControlNet on our filtered laion2b-en dataset, which contains around 15M high-quality images. The pretrained weight is available now.
- We compare IRControlNet with 6 model variants and find that IRControlNet is good enough as a backbone for generation module.
- We support three BIR tasks: BSR, BFR and BID (Blind Image Denoising), while different tasks share the same IRControlNet. Visual examples can be found here.
- During inference, we directly use off-the-shelf restoration models for degradation removal. More details can be found here.
- We propose region-aware restoration guidance to better achieve a trade-off between quality and fidelity.
News About Code Base 👨💻
keep it simple and stupid.
- Free from pytorch lightning and LDM code base. Now the code has been rearranged as simple as possible.
- Lightning modules have been deleted.
- Put all model-related code (UNet, VAE, CLIP, etc.) to a single directory.
- Provide two minimal training scripts for training stage1 and stage2 model, built upon accelerate with the simplest training-loop style.
- Upgrade pytorch to 2.2.2 for 1) built-in sdp attention 2) torch.compile.
- Copy the clip-related code from open-clip. Now chinese users can free from the warning of failed connection with Hugging Face.
- Only save the paramters of IRControlNet, which reduces the size of checkpoint from 9GB to 1GB.