ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet

Soon Yau Cheong Armin Mustafa Andrew Gilbert

visconet_banner_full.mp4

Requirements

A suitable conda environment named control can be created and activated with:

conda env create -f environment.yaml
conda activate control

Files

All model and data files are in here. Including eval.zip containing all images used in human evaluation.

Gradio App

run python gradio_visconet.py

Citation

@article{cheong2023visconet,
        author    = {Cheong, Soon Yau and Mustafa, Armin and Gilbert, Andrew},
        title     = {ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet},
        journal = {Arxiv Preprint 2312.03154},
        month     = {December},
        year      = {2023}}