If you have a CUDA environment, please make sure the environment variable CUDA_HOME
is set. It will be compiled under CPU-only mode if no CUDA available.
Please make sure following the installation steps strictly, otherwise the program may produce:
NameError: name '_C' is not defined
echo $CUDA_HOME
If it print nothing, then it means you haven't set up the path/
Run this so the environment variable will be set under current shell.
export CUDA_HOME=/path/to/cuda-11.3
Notice the version of cuda should be aligned with your CUDA runtime, for there might exists multiple cuda at the same time.
If you want to set the CUDA_HOME permanently, store it using:
echo 'export CUDA_HOME=/path/to/cuda' >> ~/.bashrc
after that, source the bashrc file and check CUDA_HOME:
source ~/.bashrc
echo $CUDA_HOME
In this example, /path/to/cuda-11.3 should be replaced with the path where your CUDA toolkit is installed. You can find this by typing which nvcc in your terminal:
For instance, if the output is /usr/local/cuda/bin/nvcc, then:
export CUDA_HOME=/usr/local/cuda
git clone https://github.com/sterzhang/image-textualization.git
cd image-textualization
conda create --name image-textualization python=3.8 -y
conda activate image-textualization
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
git checkout cc87e7ec
pip install -e .
cd ../image-textualization/filter
git clone https://github.com/IDEA-Research/GroundingDINO.git
cd GroundingDINO/
pip install -e .
Follow the SAM repo to install packages.
Follow the instruction in GRiT repo.
Follow the instruction in LLama3 repo.
cd ../image-textualization
mkdir ckpt && cd ckpt
wget https://datarelease.blob.core.windows.net/grit/models/grit_b_densecap_objectdet.pth
wget -q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth