add requirements version

KyanChen · Feb 27, 2024 · f4e73c3 · f4e73c3
1 parent a3677ab
commit f4e73c3
Show file tree

Hide file tree

Showing 671 changed files with 150 additions and 101 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,3 +1,8 @@
+.idea
+.DS_Store
+__pycache__
+work_dirs/
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

diff --git a/README.md b/README.md
@@ -122,36 +122,37 @@ conda create -n rsprompter python=3.10 -y
 conda activate rsprompter
 ```
 
-**Step 2**: Install [PyTorch](https://pytorch.org/get-started/locally/).
+**Step 2**: Install [PyTorch2.1.x](https://pytorch.org/get-started/locally/).
 
-Linux:
+Linux/Windows:
 ```shell
-pip install torch torchvision torchaudio
+pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121
 ```
-Windows:
+Or
+
 ```shell
-pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
+conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
 ```
 
-**Step 3**: Install [MMCV](https://mmcv.readthedocs.io/en/latest/get_started/installation.html).
+**Step 3**: Install [MMCV2.1.x](https://mmcv.readthedocs.io/en/latest/get_started/installation.html).
 
 ```shell
 pip install -U openmim
-mim install "mmcv>=2.0.0"
+mim install mmcv==2.1.0
 ```
 
 **Step 4**: Install other dependencies.
 
 ```shell
-pip install -U transformers wandb einops pycocotools shapely scipy terminaltables importlib peft mat4py
+pip install -U transformers==4.38.1 wandb==0.16.3 einops pycocotools shapely scipy terminaltables importlib peft==0.8.2 mat4py==0.6.0 mpi4py
 ```
 
 **Step 5**: [Optional] Install DeepSpeed.
 
 If you want to use DeepSpeed to train the model, you need to install DeepSpeed. The installation method of DeepSpeed can refer to the [DeepSpeed official document](https://github.com/microsoft/DeepSpeed).
 
 ```shell
-pip install deepspeed
+pip install deepspeed==0.13.4
 ```
 
 Note: The support for DeepSpeed under the Windows system is not perfect yet, we recommend that you use DeepSpeed under the Linux system.
@@ -239,7 +240,7 @@ We provide the configuration files of the SAM-based models used in the paper, wh
 - `vis_backends-WandbVisBackend`: Configuration of network-side visualization tools, **after opening the comment, you need to register an account on the `wandb` official website, and you can view the visualization results during training in the web browser**.
 - `num_classes`: The number of categories in the dataset, **which needs to be modified according to the number of categories in the dataset**.
 - `prompt_shape`: The shape of the Prompt, the first parameter represents $N_p$, and the second parameter represents $K_p$, which generally does not need to be modified.
-- `hf_sam_pretrain_name`: The name of the SAM model on HuggingFace Spaces, which generally does not need to be modified.
+- `hf_sam_pretrain_name`: The name of the SAM model on HuggingFace Spaces, **which needs to be modified to your own path**, you can use the [download script](tools/rsprompter/download_hf_sam_pretrain_ckpt.py) to download.
 - `hf_sam_pretrain_ckpt_path`: The checkpoint path of the SAM model on HuggingFace Spaces, **which needs to be modified to your own path**, you can use the [download script](https://github.com/KyanChen/RSPrompter/blob/release/tools/rsprompter/download_hf_sam_pretrain_ckpt.py) to download.
 - `model-decoder_freeze`: Whether to freeze the parameters of the SAM decoder, which generally does not need to be modified.
 - `model-neck-feature_aggregator-hidden_channels`: The hidden channel number of the feature aggregator, which generally does not need to be modified.
@@ -360,6 +361,13 @@ Note: Low-resolution input images can effectively reduce the memory usage of the
 If you encounter the error `Bad substitution` when running `dist_train.sh`, please use `bash dist_train.sh` to run the script.
 
 
+### 5. Unable to access and download the model on HuggingFace Spaces
+
+If you are unable to access and download the model on HuggingFace Spaces, please use the [download script](tools/rsprompter/download_hf_sam_pretrain_ckpt.py) to download.
+Please refer to the [official processing method](https://huggingface.co/docs/transformers/installation#offline-mode).
+
+
+
 </details>
 
 ## Acknowledgement
@@ -371,11 +379,12 @@ This project is developed based on the [MMDetection](https://github.com/open-mml
 If you use the code or performance benchmarks of this project in your research, please refer to the bibtex below to cite RSPrompter.
 
 ```
-@article{chen2023rsprompter,
+@article{chen2024rsprompter,
   title={RSPrompter: Learning to prompt for remote sensing instance segmentation based on visual foundation model},
   author={Chen, Keyan and Liu, Chenyang and Chen, Hao and Zhang, Haotian and Li, Wenyuan and Zou, Zhengxia and Shi, Zhenwei},
-  journal={arXiv preprint arXiv:2306.16269},
-  year={2023}
+  journal={IEEE Transactions on Geoscience and Remote Sensing},
+  year={2024},
+  publisher={IEEE}
 }
 ```
 

diff --git a/README_zh-CN.md b/README_zh-CN.md
@@ -44,7 +44,7 @@
 
 本项目仓库是论文 [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://arxiv.org/abs/2306.16269) 的代码实现，基于 [MMDetection](https://github.com/open-mmlab/mmdetection/tree/main) 项目进行开发。
 
-当前分支在PyTorch 2.x 和 CUDA 12.1 下测试通过，支持 Python 3.7+，能兼容绝大多数的 CUDA 版本。
+当前分支在PyTorch 2.x 和 CUDA 12.1 下测试通过，支持 Python 3.8+，能兼容绝大多数的 CUDA 版本。
 
 如果你觉得本项目对你有帮助，请给我们一个 star ⭐️，你的支持是我们最大的动力。
 
@@ -101,14 +101,14 @@
 ### 依赖项
 
 - Linux 或 Windows
-- Python 3.7+，推荐使用 3.10
+- Python 3.8+，推荐使用 3.10
 - PyTorch 2.0 或更高版本，推荐使用 2.1
 - CUDA 11.7 或更高版本，推荐使用 12.1
 - MMCV 2.0 或更高版本，推荐使用 2.1
 
 ### 环境安装
 
-我们推荐使用 Miniconda 来进行安装，以下命令将会创建一个名为 `rsprompter` 的虚拟环境，并安装 PyTorch 和 MMCV。
+推荐使用 Miniconda 来进行安装，以下命令将会创建一个名为 `rsprompter` 的虚拟环境，并安装 PyTorch 和 MMCV。
 
 注解：如果你对 PyTorch 有经验并且已经安装了它，你可以直接跳转到下一小节。否则，你可以按照下述步骤进行准备。
 
@@ -123,36 +123,36 @@ conda create -n rsprompter python=3.10 -y
 conda activate rsprompter
 ```
 
-**步骤 2**：安装 [PyTorch](https://pytorch.org/get-started/locally/)。
+**步骤 2**：安装 [PyTorch2.1.x](https://pytorch.org/get-started/locally/)。
 
-Linux:
+Linux/Windows:
 ```shell
-pip install torch torchvision torchaudio
+pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121
 ```
-Windows:
+或者
 ```shell
-pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
+conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia
 ```
 
-**步骤 3**：安装 [MMCV](https://mmcv.readthedocs.io/en/latest/get_started/installation.html)。
+**步骤 3**：安装 [MMCV2.1.x](https://mmcv.readthedocs.io/en/latest/get_started/installation.html)。
 
 ```shell
 pip install -U openmim
-mim install "mmcv>=2.0.0"
+mim install mmcv==2.1.0
 ```
 
 **步骤 4**：安装其他依赖项。
 
 ```shell
-pip install -U transformers wandb einops pycocotools shapely scipy terminaltables importlib peft mat4py
+pip install -U transformers==4.38.1 wandb==0.16.3 einops pycocotools shapely scipy terminaltables importlib peft==0.8.2 mat4py==0.6.0 mpi4py
 ```
 
 **步骤 5**：[可选] 安装 DeepSpeed。
 
 如果您想使用 DeepSpeed 训练模型，您需要安装 DeepSpeed。DeepSpeed 的安装方法可以参考 [DeepSpeed 官方文档](https://github.com/microsoft/DeepSpeed)。
 
 ```shell
-pip install deepspeed
+pip install deepspeed==0.13.4
 ```
 
 注解：Windows 系统下对 DeepSpeed 的支持尚未完善，我们建议您在 Linux 系统下使用 DeepSpeed。
@@ -238,7 +238,7 @@ ${DATASET_ROOT} # 数据集根目录，例如：/home/username/data/NWPU
 - `vis_backends-WandbVisBackend`：网络端可视化工具的配置，**打开注释后，需要在 `wandb` 官网上注册账号，可以在网络浏览器中查看训练过程中的可视化结果**。
 - `num_classes`：数据集的类别数，**需要根据数据集的类别数进行修改**。
 - `prompt_shape`：Prompt 的形状，第一个参数代表 $N_p$，第二个参数代表 $K_p$，一般不需要修改。
-- `hf_sam_pretrain_name`：HuggingFace Spaces 上的 SAM 模型的名称，一般不需要修改。
+- `hf_sam_pretrain_name`：HuggingFace Spaces 上的 SAM 模型的名称，可以改为自己下载到本地的路径，[下载脚本](tools/rsprompter/download_hf_sam_pretrain_ckpt.py)。
 - `hf_sam_pretrain_ckpt_path`：HuggingFace Spaces 上的 SAM 模型的检查点路径，**需要修改为你自己的路径**，可以使用[下载脚本](tools/rsprompter/download_hf_sam_pretrain_ckpt.py)来下载。
 - `model-decoder_freeze`：是否冻结SAM解码器的参数，一般不需要修改。
 - `model-neck-feature_aggregator-hidden_channels`：特征聚合器的隐藏通道数，一般不需要修改。
@@ -358,6 +358,11 @@ python zero_to_fp32.py . $SAVE_CHECKPOINT_NAME -t $CHECKPOINT_DIR  # $SAVE_CHECK
 如果您在运行`dist_train.sh`时出现了`Bad substitution`的错误，请使用`bash dist_train.sh`来运行脚本。
 
 
+### 5. 无法下载访问和下载HuggingFace Spaces上的模型
+
+如果您无法访问和下载HuggingFace Spaces上的模型，请使用[下载脚本](tools/rsprompter/download_hf_sam_pretrain_ckpt.py)来下载。
+请参考[官方处理方式](https://huggingface.co/docs/transformers/installation#offline-mode)。
+
 </details>
 
 ## 致谢
@@ -369,11 +374,12 @@ python zero_to_fp32.py . $SAVE_CHECKPOINT_NAME -t $CHECKPOINT_DIR  # $SAVE_CHECK
 如果你在研究中使用了本项目的代码或者性能基准，请参考如下 bibtex 引用 RSPrompter。
 
 ```
-@article{chen2023rsprompter,
+@article{chen2024rsprompter,
   title={RSPrompter: Learning to prompt for remote sensing instance segmentation based on visual foundation model},
   author={Chen, Keyan and Liu, Chenyang and Chen, Hao and Zhang, Haotian and Li, Wenyuan and Zou, Zhengxia and Shi, Zhenwei},
-  journal={arXiv preprint arXiv:2306.16269},
-  year={2023}
+  journal={IEEE Transactions on Geoscience and Remote Sensing},
+  year={2024},
+  publisher={IEEE}
 }
 ```
 

diff --git a/configs/rsprompter/_base_/samdet.py b/configs/rsprompter/_base_/samdet.py
@@ -64,7 +64,8 @@
         norm_cfg=dict(type='BN', requires_grad=True),
         norm_eval=True,
         style='pytorch',
-        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
+        init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')
+    ),
     neck=dict(
         type='FPN',
         in_channels=[256, 512, 1024, 2048],

diff --git a/configs/rsprompter/rsprompter_anchor-nwpu-peft-512.py b/configs/rsprompter/rsprompter_anchor-nwpu-peft-512.py
@@ -23,8 +23,10 @@
 #### should be changed when using different pretrain model
 
 # sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"
@@ -141,10 +143,10 @@
 
 dataset_type = 'NWPUInsSegDataset'
 #### should be changed align with your code root and data root
-code_root = '/mnt/home/cky/RSPrompter'
-data_root = '/mnt/home/cky/data/NWPU'
+code_root = '/mnt/home/cky/Code/RSPrompter'
+data_root = '/mnt/home/cky/Code/RSPrompter/data/NWPU'
 
-batch_size_per_gpu = 4
+batch_size_per_gpu = 2
 num_workers = 8
 persistent_workers = True
 train_dataloader = dict(

diff --git a/configs/rsprompter/rsprompter_anchor-nwpu.py b/configs/rsprompter/rsprompter_anchor-nwpu.py
@@ -23,8 +23,10 @@
 #### should be changed when using different pretrain model
 
 # sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"
@@ -68,11 +70,11 @@
 dataset_type = 'NWPUInsSegDataset'
 
 #### should be changed align with your code root and data root
-code_root = '/mnt/home/xx/codes/RSPrompter'
-data_root = '/mnt/home/xx/data/NWPU'
+code_root = '/mnt/home/cky/Code/RSPrompter'
+data_root = '/mnt/home/cky/Code/RSPrompter/data/NWPU'
 
 
-batch_size_per_gpu = 2
+batch_size_per_gpu = 1
 num_workers = 8
 persistent_workers = True
 train_dataloader = dict(
@@ -134,7 +136,7 @@
         weight_decay=0.05)
 )
 
-#### DeepSpeed training config
+# ### DeepSpeed training config
 # runner_type = 'FlexibleRunner'
 # strategy = dict(
 #     type='DeepSpeedStrategy',

diff --git a/configs/rsprompter/rsprompter_anchor-ssdd.py b/configs/rsprompter/rsprompter_anchor-ssdd.py
@@ -22,9 +22,10 @@
 
 #### should be changed when using different pretrain model
 
-# sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"

diff --git a/configs/rsprompter/rsprompter_anchor-whu.py b/configs/rsprompter/rsprompter_anchor-whu.py
@@ -22,9 +22,10 @@
 
 #### should be changed when using different pretrain model
 
-# sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"

diff --git a/configs/rsprompter/rsprompter_query-nwpu-peft-512.py b/configs/rsprompter/rsprompter_query-nwpu-peft-512.py
@@ -12,7 +12,7 @@
 )
 
 vis_backends = [dict(type='LocalVisBackend'),
-                dict(type='WandbVisBackend', init_kwargs=dict(project='rsprompter-nwpu', group='rsprompter-query', name="rsprompter_query-nwpu-peft-512"))
+                # dict(type='WandbVisBackend', init_kwargs=dict(project='rsprompter-nwpu', group='rsprompter-query', name="rsprompter_query-nwpu-peft-512"))
                 ]
 visualizer = dict(
     type='DetLocalVisualizer', vis_backends=vis_backends, name='visualizer')
@@ -23,8 +23,10 @@
 #### should be changed when using different pretrain model
 
 # sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"
@@ -142,10 +144,10 @@
 
 dataset_type = 'NWPUInsSegDataset'
 #### should be changed align with your code root and data root
-code_root = '/mnt/home/cky/RSPrompter'
-data_root = '/mnt/home/cky/data/NWPU'
+code_root = '/mnt/home/cky/Code/RSPrompter'
+data_root = '/mnt/home/cky/Code/RSPrompter/data/NWPU'
 
-batch_size_per_gpu = 2
+batch_size_per_gpu = 1
 num_workers = 8
 persistent_workers = True
 train_dataloader = dict(

diff --git a/configs/rsprompter/rsprompter_query-nwpu.py b/configs/rsprompter/rsprompter_query-nwpu.py
@@ -22,9 +22,10 @@
 
 #### should be changed when using different pretrain model
 
-# sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"

diff --git a/configs/rsprompter/rsprompter_query-ssdd.py b/configs/rsprompter/rsprompter_query-ssdd.py
@@ -23,9 +23,10 @@
 
 #### should be changed when using different pretrain model
 
-# sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"

diff --git a/configs/rsprompter/rsprompter_query-whu.py b/configs/rsprompter/rsprompter_query-whu.py
@@ -22,9 +22,10 @@
 
 #### should be changed when using different pretrain model
 
-# sam base model
-hf_sam_pretrain_name = "facebook/sam-vit-base"
-hf_sam_pretrain_ckpt_path = "~/.cache/huggingface/hub/models--facebook--sam-vit-base/snapshots/b5fc59950038394bae73f549a55a9b46bc6f3d96/pytorch_model.bin"
+hf_sam_pretrain_name = "work_dirs/sam_cache/sam_vit_base"
+# huggingface model name, e.g. facebook/sam-vit-base
+# or local repo path, e.g. work_dirs/sam_cache/sam_vit_base
+hf_sam_pretrain_ckpt_path = "work_dirs/sam_cache/sam_vit_base/pytorch_model.bin"
 # # sam large model
 # hf_sam_pretrain_name = "facebook/sam-vit-large"
 # hf_sam_pretrain_ckpt_path = "~/.cache//huggingface/hub/models--facebook--sam-vit-large/snapshots/70009d56dac23ebb3265377257158b1d6ed4c802/pytorch_model.bin"