This is an implementation of DeS3: Attention-driven Self and Soft Shadow Removal using ViT Similarity and Color Convergence
git clone https://github.com/jinyeying/DeS3_Deshadow.git
cd DeS3_Deshadow/
-
SRD Train|BaiduPan, Test. Shadow Masks
-
AISTD|ISTD+ [link]
-
LRSS: Soft Shadow Dataset [link]
The LRSS dataset contains 134 shadow images (62 pairs of shadow and shadow-free images).
We use 34 pairs for testing and 100 shadow images for training. For shadow-free training images, 28 from LRSS and 72 randomly selected from the USR dataset.[Dropbox] [BaiduPan (code:t9c7)] -
USR: Unpaired Shadow Removal Dataset [link]
-
UCF, UIUC: Self Shadow [link]
[Dropbox] | [BaiduPan(code:blk7)] |
---|
- set the paths of the shadow removal result and the dataset in
evaluation/demo_SRD_RMSE.m
and then run it.
demo_SRD_RMSE.m
Get the RMSE from Table 1 in the main paper on the SRD (size: 256x256):
Method | Training | Shadow | Non-Shadow | ALL |
---|---|---|---|---|
DeS3 | Paired | 5.88 | 2.83 | 3.72 |
- set the paths of the shadow removal result and the dataset in
evaluation/evaluate_SRD_PSNR_SSIM.m
and then run it.
evaluate_SRD_PSNR_SSIM.m
Get the PSNR & SSIM from Table 1 in the main paper on the SRD (size: 256x256):
PSNR | PSNR | PSNR | SSIM | SSIM | SSIM | ||
---|---|---|---|---|---|---|---|
Method | Training | Shadow | Non-Shadow | ALL | Shadow | Non-Shadow | ALL |
DeS3 | Paired | 37.45 | 38.12 | 34.11 | 0.984 | 0.988 | 0.968 |
[Dropbox] | [BaiduPan(code:blk7)] |
---|
- modify the path in
DeS3_Deshadow/datasets/aistdshadow.py
Line 30 in c294476
DeS3_Deshadow/configs/AISTDshadow.yml
Line 6 in c294476
- download the AISTD checkpoint [Dropbox] | [BaiduPan(code:aistd)]
CUDA_VISIBLE_DEVICES=1,2 python train_aistd.py --config 'AISTDshadow.yml' --resume '/home1/yeying/DeS3_Deshadow/ckpts/AISTDShadow_ddpm.pth.tar'
CUDA_VISIBLE_DEVICES=1 python eval_aistd.py --config 'AISTDshadow.yml' --resume '/home1/yeying/DeS3_Deshadow/ckpts/AISTDShadow_ddpm.pth.tar'
- set the paths of the shadow removal result and the dataset in
evaluation/demo_AISTD_RMSE.m
and then run it.
demo_AISTD_RMSE.m
Get the RMSE on the AISTD (size: 256x256):
Method | Training | Shadow | Non-Shadow | ALL |
---|---|---|---|---|
DeS3 | Paired | 6.56 | 3.40 | 3.94 |
- set the paths of the shadow removal result and the dataset in
evaluation/evaluate_AISTD_PSNR_SSIM.m
and then run it.
evaluate_AISTD_PSNR_SSIM.m
Get the PSNR & SSIM on the ISTD (size: 256x256):
PSNR | PSNR | PSNR | SSIM | SSIM | SSIM | ||
---|---|---|---|---|---|---|---|
Method | Training | Shadow | Non-Shadow | ALL | Shadow | Non-Shadow | ALL |
DeS3 | Paired | 36.49 | 34.70 | 31.38 | 0.989 | 0.972 | 0.958 |
Code is implemented based WeatherDiffusion, we would like to thank them.
The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:
- Jonathan Tan (jonathan_tano@nus.edu.sg)
If this work is useful for your research, please cite our paper.
@inproceedings{jin2024des3,
title={DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity},
author={Jin, Yeying and Ye, Wei and Yang, Wenhan and Yuan, Yuan and Tan, Robby T},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
volume={38},
number={3},
pages={2634--2642},
year={2024}
}