Direction-Aware Spatial Context Features for Shadow Detection and Removal

by Xiaowei Hu, Chi-Wing Fu, Lei Zhu, Jing Qin and Pheng-Ann Heng

This implementation is written by Xiaowei Hu at the Chinese University of Hong Kong.

---

PyTorch Version

A PyTorch version is available at https://github.com/stevewongv/DSC-PyTorch implemented by Tianyu Wang.

Results

Please find the new results (without CRF for shadow detection) at https://github.com/xw-hu/Unveiling-Deep-Shadows.

Installation

1. Please download and compile our CF-Caffe.

2. Clone the DSC repository, and we'll call the directory that you cloned as DSC-master.

   git clone https://github.com/xw-hu/DSC.git

3. Replace CF-Caffe/examples/ by DSC-master/examples/. Replace CF-Caffe/data/ by DSC-master/data/.

Test

Shadow Detection

1. Put the trained model in examples/DSC/DSC_detection/snapshot/.

2. (Matlab User) Enter the examples/DSC/ and run test_detection.m in Matlab.

3. (Python User) Enter the examples/DSC/DSC_detection/ and export PYTHONPATH in the command window such as:

   export PYTHONPATH='../../../python'

   Run the test model and resize the results to the size of original images:

   ipython notebook DSC_test.ipynb

4. Apply CRF to do the post-processing for each image.
   The code for CRF can be found in https://github.com/Andrew-Qibin/dss_crf
   *Note that please provide a link to the original code as a footnote or a citation if you plan to use it.

Shadow Removal

Enter the examples/DSC/ and run test_removal.m in Matlab.

Train

Download the pre-trained VGG16 model at http://www.robots.ox.ac.uk/~vgg/research/very_deep/.
Put this model in CF-Caffe/models/

Shadow Detection

1. Enter the examples/DSC/DSC_detection/
   Modify the image path in DSC.prototxt.

2. Run

   sh train.sh

Shadow Removal

1. Color compensation mechanism:
   Enter the /data/SRD/ or /data/ISTD/.
   Run color_transfer_function.m in Matlab.

2. Transfer the images into the LAB color sapce and do the data argumentation:
   Enter the /data/SRD/ or /data/ISTD/.
   Run ToLab.m and data_argument.m in Matlab.

3. Enter the examples/DSC/DSC_removal_SRD/ or examples/DSC/DSC_removal_ISTD/.
   Modify the image path in DSC.prototxt.

4. Run

   sh train.sh

Bibtex

If you find our work, code, or results useful, please consider citing our papers as follows:

@InProceedings{Hu_2018_CVPR,      
  author = {Hu, Xiaowei and Zhu, Lei and Fu, Chi-Wing and Qin, Jing and Heng, Pheng-Ann},      
  title = {Direction-Aware Spatial Context Features for Shadow Detection},      
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},      
  pages={7454--7462},        
  year = {2018}
}

@article{hu2020direction,   
  author = {Hu, Xiaowei and Fu, Chi-Wing and Zhu, Lei and Qin, Jing and Heng, Pheng-Ann},    
  title = {Direction-Aware Spatial Context Features for Shadow Detection and Removal},    
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},    
  volume={42},             
  number={11},        
  pages={2795--2808},           
  year  = {2020}                                  
}