Two-stage Spatial-Frequency Joint Learning for Large-Factor Remote Sensing Image Super-Resolution

Official Pytorch implementation of the paper "Two-stage Spatial-Frequency Joint Learning for Large-Factor Remote Sensing Image Super-Resolution".
Jiarui Wang, Yuting Lu, Shunzhou Wang, Binglu Wang*, Xiaoxu Wang, and Teng Long

Abstract

Super-resolution neural networks have recently achieved great progress in restoring high-quality remote sensing images at low zoom-in magnitude. However, these networks often struggle with challenges like shape distortion and blurring effects due to the severe absence of structure and texture details in large-factor remote sensing image super-resolution. Addressing these challenges, we propose a novel Two-Stage Spatial-Frequency Joint Learning Network (TSFNet). TSFNet innovatively merges insights from both spatial and frequency domains, enabling a progressive refinement of super-resolution results from coarse to fine. Specifically, different from existing frequency feature extraction approaches, we design a novel amplitude-guided-phase adaptive filter module to explicitly disentangle and sequentially recover both the global common image degradation and specific structural degradation in the frequency domain. Additionally, we introduce the cross-stage feature fusion design to enhance feature representation and selectively propagate useful information from stage one to stage two. Quantitative and qualitative experimental results demonstrate that our proposed method surpasses state-of-the-art techniques in large-factor remote sensing image super-resolution.

Datasets generation

Download the UCMerced dataset[Baidu Drive][Google Drive], AID dataset[Baidu Drive] [Google Drive]. and NWPU-RESISC45 dataset[Baidu Drive] [Google Drive], These three datasets are for training and testing which have been split them into train/val/test data. In every dataset, the original images would be taken as the HR references and the corresponding LR images are generated by bicubic down-sample.

Notebly: The first two datasets do not contain x8 and x16 LR images. Please generate the LR images by the following codes in Matlab:

cd data_scripts
run generate_mod_LR_bic.m in Matlab

Train

# UCMerced 
python demo_train.py --dataset=UCMerced 
# NWPU RESISC45
python demo_train.py --dataset=NWPU
# AID
python demo_train.py --dataset=AID --lr=2e-4

Download the trained model in this paper [[Baidu Drive](https://pan.baidu.com/s/13cPlUxW7tuxYr0FVuXlkdQ?pwd=wza7], [Google Drive].

The train/val data pathes are set in data/init.py

Test & Evaluation

Compute the evaluated results in term of PSNR, SSIM. The test data path and the save path can be edited in demo_test.py

..