PyTorch codes for "Satellite Video Super-resolution via Multi-Scale Deformable Convolution Alignment and Temporal Grouping Projection", IEEE Transactions on Geoscience and Remote Sensing (TGRS), 2022.
Authors: Yi Xiao, Xin Su, Qiangqiang Yuan*, Denghong Liu, Huanfeng Shen, and Liangpei Zhang
Wuhan University
As a new earth observation tool, satellite video has been widely used in remote-sensing field for dynamic analysis. Video super-resolution (VSR) technique has thus attracted increasing attention due to its improvement to spatial resolution of satellite video. However, the difficulty of remote-sensing image alignment and the low efficiency of spatial–temporal information fusion make poor generalization of the conventional VSR methods applied to satellite videos. In this article, a novel fusion strategy of temporal grouping projection and an accurate alignment module are proposed for satellite VSR. First, we propose a deformable convolution alignment module with a multiscale residual block to alleviate the alignment difficulties caused by scarce motion and various scales of moving objects in remote-sensing images. Second, a temporal grouping projection fusion strategy is proposed, which can reduce the complexity of projection and make the spatial features of reference frames play a continuous guiding role in spatial–temporal information fusion. Finally, a temporal attention module is designed to adaptively learn the different contributions of temporal information extracted from each group. Extensive experiments on Jilin-1 satellite video demonstrate that our method is superior to current state-of-the-art VSR methods.
git clone https://github.com/XY-boy/MSDTGP.git
- CUDA 10.0
- pytorch 1.x
- build DCNv2
Please download our dataset in
- Baidu Netdisk Jilin-189 Code:31ct
- Zenodo:
You can also train your dataset following the directory sturture below!
trainset--
| train--
| LR4x---
| 000.png
| ···.png
| 099.png
| GT---
| Bicubic4x---
testset--
| eval--
| LR4x---
| 000.png
| ···.png
| 099.png
| GT---
| Bicubic4x---
python main.py
python eval.py
If you have any questions or suggestions, feel free to contact me. 😊
Email: xiao_yi@whu.edu.cn; xy574475@gmail.com
If you find our work helpful in your research, please consider citing it. Thank you! 😊😊
@ARTICLE{xiao2022msdtgp,
author={Xiao, Yi and Su, Xin and Yuan, Qiangqiang and Liu, Denghong and Shen, Huanfeng and Zhang, Liangpei},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection},
year={2022},
volume={60},
number={},
pages={1-19},
doi={10.1109/TGRS.2021.3107352}
}
Our work is built upon RBPN and TDAN.
Thanks to the author for the source code !