Table of Contents:
- Monitoring Urban Changes in Mariupol/Ukraine
- Pre-processing with rsdtlib
- Training/Validation Datasets
- Training
- Trained Models
- Integration into Floreon
- Other Use Cases
- Paper and Citation
- Contact
- Acknowledgments
- License
- Disclaimer
- Ethical Statement
This repository demonstrates the transferred ERCNN-DRS to monitor urban changes in Mariupol/Ukraine in 2022/23.
Prediction values (probabilities) over all detected urban changes with a half-year sliding window, on a static background (for reference only) and superimposed in colors according to the legend (Nov. 2021 - Nov. 2023):
This video is located in this repository here, without YouTube compression artifacts.
We continue to update the visualizations as time progresses. The ones from the paper (see below) are available as well.
Maximum values over all detected urban changes with a sliding window, superimposed in red (Nov. 2021 - Dec. 2023):
The GeoTIFF file is here.
Prediction values (probabilities) over all detected urban changes with a half-year sliding window (Nov. 2021 - Dec. 2023):
Video file is located here.
The older counterparts from the paper and following months can be found here.
Note: These visualizations were created with 93x93 tiles and an 8 pixel overlap (dead zone of 4 in rsdtlib) to provide a seamless presentation without the issues of tile borders.
The urban changes were detected and monitored with a transferred version of the pre-trained ERCNN-DRS model for Sentinel 1 & 2 missions.
The windowed observations were pre-processed with rsdtlib. This library downloads all Sentinel 1 & 2 observations from Sentinel Hub and pre-processes the observations for time series analysis (windowing). This library was used for training and inference. We used as time frame November 2021 up to today (December 2023), with sliding windows of six month duration.
The scripts to download and process the Sentinel 1 and Sentinel 2 observations to windowed time series are available at the rsdtlib repository in the subdirectory urban_change_monitoring_mariupol_ua.
See below for the already processed and ready-to-go training/validation datasets.
Thanks to the data providers, we can make available the training/validation datasets on Google Drive.
Note: The training/validation datasets are TFRecord files, with one file for each tile and each tile containing all windows from 2017-2020. Two features are availble, with one describing the time series of observations for each window and a label. The label is the synthetic ground truth which is not used for transfer learning! Instead labels need to be loaded separately from folder training/numpy_ground_truth.
ATTENTION, these files are large!
Extract the tar balls V[1-4].tar in the respective subdirectories ./training/V1/, ./training/V2/, ./training/V3/, and ./training/V4/.
Versions V[1-4] are using different subsets of tiles for training, with valiation tiles being disjunct.
Execute the training script training/train.py. It is recommended to use the NVIDIA GPU Cloud Tensorflow container docker://nvcr.io/nvidia/tensorflow:22.02-tf2-py3 and at least eight GPUs with a total of 320 GB of memory (8x40 GB).
Change the variable exp to the version to train, e.g. exp = "V1".
We provide all trained models:
- Pre-trained model:
- Transferred models:
V1_transfer_116.h5: First transferred model from partial cross-validation (epoch 116)V2_transfer_113.h5: Second transferred model from partial cross-validation (epoch 113)V3_transfer_146.h5: Third transferred model from partial cross-validation (epoch 146)V4_transfer_109.h5: Fourth transferred model from partial cross-validation (epoch 109)
The data is also available in an experimental layer in Floreon:
Set the time units to Days to scroll faster through time. Please note that the window with the nearest start time is shown in the timeline and that windows are only available in the ranges mentioned above.
-
The origin:
The basic transfer method has been used to monitor urban changes in Liège/Belgium 2017-2020. That use case is hosted as a dedicated project here.
-
Experimental: Bakhmut site, Ukraine in Floreon:
Set the time units to
Daysto scroll faster through time. Please note that the window with the nearest start time is shown in the timeline and that windows are only available in the ranges mentioned above for Mariupol, Ukraine.Note: The ERCNN-DRS was not transferred to this site but the transferred models (V1-4) for Mariupol were used.
The full paper can be found at IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.
@ARTICLE{10423108,
author={Zitzlsberger, Georg and Podhoranyi, Michal},
journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
title={Monitoring of Urban Changes With Multimodal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23},
year={2024},
volume={17},
number={},
pages={5245-5265},
doi={10.1109/JSTARS.2024.3362688},
url={https://doi.org/10.1109/JSTARS.2024.3362688}
}
Should you have any feedback or questions, please contact the main author: Georg Zitzlsberger (georg.zitzlsberger(a)vsb.cz).
This research was funded by the Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPS II) project “IT4Innovations excellence in science - LQ1602” and by the IT4Innovations Infrastructure, which is supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90140), and via the Open Access Grant Competition (OPEN-25-24 and OPEN-27-1). This work was also supported by ESA Network of Resources Initiative (ID:2923ca) to provide access to Sentinel Hub, and Airbus Pléiades.
We also would like to thank CESNET Meta Centrum for providing us access to a DGX H100 node.
This project is made available under the GNU General Public License, version 3 (GPLv3).
This work and results is active research and subject of change. They hence are provided "as is", without guarantee of correctness or liability. Use at your own risk.
Due to the ongoing Russian-Ukrainian war, the selection of locations of visual samples was done with care to minimize risks of influence and harm. To the best of our knowledge, we only selected locations and data that did not give direct insight to the ongoing war, but only documented the resulting (urban) changes. We also would like to underline that our monitoring methods used six-month windows and hence did not and shall not provide real-time information that could be used for military purposes. Our methods are optimized for inertial urban changes that manifest over longer periods.