- Dataset presentation
PASTIS is a benchmark dataset for panoptic and semantic segmentation of agricultural parcels from satellite time series. It contains 2,433 patches within the French metropolitan territory with panoptic annotations (instance index + semantic labelfor each pixel). Each patch is a Sentinel-2 multispectral image time series of variable lentgh.
We propose an official 5 fold split provided in the dataset's metadata, and evaluated several of the top-performing image time series networks. You are welcome to use our numbers and to submit your own entries to the leaderboard!
- Dataset in numbers
- 🔥 NEW: Radar extension (PASTIS-R)
We also propose an extended version of PASTIS which contains all radar observations of Sentinel-1
for all 2433 patches in addition to the Sentinel-2 images. For each patch, approximately 70 observations of Sentinel-1 in ascending orbit,
and 70 observations in descending orbit are added to the dataset. The PASTIS-R extension can thus be used to evaluate optical-radar fusion
methods for parcel-based classification, semantic segmentation, and panoptic segmentation.
For more details on PASTIS-R, refer to our recent paper on multi-modal fusion with attention-based models (link coming soon).
- Download
The dataset can be downloaded from zenodo in different formats:
- PASTIS (29 GB zipped) : The original PASTIS dataset for semantic and panoptic segmentation
on Sentinel-2 time series (format used for the ICCV 2021 paper).
- PASTIS-R (54 GB zipped) : The extended version with Sentinel-1 observations.
- PASTIS-R (pixel-set format) (27 GB zipped) : The PASTIS-R dataset prepared in pixel-set format
for parcel-based classification only. See this repo and
paper
for more details on this format.
- Data loading
This repository also contains a PyTorch dataset class in code/dataloader.py
that can be readily used to load data for training models on PASTIS and PASTIS-R.
For the pixel-set dataset, use the dataloader in code/dataloader_pixelset.py.
The time series contained in PASTIS have variable lengths. The code/collate.py
contains a pad_collate function that you can use in
the pytorch dataloader to temporally pad shorter sequences.
The demo.ipynb notebook shows how to use these classes and methods to load data from PASTIS.
- Metrics
A PyTorch implementation is also given in code/panoptic_metrics.py to compute
the panoptic metrics. In order to use these metrics, the model's output should contain an instance prediction
and a semantic prediction. The first one allocates an instance id to each pixel of the image,
and the latter a semantic label.
Please open an issue to submit new entries. Do mention if the work has been published and wether the code accessible for reproducibility. We require that at least a preprint is available to present the method used.
| Model name | #Params | OA | mIoU | Published |
|---|---|---|---|---|
| U-TAE | 1.1M | 83.2% | 63.1% | ✔️ link |
| Unet-3d* | 1.6M | 81.3% | 58.4% | ✔️ link |
| Unet-ConvLSTM* | 1.5M | 82.1% | 57.8% | ✔️ link |
| FPN-ConvLSTM* | 1.3M | 81.6% | 57.1% | ✔️ link |
| Models that we re-implemented ourselves are denoted with a star (*). |
| Model name | #Params | OA | mIoU | Published |
|---|---|---|---|---|
| Late Fusion (U-TAE) + Aux + TempDrop | 1.7M | 84.2% | 66.3% | ✔️ link |
| Early Fusion (U-TAE) + TempDrop | 1.6M | 83.8% | 65.9% | ✔️ link |
| Model name | #Params | SQ | RQ | PQ | Published |
|---|---|---|---|---|---|
| U-TAE + PaPs | 1.3M | 81.3 | 49.2 | 40.4 | ✔️ link |
| Model name | #Params | SQ | RQ | PQ | Published |
|---|---|---|---|---|---|
| Early Fusion (U-TAE + PaPs) + Aux + TempDrop | 1.8M | 82.2 | 50.6 | 42.0 | ✔️ link |
| Late Fusion (U-TAE + PaPs) + TempDrop | 2.4M | 81.6 | 50.5 | 41.6 | ✔️ link |
The agricultural parcels are grouped into 18 different crop classes as shown in the
table below. The backgroud class corresponds to non-agricultural land, and the void label for parcels that are mostly outside their patch.
Additional information about the dataset can be found in the documentation/pastis-documentation.pdf document.
If you use PASTIS please cite the related paper:
@article{garnot2021panoptic,
title={Panoptic Segmentation of Satellite Image Time Series
with Convolutional Temporal Attention Networks},
author={Sainte Fare Garnot, Vivien and Landrieu, Loic },
journal={ICCV},
year={2021}
}
For the PASTIS-R optical-radar fusion dataset, please also cite this paper:
@article{garnot2021mmfusion,
title={Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series},
author={Sainte Fare Garnot, Vivien and Landrieu, Loic and Chehata, Nesrine },
journal={arxiv},
year={2021}
}
-
The satellite imagery used in PASTIS was retrieved from THEIA: "Value-added data processed by the CNES for the Theia www.theia.land.fr data cluster using Copernicus data. The treatments use algorithms developed by Theia’s Scientific Expertise Centres. "
-
The annotations used in PASTIS stem from the French land parcel identification system produced by IGN, the French mapping agency.
-
This work was partly supported by ASP, the French Payment Agency.
-
We also thank Zenodo for hosting the datasets.