DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops

PyTorch implementation and pretrained models for DINO-MC and DINO-TP. For details, please see our paper.

Pretrained models

Our models are pre-trained on SeCo-100K, and we list their k-nn and linear probing accuracy on EuroSAT. You can download the full checkpoint of pre-trained model with training infomation as well as weights of teacher and student networks used for the downstream tasks.

model	arch	params	k-nn	linear	download
DINO-MC	ViT-S/8	21M	93.41%	94.09%	pre-trained ckpt
DINO-MC	ResNet-50	23M	93.94%	95.59%	pre-trained ckpt
DINO-MC	WRN-50	69M	95.65%	95.70%	pre-trained ckpt
DINO-MC	Swin-t	28M	93.22%	90.54%	pre-trained ckpt
DINO-TP	ViT-S/8	21M	93.15%	93.89%	pre-trained ckpt
DINO-TP	ResNet-50	23M	79.05%	86.70%	pre-trained ckpt
DINO-TP	WRN-50	69M	86.27%	88.15%	pre-trained ckpt
DINO-TP	Swin-t	28M	92.83%	91.94%	pre-trained ckpt

Training

Our codes refer to DINO and SeCo. If you want to pre-train DINO-MC based on your datasets:

python run_with_submitit.py --nodes 1 --ngpus 4 --arch vit_small --data_mode mc --data_path /path/to/dataset/train --output_dir /path/to/saving_dir

Fine-tuning

After pre-training, you can evaluate the representations on three end-to-end fine-tuning downstream tasks.

model	arch	EuroSAT	download
DINO	ViT-S/8	97.98%	EuroSAT
DINO-MC	ViT-S/8	98.15%	EuroSAT
DINO-MC	Swin-tiny	98.43%	EuroSAT
DINO-MC	ResNet-50	98.69%	EuroSAT
DINO-MC	WRN-50-2	98.78%	EuroSAT

model	arch	BigEarthNet-10%	download	BigEarthNet	download
DINO	ResNet-50	79.67%	BigEarthNet-10% ckpt	85.38%	BigEarthNet ckpt
DINO-TP	ResNet-50	80.10%	BigEarthNet-10% ckpt	85.20%	BigEarthNet ckpt
DINO-MC	ResNet-50	82.55%	BigEarthNet-10% ckpt	86.86%	BigEarthNet ckpt
DINO-MC	WRN-50-2	82.67%	BigEarthNet-10% ckpt	87.22%	BigEarthNet ckpt
DINO-MC	Swin-tiny	83.84%	BigEarthNet-10% ckpt	88.75%	BigEarthNet ckpt
DINO-MC	ViT-S/8	84.20%	BigEarthNet-10% ckpt	88.69%	BigEarthNet ckpt

model	arch	Pre.	Rec.	F1	download
DINO	ResNet-50	57.37	44.21	49.53	OSCD
DINO-MC	ResNet-50	51.94	54.04	52.46	OSCD
DINO-TP	ResNet-50	51.10	49.03	49.74	OSCD
DINO	WRN-50-2	53.58	52.28	52.41	OSCD
DINO-MC	WRN-50-2	49.99	56.81	52.70	OSCD
DINO-TP	WRN-50-2	55.77	47.30	50.61	OSCD

Citation

If you find this repository useful, please consider giving a star ⭐ and citation:

@misc{wanyan2023dinomc,
      title={DINO-MC: Self-supervised Contrastive Learning for Remote Sensing Imagery with Multi-sized Local Crops}, 
      author={Xinye Wanyan and Sachith Seneviratne and Shuchang Shen and Michael Kirley},
      year={2023},
      eprint={2303.06670},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}