Skip to content
/ GeoSynth Public

A PyTorch implementation of "GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis"

License

Notifications You must be signed in to change notification settings

mvrl/GeoSynth

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

20 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis

This repository is the official implementation of GeoSynth [CVPRW, EarthVision, 2024]. GeoSynth is a suite of models for synthesizing satellite images with global style and image-driven layout control.

Models available in 🤗 HuggingFace diffusers:

GeoSynth: Hugging Face Model

GeoSynth-OSM: Hugging Face Model

GeoSynth-SAM: Hugging Face Model

GeoSynth-Canny: Hugging Face Model

All model ckpt files available here - Model Zoo

⏭️ Next

  • Update Gradio demo
  • Release Location-Aware GeoSynth Models to 🤗 HuggingFace
  • Release PyTorch ckpt files for all models
  • Release GeoSynth Models to 🤗 HuggingFace

🌏 Inference

Example inference using 🤗 HuggingFace pipeline:

from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
import torch
from PIL import Image

img = Image.open("osm_tile_18_42048_101323.jpeg")

controlnet = ControlNetModel.from_pretrained("MVRL/GeoSynth-OSM")

pipe = StableDiffusionControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", controlnet=controlnet)
pipe = pipe.to("cuda:0")

# generate image
generator = torch.manual_seed(10345340)
image = pipe(
    "Satellite image features a city neighborhood",
    generator=generator,
    image=img,
).images[0]

image.save("generated_city.jpg")

📍 Geo-Awareness

Our model is able to synthesize based on high-level geography of a region:

🧑‍💻 Setup and Training

Style for OSM imagery is created using MapBox. The style file can be downloaded from here. The dataset can be downloaded from here. Look at train.md for details on setting up the environment and training models on your own data.

🐨 Model Zoo

Download GeoSynth models from the given links below:

Control Location Download Url
- Link
OSM Link
SAM Link
Canny Link
- Link
OSM Link
SAM Link
Canny Link

📑 Citation

@inproceedings{sastry2024geosynth,
  title={GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis},
  author={Sastry, Srikumar and Khanal, Subash and Dhakal, Aayush and Jacobs, Nathan},
  booktitle={IEEE/ISPRS Workshop: Large Scale Computer Vision for Remote Sensing (EARTHVISION),
  year={2024}
}

🔍 Additional Links

Check out our lab website for other interesting works on geospatial understanding and mapping:

  • Multi-Modal Vision Research Lab (MVRL) - Link
  • Related Works from MVRL - Link

About

A PyTorch implementation of "GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published