Workflow

Data preparation

• labels.geojson: with bounding boxes location, uid, class type (damaged or non-damaged buildings), and tiff id (2048 x 2048 tiff names)

• a folder of tif chips of the size 2048 * 2048. Every chip contains at least one bbox.

1. Convert Geotiff to tiff

2. Manual inspect and remove bad bounding boxes

• Plot bounding boxes over big tiffs with bbox uids. Manual record the uids of bad labels
• Delete uids of bad labels from geojson, form a new geojson label file

3. Train-test split

• Split images into training, validation and test data
• Split geojson file to form training label geojson file and test label geojson file

4. Get statistics of training and testing data

• number of training big chips:
• number of test big chips:
• number of training bboxes for class 1:
• number of training bboxes for class 2:
• number of test bboxes for class 1:
• number of test bboxes for class 2:

5. Chip big tifs into smaller tifs

• Chip 2048 tif into smaller tifs
• discard black small tifs and their bboxes on the fly for train and val
• automatically removing cloud-covered bboxes on the fly for train and val
• remove small chips that do not contain any bboxes

6. augmentation

• Augment damaged buildings various combinations of augmentation techiniques

7. convert into TF-record

• convert training and validation images and bboxes to TF-record formats.
• create .pbtxt file as label map (1: damaged buildings, 2: non-damaged buildings)

8. Training

https://github.com/tensorflow/models/blob/1f562faacb0f0a083be9bcd7f9e72d94bb23e310/research/object_detection/g3doc/running_locally.md

9. Inference and scoring

• Create TF record for test data
• Generate detection results using tensorflow inference code. The results are written in a TF record file
• Compute metrics using TF record file with tensorflow code, see here: https://github.com/tensorflow/models/blob/1f562faacb0f0a083be9bcd7f9e72d94bb23e310/research/object_detection/g3doc/oid_inference_and_evaluation.md

10. Visualization

• Piece together detections on small tiffs to make 2048 x 2048 tiff files.
• Plot ground truth and predictions side by side in one image.