-
Notifications
You must be signed in to change notification settings - Fork 69
/
cloudless_lab_notebook.txt
26 lines (19 loc) · 3.35 KB
/
cloudless_lab_notebook.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Cloudless lab notebook with consolidated details on various runs:
Note: We have 65 different days of the same San Francisco Bay Area location via RapidEye
Note: Training/validation split for all runs is 80%/20%
Note: Best final result ended up being output0005
output0003: PlanetLab images, ~700 labelled images with bounding boxes, 62.5% accuracy, see PlanetLab Run folder for bounding box images, see output0003.statistics.txt for detailed statistics and graphs. Model for this trained at logs/output0003.bvlc_alexnet_finetuned.caffemodel
Representative images for this before and after with inference bounding boxes:
logs/output0003.image_with_clouds.jpg
logs/output0003.image_with_clouds_bounding_box.png
output0005: RapidEye images, ~4000 labelled images with bounding boxes, 89.77% accuracy, see RapidEye Run folder for bounding box images, see output0005.statistics.txt for detailed statistics and graphs. Trained model for this in logs/output0005.bvlc_alexnet_finetuned.caffemodel
Some commands used to generate RapidEye bounding boxes experimenting with different hyperparameters:
./localization.py -i rapideye_cloud_5.jpg --classes cloud-classes.txt --config ../../caffe_model/bvlc_alexnet/bounding_box.prototxt --weights ../../../logs/latest_bvlc_alexnet_finetuned.caffemodel --ks 5 --max_regions 600 --only_for_class 1 --platform gpu --threshold 8.5
./localization.py -i rapideye_cloud_4.jpg --classes cloud-classes.txt --config ../../caffe_model/bvlc_alexnet/bounding_box.prototxt --weights ../../../logs/latest_bvlc_alexnet_finetuned.caffemodel --ks 5 --max_regions 600 --only_for_class 1 --platform gpu --threshold 9.0
./localization.py -i rapideye_cloud_3.jpg --classes cloud-classes.txt --config ../../caffe_model/bvlc_alexnet/bounding_box.prototxt --weights ../../../logs/latest_bvlc_alexnet_finetuned.caffemodel --ks 5 --max_regions 600 --only_for_class 1 --platform gpu --threshold 8.0
./localization.py -i rapideye_cloud_2.jpg --classes cloud-classes.txt --config ../../caffe_model/bvlc_alexnet/bounding_box.prototxt --weights ../../../logs/latest_bvlc_alexnet_finetuned.caffemodel --ks 1 --max_regions 600 --only_for_class 1 --platform gpu --threshold 9.0
output0006: *006 is where I only do manual augmentation for rotations, leaving cropping and mirroring to Caffe's standard transformations. This also has full correct data and the 4000 RapidEye labelled images. Overall this didn't do better than *005 though!
Best threshold: 10%
output0007: Things in *007 have full manual data augmentation (cropping, mirroring, rotating). Due to the amount of data glitches happened getting the leveldb to save and some training images were lost, so it might not be exactly the amount given in the statistics file. Accuracy ended up only being 70%. More of an experiment. The proper way to do this would have been as a run-time custom Caffe Python data-layer. The experiment in *006 seemed to show that rotation data augmentations weren't really more effective then the standard Caffe-given mirrors and crops so this path abandoned.
output0008: Re-running same as output0005, but through latest pipeline code. The accuracy is only about ~82% though, I think because I did a test_iter of 50 instead of 1000. The threshold was also higher, about 32%.
output0009: Same as output0008, but with test_iter 1000. Accuracy is ~86% with threshold ~28.0%. output0005 is still a bit better, going with that one.