Object detection (#15)

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Delete models/slim directory

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Delete models/marco directory

* Delete models/cognitive_planning directory

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Delete models directory

* Updated the pathing for main.

* Configured Git-lfs to work with models

* Updates to the model to run mounting config pipeline

* updated the pipeline to including mounting model

* Updated with the new Detecto object detection pipeline.

* Added in the pipeline for object detection.

* Added unit tests for new mounting configuration functions.

* Updated more of the unit tests for training the classifier model.

* Updating panel detection methods.

* updated the unit tests with assertions

* adding in example files and the mounting config model.

* Updated the import.

* debugging unit testing

* forgot to save unit tests...

* Fixed import error.

* added back in missing layer dict for segmentation model.

* Debugging unit tests.

* Updated the Sphinx documentation.

* Blocked off detecto package unit test.

* Updated the examples to fix memory error.

* Updated from CUDA to CPU.

* Updated to remove memory error.

* Updated unit tests--debugging.

* Updated unit testing.

* Added the overlapping boxes functions.

* Updated scripts to fix memory error.

* Updated the object detection routine with NMS.

* update imports

* updated NMS threshold.

* Updated unit tests.

* more unit test debugging.

* added back in method for testing detecto training.

* Pep8 formatted the package.

* fixed last of pep8 formatting issues

* added git workflows for testing flake8 + requirements.

* removed unnecessary files

* fixed pep8 issues in setup.py file.

* fixed requirement cv2 version

* more opencv updates.

* more debug of cv2

* removed cx_freeze dependency.

* fixed the routine to pass unit tests.

* debugging unit tests.

* upgraded sphinx to stop read the docs error.

* clean up doc strings

* specified jinja2 version.

* documentation updates

* removed parameters from the site pipeline routine.

* update the jupyter notebook.

* Cleanup for PR--remove all caps variables.

* update the unit tests with not all caps.

* Update panel_segmentation/panel_train.py

Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>

* changed docstring formatting to remove () around passed/returned parameters.

* updated the requirements +setup scripts

* Update panel_segmentation/panel_detection.py

Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>

* added pytest fixtures for the panel segmentation unit tests

* updated docstring mismatch

* updates the api.rst to include missing functions.

* added torch and torchvision dependencies to package.

* Update the open/close logic to write only when open

* updated the valueerror pytest check to isolate the generateSatelliteImage() function.

* updated pep8 issues + added the packages as needed.

* update the unit test assertions.

* updated the changelog.rst file with pyfunc formatting.

* repush updates based on @kanderso-nrel's recommendations.

* updated the changelog for push

Co-authored-by: Christopher Campos <chris.acampos@yahoo.com>
Co-authored-by: Perry <kperry@nrel.gov>
Co-authored-by: Perry <kperry>
Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>

.gitattributes

@@ -1,3 +1,5 @@
 pvpa/_version.py export-subst
 panel_segmentation/_version.py export-subst
 *.h5 filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+

.github/workflows/flake8.yaml

@@ -0,0 +1,34 @@
+name: flake8
+
+# since we're only checking PR diffs, only trigger on the pull_request event type.
+# note: the pull_request event happens on opened, reopened, and synchronize activities;
+# synchronize includes commits being pushed to the PR branch.
+# https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#pull_request
+on: [pull_request]
+
+jobs:
+  lint:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+
+    steps:
+    - uses: actions/checkout@v2
+      with:
+        fetch-depth: 0  # fetch all branches, needed so we can diff against the target branch
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install flake8
+      run: |
+        python -m pip install --upgrade pip
+        pip install flake8
+    - name: List changed files
+      run: |
+        git diff --compact-summary "origin/$GITHUB_BASE_REF"
+    - name: Run linter on changed files
+      run: |
+        git diff "origin/$GITHUB_BASE_REF" -- "*.py" | flake8 . --config=.flake8 --diff --count --statistics --show-source

.github/workflows/requirements.yaml

@@ -0,0 +1,23 @@
+name: requirements
+
+on: [pull_request, push]
+
+jobs:
+  requirements:
+
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+
+    runs-on: ${{ matrix.os }}
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.7
+    - name: Install notebook environment
+      run: |
+        python -m pip install --upgrade pip wheel
+        pip install -r requirements.txt

.gitignore

@@ -10,3 +10,6 @@ config.yaml
 # sphinx
 build
 generated
+
+venv/
+.idea/

MANIFEST.in

@@ -2,5 +2,6 @@ include versioneer.py
 include panel_segmentation/_version.py
 include panel_segmentation/VGG16_classification_model.h5
 include panel_segmentation/VGG16Net_ConvTranpose_complete.h5
+include panel_segmentation/object_detection_model.pth
 include panel_segmentation/examples/*
 include panel_segmentation/tests/*

README.md

@@ -12,6 +12,10 @@ classification model (VGG16_classification_model.h5).
 image on a pixel-by-pixel basis, using an image segmentation model (VGG16Net_ConvTranpose_complete.h5).
 - Using connected components clustering, isolate individual solar arrays in the masked image.
 - Perform azimuth estimation on each solar array cluster in the masked image.
+- Using an object detection model (Faster R-CNN Resnet 50 trained via transfer learning), detect
+and classify mounting type and configuration of solar installations in satellite imagery. This includes
+classification of fixed tilt, single-axis trackers, and double-axis trackers, as well as the rooftop, 
+ground, and carport mounting configurations.
 
 To install Panel-Segmentation, perform the following steps:
 

panel_segmentation/__init__.py

@@ -3,4 +3,4 @@
 
 from ._version import get_versions
 __version__ = get_versions()['version']
-del get_versions
+del get_versions

panel_segmentation/examples/Test_Mount_Object_Detection/annotations.csv

@@ -0,0 +1,47 @@
+filename,width,height,class,xmin,ymin,xmax,ymax,image_id
+1000.png,640,640,carport-fixed,457,35,640,49,0
+1000.png,640,640,carport-fixed,472,66,640,81,0
+1001.png,640,640,carport-fixed,329,1,354,47,1
+1001.png,640,640,carport-fixed,382,262,579,287,1
+1002.png,640,640,carport-fixed,219,197,245,270,2
+1002.png,640,640,carport-fixed,258,194,284,267,2
+1002.png,640,640,carport-fixed,300,193,326,266,2
+1002.png,640,640,carport-fixed,343,193,369,266,2
+1002.png,640,640,carport-fixed,235,162,385,181,2
+1002.png,640,640,carport-fixed,80,162,144,181,2
+1002.png,640,640,carport-fixed,335,137,384,154,2
+1002.png,640,640,carport-fixed,333,111,382,128,2
+1002.png,640,640,carport-fixed,348,90,383,107,2
+1003.png,640,640,carport-fixed,1,376,116,404,3
+1003.png,640,640,carport-fixed,216,370,268,423,3
+1004.png,640,640,carport-fixed,114,231,137,275,4
+1004.png,640,640,carport-fixed,131,285,156,339,4
+1004.png,640,640,carport-fixed,146,232,201,315,4
+1004.png,640,640,carport-fixed,189,214,244,297,4
+1004.png,640,640,carport-fixed,244,186,302,215,4
+1004.png,640,640,carport-fixed,219,143,292,187,4
+1004.png,640,640,carport-fixed,130,175,203,219,4
+1004.png,640,640,ground-fixed,348,157,544,372,4
+1005.png,640,640,carport-fixed,424,432,461,554,5
+1005.png,640,640,carport-fixed,562,366,640,387,5
+1006.png,640,640,carport-fixed,69,481,158,570,6
+1006.png,640,640,carport-fixed,330,389,461,518,6
+1007.png,640,640,carport-fixed,214,445,320,472,7
+1007.png,640,640,carport-fixed,210,409,289,433,7
+1007.png,640,640,carport-fixed,207,326,316,348,7
+1008.png,640,640,carport-fixed,158,366,266,413,8
+1008.png,640,640,carport-fixed,281,357,326,417,8
+1008.png,640,640,carport-fixed,204,80,296,98,8
+1008.png,640,640,carport-fixed,199,41,296,66,8
+1008.png,640,640,carport-fixed,310,41,347,67,8
+1008.png,640,640,carport-fixed,311,82,345,97,8
+1009.png,640,640,carport-fixed,135,326,199,368,9
+1009.png,640,640,carport-fixed,411,526,467,598,9
+1009.png,640,640,carport-fixed,251,183,301,207,9
+1009.png,640,640,carport-fixed,150,183,223,207,9
+1009.png,640,640,carport-fixed,4,183,77,207,9
+1010.png,640,640,carport-fixed,132,250,172,274,10
+1010.png,640,640,carport-fixed,59,394,99,418,10
+1010.png,640,640,carport-fixed,146,467,175,490,10
+1010.png,640,640,carport-fixed,180,372,220,396,10
+1010.png,640,640,carport-fixed,145,416,231,464,10

panel_segmentation/examples/Test_Mount_Object_Detection/annotations/1000.xml

@@ -0,0 +1,38 @@
+<annotation>
+	<folder>master_sat_imgs</folder>
+	<filename>1000.png</filename>
+	<path>1000.png</path>
+	<source>
+		<database>Unknown</database>
+	</source>
+	<size>
+		<width>640</width>
+		<height>640</height>
+		<depth>3</depth>
+	</size>
+	<segmented>0</segmented>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>1</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>457</xmin>
+			<ymin>35</ymin>
+			<xmax>640</xmax>
+			<ymax>49</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>1</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>472</xmin>
+			<ymin>66</ymin>
+			<xmax>640</xmax>
+			<ymax>81</ymax>
+		</bndbox>
+	</object>
+</annotation>

panel_segmentation/examples/Test_Mount_Object_Detection/annotations/1001.xml

@@ -0,0 +1,38 @@
+<annotation>
+	<folder>images</folder>
+	<filename>1001.png</filename>
+	<path>C:\Users\kperry\Documents\source\repos\Panel-Segmentation\panel_segmentation\examples\Test_Mount_Object_Detection\images\1001.png</path>
+	<source>
+		<database>Unknown</database>
+	</source>
+	<size>
+		<width>640</width>
+		<height>640</height>
+		<depth>3</depth>
+	</size>
+	<segmented>0</segmented>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>1</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>329</xmin>
+			<ymin>1</ymin>
+			<xmax>354</xmax>
+			<ymax>47</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>382</xmin>
+			<ymin>262</ymin>
+			<xmax>579</xmax>
+			<ymax>287</ymax>
+		</bndbox>
+	</object>
+</annotation>

panel_segmentation/examples/Test_Mount_Object_Detection/annotations/1002.xml

@@ -0,0 +1,122 @@
+<annotation>
+	<folder>images</folder>
+	<filename>1002.png</filename>
+	<path>C:\Users\kperry\Documents\source\repos\Panel-Segmentation\panel_segmentation\examples\Test_Mount_Object_Detection\images\1002.png</path>
+	<source>
+		<database>Unknown</database>
+	</source>
+	<size>
+		<width>640</width>
+		<height>640</height>
+		<depth>3</depth>
+	</size>
+	<segmented>0</segmented>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>219</xmin>
+			<ymin>197</ymin>
+			<xmax>245</xmax>
+			<ymax>270</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>258</xmin>
+			<ymin>194</ymin>
+			<xmax>284</xmax>
+			<ymax>267</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>300</xmin>
+			<ymin>193</ymin>
+			<xmax>326</xmax>
+			<ymax>266</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>343</xmin>
+			<ymin>193</ymin>
+			<xmax>369</xmax>
+			<ymax>266</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>235</xmin>
+			<ymin>162</ymin>
+			<xmax>385</xmax>
+			<ymax>181</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>80</xmin>
+			<ymin>162</ymin>
+			<xmax>144</xmax>
+			<ymax>181</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>335</xmin>
+			<ymin>137</ymin>
+			<xmax>384</xmax>
+			<ymax>154</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>333</xmin>
+			<ymin>111</ymin>
+			<xmax>382</xmax>
+			<ymax>128</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>348</xmin>
+			<ymin>90</ymin>
+			<xmax>383</xmax>
+			<ymax>107</ymax>
+		</bndbox>
+	</object>
+</annotation>

panel_segmentation/examples/Test_Mount_Object_Detection/annotations/1003.xml

@@ -0,0 +1,38 @@
+<annotation>
+	<folder>master_sat_imgs</folder>
+	<filename>1003.png</filename>
+	<path>1003.png</path>
+	<source>
+		<database>Unknown</database>
+	</source>
+	<size>
+		<width>640</width>
+		<height>640</height>
+		<depth>3</depth>
+	</size>
+	<segmented>0</segmented>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>1</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>1</xmin>
+			<ymin>376</ymin>
+			<xmax>116</xmax>
+			<ymax>404</ymax>
+		</bndbox>
+	</object>
+	<object>
+		<name>carport-fixed</name>
+		<pose>Unspecified</pose>
+		<truncated>0</truncated>
+		<difficult>0</difficult>
+		<bndbox>
+			<xmin>216</xmin>
+			<ymin>370</ymin>
+			<xmax>268</xmax>
+			<ymax>423</ymax>
+		</bndbox>
+	</object>
+</annotation>