drone-object-detection-using-haar-cascades -- coding: utf-8 --

Variable	Content
Author	Thomas Sabbe
Mail	thomas.sabbe@student.kuleuven.be
Date	25/03/2021

---

Welcome to the repository of object detection using haar cascades. This is officially finished.

Disclaimer: some of the functions inside this repo have options to debug. This contains text output and/or visual output. This is not implemented in this README-file, but can be found when opening 'main.py'. If you wish to see a practical tutorial for each section, please visit this link

Content of this repository, in correct order of dronevision_library.py:

1. Color detection: Determine the color of an object using the KNN algorithm. There is also the option to visualize the result.
2. Drone Vision: Given two parameters (scantime and desired color), return a pixel-distance between drone and object, which has been detected with a cascade model.
3. Data augmentation: Augment positive images (images with desired object) to create a larger dataset.
4. Test accuracy of a haar-cascade model (generated with OpenCV 'train cascade'; see documentation)
5. Test accuracy of color detection, using KNN Nearest Neighbours and Kmeans clustering: Detect color of an object, and print this on an image.

How do I install and run the repository?

In order to obtain the best results, clone the whole repository with all its subfolders!

1. Clone the repository to your desired location.
2. Open the 'real' directory with PyCharm or any Python IDE.
3. Make sure following files are present and in the correct folder:

Type of function	Data needed
For object detection	cascade.xml: '../src/cascade/cascade.xml', see how to train cascade
For color detection	%color%.png: '../data/training_dataset/%color%.png' (for each color)
For data-augmentation	sample_images: Folder with images of desired object, to be augmented (is an argument so choice is arbitrary where the folder is located)
For cascade and color accuracy	N.A.

4. In the main script, call any desired method. Read the required parameters and explanation and run the script. There's NO need to open the 'VisualDetectionLibrary'. Every parameter necessary is stated in the 'main' script.
5. Check your results in the folders (if applicable) or read your outputs in the console (if applicable). Have fun!  

(optional) Detailed behaviour about content of dronevision_library.py:

1. Color Detection

Uses following libraries:

• <import csv> <import operator> <import os> <from matplotlib import pyplot as plt> <from matplotlib import colors> <import os>

Name of definition(s), input parameter(s) and return variable(s):

/ definitions

Definition name	Context
def color_controller(source_image)	Name: Main definition that returns color based upon child definitions below.
""	File usage: Creates these files if it doesn't exist yet: ../src/training.data & ../src/test.data
def training	Name: For every training-image, 'color_histogram_of_training_image' is called to create a histogram for the colors.
""	Folder usage: requires following folder: ../data/training_dataset/%color% (with color images)
def color_histogram_of_training_image	Name:Creates a .data file that contains color-cluster data. This definition is called by 'training'.
""	File usage: Creates these files if it doesn't exist yet: ../src/training.data
def color_histogram_of_test_image	Name: Creates a .data file that contains color-cluster data. This color data is extracted from a test image that is presented to the training.data model/file.
""	File usage: Creates these files if it doesn't exist yet: ../src/test.data
def knn_classifiermain	Name: Main definition for returning a prediction of the model to 'color_controller'. Uses both test.data and training.data files created in the definitions above, if they didn't exist yet.
def load_dataset	Name: Converts the test.data and training.data files to .csv vector files, in order to create training feature vectors and a test feature vector.
def response_of_neighbors	Name: Returns the amount votes of each neighbor around the color, presented in the test-image. This definition is called by 'knn_classifiermain', but uses argument 'neighbors' from 'k_nearest_neighbors' (in order to count the nearest neighbors, ofcourse...)
def k_nearest_neighbors	Name: Returns the k-amount nearest neighbors (and their distance) around the color value of the test-image. If k is set to '3', three possible clusters that are close to the value of the test-image will be returned, in an array.
def calculate_euclideandistance	Name: Used by 'k_nearest_neighbors' to determine the nearest distance between two positions (pixel-coördinates).
def rgb_to_hsv	Name: Used by a definition used in the accuracy test to convert RGB values to HSV, in order to have a better understanding of the result.
def plot_test_and_training_data	Name: Plots test and training data together, to have a better understanding of the KNN algorithm how a color is assigned.
def plotcolor	Name: Plots every pixel its color of an image in the desired color spectrum. There's a choice between 'HSV' as argument or 'RGB'
/ parameters necessary to use function

Parameter name	Context
source_image	Source: image needed for color_controller definition. This is the only parameter of the whole 'color detection'-section.
debugparam	Source: Parameter that determines whether there should be text output during the color detection process.
debugcolor	Source: Choose between HSV or RGB here to display a plot after color detection. Only active when debugparam = True. Has to be a string ("...")
print_test_and_training	Source: Prints a 3D plot of the HSV value of the observed picture inside the clusters of the training dataset.

Example to determine the color of an image of an object, as displayed in main.py:

• <color_controller(image.png, True, "HSV", True)> image.png : the image where the color needs to be determined.
• True : Will enable text output if the training dataset needs to be trained or not.
• HSV : The desired color spectrum (HSV or RGB)
• True : Will generate a 3D plot of the clustered test image its HSV value, together with the training cluster (HSV * values).

Debug options

• There is an option to enable text output to display if the .data-files need to be generated or not.
• There is also the option to enable a color plot of the training datacluster, with the value of the color of the test image plotted in the same 3D plot.

 

2. Drone Vision

MAKE SURE YOU HAVE ONLY ONE VIDEO CAPTURE DEVICE CONNECTED.

Uses following libraries:

• <import datetime> <import os> <from math import sqrt> <import math> <import cv2> <import operator> <import time>

Name of definition(s), input parameter(s) and return variable(s):

/ definitions

Definition name	Context
def average	Name: Determines the average of a list.
def addtolist	Name: Defintion to add a variable to a list.
def color_switch	Name: Switches a color (str format) to RGB values (syntax: int, int, int)
def calculate_radialdistance_theta	Name: Calculate euclidian distane and theta, given a safezone and distance between camera and object.
def visual_algorithm	Name: Main definition of the object-detection script. Main goal is to return the return variables from 'calculate_radialdistance_theta'.

/ parameters necessary to use function

Parameter name	Context
safezonesize	ONLY FOR DEBUGGING USE: Default 50, but change this if debugging is enabled.This will print a rectangle on the screen as an indication if the cross-object would be above the drone.
scantime	The time to scan for cross objects before def 'visual_algorithm' returns a radialdistance and direction theta.
desiredcolor	The color of the object you want to detect. For example; red, because your cross-object should be red and not blue.
videosource	Parameter to import a video file instead of webcam live-feed. If empty or not found (video file), it will use webcam.

Example to determine the position of an object, with respect to the center of the camera frame:

• <visual_algorithm(2, 50, "white", '../folder/videosource.avi')> This request will make the function scan 2 seconds for a white cross (in videosource.avi) and return the radial distance to that object, in case it has been found. If debug has been enabled, then, every two seconds, a white bounding box should appear on the screen output. DISCLAIMER! If no object has been found in the 2 second period, the definition will not stop until an object has been found.

Debug options

• There is an option to enable text output in the runtime box and to display a live-view image of where the object is (the bounding box of the object), for every 'scantime' seconds.

 

3. Data augmentation

Uses following libraries:

• <import numpy as npe> <import torchvision.transforms as transforms> <import torchvision.datasets as datasets> <from torchvision.utils import save_image>

Name of definition(s), input parameter(s) and return variable(s):

/ definitions

Definition name	Context
def create_transformation	Name: Main and only definition to create a series of augmented images.

/ parameters necessary to use function

Parameter name	Context
path_to_importfolder	Folder with images that need to be augmented.
path_to_exportfolder	Folder where augmented images should be stored.
start_number	Starting number for image output. Change to something else than 0 if you would perform Data Augmentation multiple times in the same folder.
repeat_variable	How many times the same image needs to be augmented. For ex.: 101 sample pictures will result into 2020 pictures, if repeat_variable is set to 20.

Attention: no return variable(s) exist for this function!

Debug options

• No debug options provided

Example to call function:

• <create_transformation(G:\PyCharmProjects\sample_images, G:\PyCharmProjects\augmented_images, 0, 10))>

 

4. & 5.: Accuracy of haar-cascade model and color detection.

NOTICE THAT THE RESULTS WILL BE STORED TO "../data/accuracy_results.txt"! This cannot be changed from the main method

Uses following libraries:

• <import cv2>

Name of definition(s), input parameter(s) and return variable(s):

/ definitions

Definition name	Context
def calculate_imagecount	Name: Counts the amount of images inside the 'sample_images'-folder.
def test_accuracy	Name: For each image in the 'sample_images'-folder, it lets the cascade determine an object (if there is an object present). Then, it determines the color of the object and prints it on an image in the 'export_images' folder.
""	Case: If no object is found, the image will be written in the 'export_images' folder as "IMGNAME+NO-OBJECT".
""	Case: If there is an object, the image will be written in in the 'export_images' folder as "IMGNAME", with the bounding box on the detected object printed on the same image. Also, a "IMGNAME+CUT" image will be written. This is a cut image of only the area that the bounding box covers.

/ parameters necessary to use function

Parameter name	Context
sample_images	Images that need to be examined by the cascade-model or the color-clustering algorithm
export_images	Export folder for the results.

Example to determine accuracy of a model using a sample dataset:

• <test_accuracy("../data/sample_images/", "../data/export_images/")>

Debug options

• No debug options provided, because this definition has already all debug functionalities by default.

END OF README.md from drone-object-detection-using-haar-cascades*