Improved docs (#9)

* Improved docs with detector, tracker, preproc and postproc parameters * Improved troubleshooting section of README + Small typos
Trusted-AI-Labs · Oct 28, 2022 · bb7c6f9 · bb7c6f9
1 parent 822e765
commit bb7c6f9
Show file tree

Hide file tree

Showing 18 changed files with 370 additions and 95 deletions.
diff --git a/README.md b/README.md
@@ -2,7 +2,7 @@
 
 ![Logo](img/logo.png) 
 
-[![Documentation Status](https://readthedocs.org/projects/aptitude-toolbox/badge/?version=latest)](https://aptitude-toolbox.readthedocs.io/en/latest/?badge=latest)
+![Github Stars](https://badgen.net/github/stars/Trusted-AI-Labs/APTITUDE_Toolbox?icon=github) [![Documentation Status](https://readthedocs.org/projects/aptitude-toolbox/badge/?version=latest)](https://aptitude-toolbox.readthedocs.io/en/latest/?badge=latest)
 
 ## Table of Contents
 1. [Description](#description)
@@ -17,7 +17,7 @@
 
 The APTITUDE Toolbox gathers open-source implementations of state-of-the-art algorithms for the detection and tracking of moving objects. It integrates popular frameworks such as OpenCV, Tensorflow and Pytorch and it is made easy-to-use via an all-in-one interface.
 
-Once installed, the toolbox takes a video and a config file as input and can output an annotated video as well as the result of the detection and tracking in MOT format. Preprocessing methods are available to resize, apply a mask or add borders. Also, postprocessing methods can be used to filter the results with different thresholds. The following figure illustrates the pipeline:
+Once installed, the toolbox takes a video and configuration files as input and can output an annotated video as well as the result of the detection and tracking in MOT format. Preprocessing methods are available to resize, apply a mask or add borders. Also, postprocessing methods can be used to filter the results with different thresholds. The following figure illustrates the pipeline:
 
 ![Pipeline](img/pipeline.png)
 
@@ -30,8 +30,8 @@ A video demo showing the results of the inference can be consulted [here](https:
 
 The APTITUDE toolbox can be run on CPU. However, it is highly recommended to use a high-end GPU in order to obtain real-time performance. 
 
-**Thus, to use the toolbox with GPU, you should have CUDA and cuDNN installed before proceeding to the installation. It was tested with CUDA 11.3 and cuDNN 8.3.2, using previous versions may lead to unexpected results**
-<!-- Refer to [this link]() for their installation. -->
+**Thus, to use the toolbox with GPU, you should have CUDA and cuDNN installed before proceeding to the installation. 
+It was tested with CUDA 11.3 and cuDNN 8.3.2, using previous versions may lead to unexpected results**
 
 
 <a name="installation"></a>
@@ -42,10 +42,12 @@ The APTITUDE toolbox can be run on CPU. However, it is highly recommended to use
 Since version 0.2.3, deployment via Docker is only available to TRAIL affiliates.
 Becoming a TRAIL affiliate allows you to get in touch with a community of researchers, industries and entrepreneurs in the domain of AI. 
 Learn more on [https://trail.ac/](https://trail.ac/)
+If you are affiliated to the TRAIL community, have a look at [this repository](https://github.com/Trusted-AI-Labs/APTITUDE_Toolbox_docker).
 
 ### Using Anaconda
 
-Before anything else, please install [ffmpeg](https://ffmpeg.org/download.html) on your machine
+Before anything else, please install [ffmpeg](https://ffmpeg.org/download.html) on your machine.
+Optionally, you can also install [libjpeg-turbo](https://github.com/libjpeg-turbo/libjpeg-turbo/releases) to read sequence of JPEG images faster. 
 
 If you don't want to use Docker or if you want to contribute to the project, you can clone and manually install the dependencies.
 
@@ -58,7 +60,9 @@ pip install -r requirements/requirements.txt
 pip install -e .
 ```
 
-Then, you can use the wheel file to install OpenCV, which is already built with CUDA. 
+Then, you can use the wheel file that comes with the toolbox to install OpenCV, which is already built with CUDA.
+However, it may not work with all GPU architectures, you might need to build it yourself. 
+Please see the section Troubleshooting below if it does not work as expected. 
 
 For Windows:
 ```
@@ -69,24 +73,31 @@ For Linux:
 pip install requirements/opencv_contrib_python-4.5.5.64-cp39-cp39-linux_x86_64.whl
 ```
 
+Otherwise, refer to [this link](https://pypi.org/project/opencv-python/) (manual builds section) to build it yourself.
+
 Next, enter the following commands to install PyTorch with targeting the CUDA version:
 ```
 pip install torch==1.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
 pip install torchvision==0.11.3+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
 ```
-Otherwise, refer to [this link](https://pypi.org/project/opencv-python/) (manual builds section) to build it yourself.
 
 Additionally, if you are on Linux want to use Detectron2, you can install the following dependencies:
 ```
 pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html
 ```
 
-However, if you are on Windows, you must build it from source. Refer to their [repository](https://github.com/facebookresearch/detectron2) or refer to [this issue](https://github.com/Trusted-AI-Labs/APTITUDE_Toolbox/issues/5) to have some hints on how to install on Windows.
+However, if you are on Windows, you must build it from source. 
+Refer to their [repository](https://github.com/facebookresearch/detectron2) 
+or refer to [this issue](https://github.com/Trusted-AI-Labs/APTITUDE_Toolbox/issues/5) to have some hints on how to install on Windows.
 
-To accelerate the reading of image sequences, you can install libjpeg-turbo. This way, the jpeg images will be read via this faster library instead of openCV. Refer to their [releases](https://github.com/libjpeg-turbo/libjpeg-turbo/releases) to install it on your system. It was tested with version 2.0.90. The library is supported thanks to PyTurboJPEG that is installed via `requirements.txt`
+To accelerate the reading of image sequences, you can install libjpeg-turbo. 
+This way, the jpeg images will be read via this faster library instead of openCV. 
+Refer to their [releases](https://github.com/libjpeg-turbo/libjpeg-turbo/releases) 
+to install it on your system. It was tested with version 2.0.90. The library is supported thanks to PyTurboJPEG that is installed via `requirements.txt`
 
 ### Troubleshooting 🔫
 
+#### OpenCV
 
 On Windows, in case you get the following error when importing cv2:
 
@@ -98,24 +109,47 @@ ImportError: DLL load failed: The specified module could not be found
 This might be because the path to a DLL is missing. Try to add to your path the following folders:
 ```
 <your-path>\Anaconda3\envs\apt_tb
-<your-path>\envs\apt_tb\Lib\site-packages\h5py
+<your-path>\Anaconda3\envs\apt_tb\Lib\site-packages\h5py
+```
+
+The first one is for the python39.dll, the second one is for hdf5.dll. 
+If this is not sufficient, try to use [Dependencies](https://github.com/lucasg/Dependencies) 
+to look for any other missing DLL  of `<your-path>\Anaconda3\envs\apt_tb\Lib\site-packages\cv2\python-3\cv2.pyd`.
+
+If you cannot make it work, you might need to build OpenCV yourself according to your GPU architecture.
+Still, you can install OpenCV without CUDA, using the following command:
+```
+pip install opencv-python==4.5.5.64
 ```
+Note that YOLOv4 will not be able to run on GPU.
 
-The first one is for the python39.dll, the second one is for hdf5.dll. If this is not sufficient, try to use [Dependencies](https://github.com/lucasg/Dependencies) to look for any other missing DLL  of `<your-path>\Anaconda3\envs\apt_tb\Lib\site-packages\cv2\cv2.cp37-win_amd64.pyd`.
+#### FFMPEG
+
+If you get an error while reading a video:
+
+```
+FileNotFoundError: [WinError 2] The system cannot find the file specified
+```
+
+It is probably because you installed FFMPEG, but the program is not referenced in your Windows PATH variable.
+To solve the issue, simply add the ```/bin``` of FFMPEG folder (containing 3 .exe files) in your Windows PATH environment variable.
 
 <a name="usage"></a>
 ## Usage
 
 ### Models
 
-Models trained on [MIO-TCD Localization dataset](https://tcd.miovision.com/challenge/dataset.html) are available on our [drive](https://drive.google.com/drive/folders/1di8gys9LwbLFeFTY7dLA6PqG4nX5TLSD?usp=sharing). 
+Models trained on [MIO-TCD Localization dataset](https://tcd.miovision.com/challenge/dataset.html) 
+are available on our [drive](https://drive.google.com/drive/folders/1di8gys9LwbLFeFTY7dLA6PqG4nX5TLSD?usp=sharing). 
 Feel free to try them with the APTITUDE Toolbox before training your own.
 
 We advise to create a `/models/` and a `/videos/` folder so that you can easily switch from a model/video to another. 
 
 ### Use as a Black Toolbox 🧰
 
-The toolbox comes with two "client" scripts that can be used to produce an annotated video and/or to get the results of the inference in MOT format. `tracker_client_video.py` processes a sequence of images while `tracker_client_video` processes a video file. In both cases, `clients/main.py` is the script to run. In accordance with the input type (folder of file), it calls the right script. In the following table, the parameters of this script are described. The bold lines indicate those that are mandatory.
+The toolbox comes with two "client" scripts that can be used to produce an annotated video and/or to get the results of the inference in MOT format. 
+`tracker_client_images.py` processes a sequence of images while `tracker_client_video.py` processes a video file. In both cases, `clients/main.py` is the script to run. 
+In accordance with the input type (folder of file), it calls the right script. In the following table, the parameters of this script are described. The bold lines indicate those that are mandatory.
 
 | Parameter | | Description | Default
 | --------| ------ | ----------- | ----------- |
@@ -132,14 +166,19 @@ The toolbox comes with two "client" scripts that can be used to produce an annot
 | -rs |--record_size | Dimension of the recorded video: width,height. Each image is resized just before being written. |
 | -mp |--mot_path | Path to the result of tracking in MOT format.
 | -a | --async | For video files only. Whether the video reading is asynchronous. | False
+| -l | --log_level | Level of the logger. Choose between: "NOTSET", "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL" | INFO
+
+#### Parameters of the configuration file for detector and tracker
 
-The detector and tracker configuration JSON files can be created using the *APTITUDE Toolbox Config Creator*. Simply open `config_creator/index.html` in your web browser. The *Video* fields are not needed since it is given using the above parameters.
+There was a webtool to create configuration file (available under `/config_creator/`). However, this is not up to date with the latest structure of the config files.
 
-Example config files are available in /configs/, you can try them directly after changing the paths to your own model paths.
+Instead, example of config files are available in `/configs/`, you can try them directly after changing the paths to your own model paths. 
+Additionally, `all-preproc-postproc.json` contains all the parameters for image transformation or result filtering.
+More information about all the parameters are available in the [documentation](https://aptitude-toolbox.readthedocs.io/), under "bboxes_2d_detector", "bboxes_2d_tracker", "Outputs & Postproc", "Utils & Preproc"
 
-NB: The creation of a MaskRCNN config is not (yet) available in the config creator. Example files can be provided on demand.
+#### Classes configuration file
 
-The classes JSON config file contains the name of the object classes that can be detected by your detector. It will be displayed along with the detected objects. For example: 
+The classes JSON config file contains the name of the object classes that can be detected by your detector, in the order. It will be displayed along with the detected objects. For example: 
 ```
 {
     "classes": [
@@ -182,7 +221,9 @@ YOLO & Tiny-YOLO (v2-4) inference: [OpenCV Documentation - DNN module](https://d
 
 YOLOv5 inference: [Ultralytics - YOLOv5](https://github.com/ultralytics/yolov5)
 
-MaskRCNN: [Pytorch models](https://pytorch.org/vision/stable/models.html)
+MaskRCNN/FasterRCNN: [Pytorch models](https://pytorch.org/vision/stable/models.html)
+
+Detectron2: [facebookresearch - Detectron2](https://github.com/facebookresearch/detectron2)
 
 SORT: [abewley - SORT](https://github.com/abewley/sort)
 

diff --git a/clients/tracker_client_images.py b/clients/tracker_client_images.py
@@ -112,7 +112,7 @@ def main(cfg_detect, cfg_track, cfg_classes, folder_path, frame_interval, record
 
     for image_name in tqdm(file_list_sorted):
 
-        # Check if a key was pressed but with some delay, as it resource consuming
+        # Check if a key was pressed but with some delay, as it is resource consuming
         time_update = default_timer()
         if not headless and time_update - last_update > (1/10):
             k = cv2.waitKey(1) & 0xFF
@@ -153,7 +153,7 @@ def main(cfg_detect, cfg_track, cfg_classes, folder_path, frame_interval, record
         else:
             res = tracking_manager.track(det)
             log.debug("After tracking, without frames.")
-        # Don't count the time of the first 5 iteration if we consider there is a 'warmup time'
+        # Don't count the time of the first 5 iterations if we consider there is a 'warmup time'
         if counter <= 5:
             warmup_time += default_timer() - warmup_time_start
 

diff --git a/clients/tracker_client_video.py b/clients/tracker_client_video.py
@@ -120,7 +120,7 @@ def main(cfg_detect, cfg_track, cfg_classes, video_path, frame_interval, record_
 
     while is_reading:
 
-        # Check if a key was pressed but with some delay, as it resource consuming
+        # Check if a key was pressed but with some delay, as it is resource consuming
         time_update = default_timer()
         if not headless and time_update - last_update > (1/10):
             k = cv2.waitKey(1) & 0xFF

diff --git a/configs/all-preproc-postproc.json b/configs/all-preproc-postproc.json
@@ -9,11 +9,10 @@
             "height": 416
         },
         "roi": {
-            "coords": "(0, 0), (450, 0), (450, 200), (0, 200)"
-        },
-        "resize_results": {
-            "width": 1280,
-            "height": 720
+            "_comment": "Use coords OR path, never use both!",
+
+            "coords": "(0, 0), (450, 0), (450, 200), (0, 200)",
+            "path": "/path/to/roi.jpg"
         }
     },
     "postproc": {
@@ -32,6 +31,10 @@
         "roi": {
             "path": "/path/to/roi.jpg",
             "max_outside_roi_thresh": 0.25
+        },
+        "resize_results": {
+            "width": 1280,
+            "height": 720
         }
     }
 }
diff --git a/configs/detect-BS-frame_diff.json b/configs/detect-BS-frame_diff.json
@@ -4,7 +4,10 @@
         "output_type": "bboxes2D",
         "model_type": "BackgroundSubtractor",
         "pref_implem": "frame_diff",
-        "params": {}
+        "params": {
+            "contour_thresh": 3,
+            "intensity": 50
+        }
     },
     "preproc":{
         "resize":{

diff --git a/configs/detect-BS-mean.json b/configs/detect-BS-mean.json
@@ -5,6 +5,7 @@
         "model_type": "BackgroundSubtractor",
         "pref_implem": "mean",
         "params": {
+            "contour_thresh": 3,
             "intensity": 50,
             "max_last_images": 50
         }

diff --git a/configs/detect-BS-median.json b/configs/detect-BS-median.json
@@ -0,0 +1,29 @@
+{
+    "proc":{
+        "task": "detection",
+        "output_type": "bboxes2D",
+        "model_type": "BackgroundSubtractor",
+        "pref_implem": "median",
+        "params": {
+            "contour_thresh": 3,
+            "intensity": 50,
+            "max_last_images": 50
+        }
+    },
+    "preproc":{
+        "resize":{
+            "width" : 416,
+            "height": 416
+        }
+    },
+    "postproc":{
+        "nms": {
+            "pref_implem" : "Malisiewicz",
+            "nms_thresh" : 0.45
+        },
+        "max_height": 0.5,
+        "min_height": 0.02,
+        "max_width": 0.5,
+        "min_width": 0.02
+    }
+}
diff --git a/docs/bboxes_2d_detector.rst b/docs/bboxes_2d_detector.rst
@@ -1,16 +1,11 @@
-bboxes_2d_detector
-==========================
+.. raw:: html
+
+    <h1> bboxes_2d_detector </h1>
 
 Welcome to the documentation about the detectors for Bounding Boxes 2D.
-The main packages and modules are available in the 'Quick access' below.
 
 All functions are defined in details further down the page.
 
-bboxes_2d_detector
-"""""""""""""""""""""
-
-.. automodule:: pytb.detection.bboxes.bboxes_2d_detector.bboxes_2d_detector
-
 background_subtractor
 """"""""""""""""""""""""
 

diff --git a/docs/bboxes_2d_tracker.rst b/docs/bboxes_2d_tracker.rst
@@ -1,34 +1,29 @@
-bboxes_2d_tracker
-==========================
+.. raw:: html
+
+    <h1> bboxes_2d_tracker </h1>
 
 Welcome to the documentation about the trackers for Bounding Boxes 2D.
-The main packages and modules are available in the 'Quick access' below.
 
 All functions are defined in details further down the page.
 
-bboxes_2d_tracker
-"""""""""""""""""""""
-
-.. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.bboxes_2d_tracker
-
 centroid
 """"""""""""""""""""""""
 
 .. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.mbtracker.centroid.centroid
 
-deepsort
-""""""""""""""""""""""""
-
-.. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.mbtracker.deepsort.deepsort
-
-iou
+iou & kiou
 """"""""""""""""""""""""
 
 .. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.mbtracker.iou.iou
 
+
 sort
 """"""""""""""""""""""""
 
 .. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.mbtracker.sort.sort
 
+deepsort
+""""""""""""""""""""""""
+
+.. automodule:: pytb.tracking.bboxes.bboxes_2d_tracker.mbtracker.deepsort.deepsort