Skip to content

Deep learning model running on ESP32. Custom model used with TensorFlow Lite Micro to classify captured images of flying vehicles.

License

Notifications You must be signed in to change notification settings

EliZucker/ESP32CAM-Vehicle-Classifier

Repository files navigation

Flying Vehicle Classifier (ESP32-CAM)

Overview

This project aims to create a small, energy-efficient, standalone module that can classify several types of flying vehicles.

An ESP32-CAM microcontroller running this code will take a photo from the on-board rear camera, run inference via a local pre-trained model, and display the classification result on its OLED display. This sequence is triggered by a manual button press. Currently, the project is configured specifically for pinouts on the LILYGO TTGO T-Camera ESP32 (~$15).

Features

     -  Vehicle classification model is trained from scratch, based on the MobileNetV1 architecture, and further compressed with full integer quantization of weights.

     -  Completely local inference on the ESP32 platform.

     -  Results produced from the trained deep learning model via TensorFlow Lite Micro.


Deploying

  1. Install the ESP-IDF toolchain v4.0 with their Get Started Guide.
  2. Activate the toolchain by running get_idf (if installed normally).
  3. From the project directory, run idf.py build .
  4. With the target controller plugged in, run idf.py -p <your port> flash . If you are unsure which port to use, consult the guide linked in step 1.

Training Model

See the training_model directory.

Known Issues

  • The output classes are limited to the 5 options I chose (compatible with dataset). Can be improved.

  • With the TTGO T-Camera ESP32 module, the OLED display driver seems to have I2C issues. I addressed this for now by repeatedly re-instantiating the connection. There are corresponding error messages in serial output, but functionality is fine.

  • Due to ESP32's limited DRAM, the model network cannot get any wider, even with an increased application partition size.

  • Accuracy is not great because of an inaccurate dataset (explained in training section).

License / Credit

Due to Tensorflow Lite Micro's general lack of documentation, I consulted and used some functions from the person_detection example they provide. I have included the original license (Apache 2.0) for that project with modifications by me listed.

About

Deep learning model running on ESP32. Custom model used with TensorFlow Lite Micro to classify captured images of flying vehicles.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published