NVIDIA Redtail project

Autonomous navigation for drones and ground vehicles using deep learning. Refer to wiki for more information on how to get started.

This project contains deep neural networks, computer vision and control code, hardware instructions and other artifacts that allow users to build a drone or a ground vehicle which can autonomously navigate through highly unstructured environments like forest trails, sidewalks, etc. Our TrailNet DNN for visual navigation is running on NVIDIA's Jetson embedded platform. Our arXiv paper describes TrailNet and other runtime modules in detail.

The project's deep neural networks (DNNs) can be trained from scratch using publicly available data. A few pre-trained DNNs are also available as a part of this project. In case you want to train TrailNet DNN from scratch, follow the steps on this page.

References and Demos

• arXiv paper
• GTC 2017 talk: slides, video
• Demo video showing 250 m autonomous flight, DNN activation and control
• Demo video showing our record making 1 kilometer autonomous flight
• Demo video showing generalization to ground vehicle control and other environments

News

• 2018-02-15: added support for the TBS discovery platform.

  • Step by step instructions on how to assemble the TBS discovery drone.
  • Instructions on how to attach and use a ZED stereo camera.
  • Detailed instructions on how to calibrate, test and fly the drone.

• 2017-10-12: added full simulation Docker image, experimental support for APM Rover and support for MAVROS v0.21+.

  • Redtail simulation Docker image contains all the components required to run full Redtail simulation in Docker. Refer to wiki for more information.
  • Experimental support for APM Rover. Refer to wiki for more information.
  • Several other changes including support for MAVROS v0.21+, updated Jetson install script and few bug fixes.

• 2017-09-07: NVIDIA Redtail project is released as an open source project.

  Redtail's AI modules allow building autonomous drones and mobile robots based on Deep Learning and NVIDIA Jetson TX1 and TX2 embedded systems. Source code, pre-trained models as well as detailed build and test instructions are released on GitHub.

• 2017-07-26: migrated code and scripts to JetPack 3.1 with TensorRT 2.1.

  TensorRT 2.1 provides significant improvements in DNN inference performance as well as new features and bug fixes. This is a breaking change which requires re-flashing Jetson with JetPack 3.1.