The chatbot was trained on real-world navigation assistance dialogs to give natural responses.
The 3D enviroments were reconstructed form real-world houses.
Navlead is an AI-powered assistant chatbot for navigation in 3D enviroments. The simulation, the chatbot, and their deployment online are all part of this project. (Note: this repo contains the client-side section only)
The simulation has custom 3D assets, a chat dialog box, motion controls and game logic. The chatbot has a custom ML NLP model that has a seq2seq architecture and was trained on CVDN dataset. The client-host loads the simulation and mediates its interactions with the API server. The API server authenticates the request and runs the ML NLP model.
Inspired by the rise of VR and AI, this project aims to combine both into a navigation assistance system that can run through the browser.
- Client-side
- Simulation
- Blender (3.1)
- Unity (2020.3 LTS)
- UI
- React (Next ts)
- Simulation
- Server-side (not on this repo)
- API server
- Django
- ML service
- Pytorch
- API server
For an in-depth explanation of this project, please, see Navlead's About page
- TLDR;
- Motivation
- 3D simulation
- Client-side integration
- Server-side API
- Machine Learning
- References
- Authors
See Navlead.
npm install
npm run dev
npm run build
npm run lint
https://arxiv.org/abs/1907.04957
https://github.com/mmurray/cvdn