Django Machine Learning Model Tracker

This is a sample project to show that you could keep track of your ML experiements using Django and Celery. We are using django celery results to store the results of the tasks. With the help of django rest framework, we are able to create a REST API to list the tasks and their results. We are also using flower to monitor the tasks.

This is just a experiment not for production use.

The project is based on the Django Celery Template. The devcontainer library files are taken from the vscode-dev-containers repository. More about devcontainers can be found here and here.

Tasks:

Experiments:

Getting started

Prerequisites

• Docker
• Docker Compose
• Python 3.8
• Django
• Django Rest Framework
• Celery
• PostgreSQL
• Redis
• Flower
• Django Celery Results

Running the project

Running with Docker Compose

1. Open the project in your terminal.
2. Run docker-compose up to start the project.

Running Django

1. Open the project in your terminal.
2. Run python manage.py runserver to start the project.

Running Celery

1. Open the project in your terminal.
2. Run celery -A config worker -l info to start the project.

How to use (wine-quality example)

Sample workflow - Running the wine-quality example

1. Visit the /tasks page.
2. Click on the wine_quality link OR visit the /tasks/wine-quality page.
3. Fill out the form, provide the alpha and l1_ratio values. Click on the POST button.
4. You should see a response with the task id and task url.
5. You can now visit the task url to see the task status and results.

Flower - Monitoring Celery

1. If you are running the project with Docker Compose, visit the localhost:5566 page.
2. On clicking on the Tasks tab, you should see the tasks that have been run.
3. On cliking on one of the tasks, you should see the task details.

Django Admin - Listing the tasks

1. If you are running the project with Docker Compose, visit the localhost:8000/admin page.
2. On clicking on the Tasks tab, you should see the tasks that have been run.
3. On cliking on one of the tasks, you should see the task details.

Built With

• Django - The web framework used for the backend
• Celery - Asynchronous task queue/job queue based on distributed message passing
• Docker - Containerization platform
• Docker Compose - Tool for defining and running multi-container Docker applications
• PostgreSQL - Open source object-relational database system
• Redis - In-memory data structure store, used as a database, cache and message broker
• Flower - Real-time monitor and web admin for Celery distributed task queue
• Django Celery Results - Django application for storing Celery task results
• Django Rest Framework - Powerful and flexible toolkit for building Web APIs

Setting up the development container

GitHub Codespaces

Follow these steps to open this sample in a Codespace:

1. Click the Code drop-down menu.
2. Click on the Codespaces tab.
3. Click Create codespace on main .

For more information on creating your codespace, visit the GitHub documentation.

VS Code Dev Containers

If you already have VS Code and Docker installed, you can click the badge above or here to get started. Clicking these links will cause VS Code to automatically install the Dev Containers extension if needed, clone the source code into a container volume, and spin up a dev container for use.

Follow these steps to open this sample in a container using the VS Code Dev Containers extension:

1. If this is your first time using a development container, please ensure your system meets the prerequisites (i.e. have Docker installed) in the getting started steps.

2. To use this repository, you can either open the repository in an isolated Docker volume:

   • Press F1 and select the Dev Containers: Try a Sample... command.
   • Choose the "Python" sample, wait for the container to start, and try things out!

     Note: Under the hood, this will use the Dev Containers: Clone Repository in Container Volume... command to clone the source code in a Docker volume instead of the local filesystem. Volumes are the preferred mechanism for persisting container data.

   Or open a locally cloned copy of the code:

   • Clone this repository to your local filesystem.
   • Press F1 and select the Dev Containers: Open Folder in Container... command.
   • Select the cloned copy of this folder, wait for the container to start, and try things out!