This project belongs to the Trends.Earth project.
This repo implements the API used by the Trends.Earth plugin and web interfaces. It implements the Scripts, Users and Executions management.
Check out the other parts of the Trends.Earth project:
- The Command Line Interface. It allows to create and test custom scripts locally. It also can be used to publish the scripts to the Trends.Earth Environment (Trends.Earth CLI)
- The Trends.Earth Core Environment used for executing scripts in Trends.Earth
- A web app to explore and manage the API entities (Trends.Earth UI)
You need to install Docker in your machine if you haven't already (Docker)
- Docker is used in development and production environment
- The API is coded in Python 3.6
- It uses Flask to expose the API Endpoints and handle the HTTP requests
- It also uses SQLAlchemy as ORM (PostgreSQL)
- Celery is used to manage the background tasks (Redis)
- In production mode, the API will be deployed using Gunicorn
Follow the next steps to set up the development environment in your machine or on a cloud server
- Clone the repo and navigate to the folder
git clone https://github.com/conservationinternational/trends.earth-api
cd trends.earth-api
- Build the docker image:
docker compose -f docker-compose.staging.yml build
- Start the services:
docker compose -f docker-compose.staging.yml up
- To stop the services:
docker compose -f docker-compose.staging.yml down
- Clone the repo and navigate to the folder
git clone https://github.com/conservationinternational/trends.earth-api
cd trends.earth-api
- Build the docker image:
docker build -t 172.40.1.52:5000/trendsearth-api-staging .
docker push 172.40.1.52:5000/trendsearth-api-staging
-
Start a stack running on docker swarm
docker stack deploy -c docker-compose.staging.yml api-staging
-
Once the services are up:
-
To check the services are running:
docker stack ps api-staging
-
To check the logs for the services:
docker logs -f api_manager-staging
docker logs -f api_worker-staging
docker logs -f redis-staging
-
-
To remove the stack:
docker stack rm api-staging
- Startup a maintenance container
docker compose -f docker-compose.admin.yml up
To run in the background (so you can connect to container via another process), add
-d
:
docker compose -f docker-compose.admin.yml build
docker compose -f docker-compose.admin.yml up -d
- Run whatever needs to be done (db migration, etc.). For example
docker exec -it trendsearth-api-admin-1 /bin/bash
flask db migrate
flask db upgrade
exit
- Shut it down
docker compose -f docker-compose.admin.yml down
-
Add a
nginx-certbot.env
file specifying theCERTBOT_EMAIL
to use -
Startup the nginx container in the background
docker compose -f docker-compose-nginx.yml up -d
-
Build image and push to registry
docker build -t 172.40.1.52:5000/trendsearth-api . docker push 172.40.1.52:5000/trendsearth-api
-
Start the stack
docker stack deploy -c docker-compose.prod.yml api
The API has been packed in a Python module (gefapi). It creates and exposes a WSGI application. The core functionality has been divided in three different layers or submodules (Routes, Services and Models).
There are also some generic submodules that manage the request validations, HTTP errors and the background tasks manager.
id: <UUID>
name: <String>
slug: <String>, unique
created_at: <Date>
user_id: <UUID>
status: <String>
logs: <- [ScriptLog]
executions: <- [Execution]
public: <Boolean>
cpu_reservation: <Integer>
cpu_limit: <Integer>
memory_reservation: <Integer>
memory_limit: <Integer>
id: <UUID>
start_date: <Date>
end_date: <Date>
status: <String>
progress: <Integer>
params: <Dict>
results: <Dict>
logs: <- [ExecutionLog]
script_id: <- Script
user_id: <UUID>
is_plugin_execution: <Boolean>
deleted: <Boolean>
id: <UUID>
created_at: <Date>
email: <String>, unique
password: <String>, encrypted
role: <String>
is_plugin_user: <Boolean>
is_in_mailing_list: <Boolean>
scripts: <- [Script]
executions: <- [Execution]
deleted: <Boolean>
GET: /api/v1/script
GET: /api/v1/script/<script>
POST: /api/v1/script
PATCH: /api/v1/script/<script>
DELETE: /api/v1/script/<script>
GET: /api/v1/script/<script>/log
GET: /api/v1/script/<script>/run
GET: /api/v1/execution
GET: /api/v1/execution/<execution>
PATCH: /api/v1/execution/<execution>
GET: /api/v1/execution/<execution>/log
POST: /api/v1/execution/<execution>/log
GET: /api/v1/user
GET: /api/v1/user/<user>
GET: /api/v1/user/me
POST: /api/v1/user
PATCH: /api/v1/user/<user>
PATCH: /api/v1/user/me
DELETE: /api/v1/user/<user>
DELETE: /api/v1/user/me
POST: /api/v1/user/<user>/recover-password
POST: /auth
POST: /email