From ea6e51b029e218b702b1740f27286923f441a3b5 Mon Sep 17 00:00:00 2001 From: Maxime Beauchemin Date: Wed, 4 Sep 2024 00:52:40 -0700 Subject: [PATCH 1/5] docs: document how docker-compose-image-tag requires -dev suffixed images Realized [here](https://github.com/apache/superset/discussions/29999#discussioncomment-10490167) that `docker-compose-image-tag.yml` requires a -dev suffixed image to work as expected. Updating default TAG to latest-dev and related docs. Side mission: removing warning related to version key in docker-compose*.* that are not used/supported anymore --- docker-compose-image-tag.yml | 3 +-- docker-compose-non-dev.yml | 1 - docs/docs/installation/docker-builds.mdx | 10 ++++++++-- docs/docs/installation/docker-compose.mdx | 6 +++++- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/docker-compose-image-tag.yml b/docker-compose-image-tag.yml index 6ac6cc36024ae..868779bcc5d88 100644 --- a/docker-compose-image-tag.yml +++ b/docker-compose-image-tag.yml @@ -21,7 +21,7 @@ # create you own docker environment file (docker/.env) with your own # unique random secure passwords and SECRET_KEY. # ----------------------------------------------------------------------- -x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest} +x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev} x-superset-depends-on: &superset-depends-on - db - redis @@ -30,7 +30,6 @@ x-superset-volumes: - ./docker:/app/docker - superset_home:/app/superset_home -version: "3.7" services: redis: image: redis:7 diff --git a/docker-compose-non-dev.yml b/docker-compose-non-dev.yml index c4aba18980e37..7aa96a84276d7 100644 --- a/docker-compose-non-dev.yml +++ b/docker-compose-non-dev.yml @@ -35,7 +35,6 @@ x-common-build: &common-build cache_from: - apache/superset-cache:3.10-slim-bookworm -version: "4.0" services: redis: image: redis:7 diff --git a/docs/docs/installation/docker-builds.mdx b/docs/docs/installation/docker-builds.mdx index 9a5d38d1d1241..a2d467b1499da 100644 --- a/docs/docs/installation/docker-builds.mdx +++ b/docs/docs/installation/docker-builds.mdx @@ -32,8 +32,14 @@ for the build, and/or base image. Here are the build presets that are exposed through the `build_docker.py` script: - `lean`: The default Docker image, including both frontend and backend. Tags -without a build_preset are lean builds, e.g., `latest`. -- `dev`: For development, with a headless browser, dev-related utilities and root access. + without a build_preset are lean builds, e.g., `latest`. `lean` builds do not contain database + drivers, meaning you need to install your own. That applies to analytics databases AND + the metadata database. You'll likely want to layer either `mysqlclient` or `psycopg2-binary` + depending on the metadata database you choose for your installation, plus the required + drivers to connect to your analytics database(s). +- `dev`: For development, with a headless browser, dev-related utilities and root access. This + includes some commonly used database drivers like `mysqlclient`, `psycopg2-binary` and + some other used for development/CI - `py311`, e.g., Py311: Similar to lean but with a different Python version (in this example, 3.11). - `ci`: For certain CI workloads. - `websocket`: For Superset clusters supporting advanced features. diff --git a/docs/docs/installation/docker-compose.mdx b/docs/docs/installation/docker-compose.mdx index 4733ba3c7a004..a6bc7d0ca1f56 100644 --- a/docs/docs/installation/docker-compose.mdx +++ b/docs/docs/installation/docker-compose.mdx @@ -38,7 +38,11 @@ Note that there are 3 major ways we support to run `docker compose`: 1. **docker-compose-image-tag.yml** where we fetch an image from docker-hub say for the `3.0.0` release for instance, and fire it up so you can try it. Here what's in the local branch has no effects on what's running, we just fetch and run - pre-built images from docker-hub + pre-built images from docker-hub. For `docker-compose` to work along with the + Postgres image it boots up, you'll want to point to a `-dev`-suffixed TAG, as in + `export TAG=4.0.0-dev` or `export TAG=3.0.0-dev`, `latest-dev` being the default. + That's because The `dev` builds happen to package the `psycopg2-binary` required to connect + to the Postgres image we fire up as part of of our docker-compose builds. More on these two approaches after setting up the requirements for either. From 14932750538102aa77f932cc7c0af90ae4df2d29 Mon Sep 17 00:00:00 2001 From: Maxime Beauchemin Date: Wed, 4 Sep 2024 00:58:27 -0700 Subject: [PATCH 2/5] tweak --- docs/docs/installation/docker-builds.mdx | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/docs/installation/docker-builds.mdx b/docs/docs/installation/docker-builds.mdx index a2d467b1499da..e2a4aee31d6f5 100644 --- a/docs/docs/installation/docker-builds.mdx +++ b/docs/docs/installation/docker-builds.mdx @@ -32,9 +32,10 @@ for the build, and/or base image. Here are the build presets that are exposed through the `build_docker.py` script: - `lean`: The default Docker image, including both frontend and backend. Tags - without a build_preset are lean builds, e.g., `latest`. `lean` builds do not contain database - drivers, meaning you need to install your own. That applies to analytics databases AND - the metadata database. You'll likely want to layer either `mysqlclient` or `psycopg2-binary` + without a build_preset are lean builds (ie: `latest`, `4.0.0`, `3.0.0`, ...). `lean` + builds do not contain database + drivers, meaning you need to install your own. That applies to analytics databases **AND + the metadata database**. You'll likely want to layer either `mysqlclient` or `psycopg2-binary` depending on the metadata database you choose for your installation, plus the required drivers to connect to your analytics database(s). - `dev`: For development, with a headless browser, dev-related utilities and root access. This From 4d7c165536d905fbb10eba1369b44cdfa79e9405 Mon Sep 17 00:00:00 2001 From: Maxime Beauchemin Date: Wed, 4 Sep 2024 08:54:17 -0700 Subject: [PATCH 3/5] Update docs/docs/installation/docker-compose.mdx Co-authored-by: Sam Firke --- docs/docs/installation/docker-compose.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/installation/docker-compose.mdx b/docs/docs/installation/docker-compose.mdx index a6bc7d0ca1f56..926c45aa21ba1 100644 --- a/docs/docs/installation/docker-compose.mdx +++ b/docs/docs/installation/docker-compose.mdx @@ -38,7 +38,7 @@ Note that there are 3 major ways we support to run `docker compose`: 1. **docker-compose-image-tag.yml** where we fetch an image from docker-hub say for the `3.0.0` release for instance, and fire it up so you can try it. Here what's in the local branch has no effects on what's running, we just fetch and run - pre-built images from docker-hub. For `docker-compose` to work along with the + pre-built images from docker-hub. For `docker compose` to work along with the Postgres image it boots up, you'll want to point to a `-dev`-suffixed TAG, as in `export TAG=4.0.0-dev` or `export TAG=3.0.0-dev`, `latest-dev` being the default. That's because The `dev` builds happen to package the `psycopg2-binary` required to connect From c2ddd3c76d2855cd7db26ee8168845384847d5b5 Mon Sep 17 00:00:00 2001 From: Maxime Beauchemin Date: Wed, 4 Sep 2024 08:54:29 -0700 Subject: [PATCH 4/5] Update docs/docs/installation/docker-compose.mdx Co-authored-by: Sam Firke --- docs/docs/installation/docker-compose.mdx | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/docs/installation/docker-compose.mdx b/docs/docs/installation/docker-compose.mdx index 926c45aa21ba1..04a89c86d6bad 100644 --- a/docs/docs/installation/docker-compose.mdx +++ b/docs/docs/installation/docker-compose.mdx @@ -42,7 +42,8 @@ Note that there are 3 major ways we support to run `docker compose`: Postgres image it boots up, you'll want to point to a `-dev`-suffixed TAG, as in `export TAG=4.0.0-dev` or `export TAG=3.0.0-dev`, `latest-dev` being the default. That's because The `dev` builds happen to package the `psycopg2-binary` required to connect - to the Postgres image we fire up as part of of our docker-compose builds. + to the Postgres database launched as part of the `docker compose` builds. +`` More on these two approaches after setting up the requirements for either. From f1117a25643ff51452e42e0a09635d123ef3a2c9 Mon Sep 17 00:00:00 2001 From: Maxime Beauchemin Date: Wed, 4 Sep 2024 08:54:41 -0700 Subject: [PATCH 5/5] Update docs/docs/installation/docker-compose.mdx Co-authored-by: Sam Firke --- docs/docs/installation/docker-compose.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/installation/docker-compose.mdx b/docs/docs/installation/docker-compose.mdx index 04a89c86d6bad..b48f53e0a6714 100644 --- a/docs/docs/installation/docker-compose.mdx +++ b/docs/docs/installation/docker-compose.mdx @@ -40,7 +40,7 @@ Note that there are 3 major ways we support to run `docker compose`: the local branch has no effects on what's running, we just fetch and run pre-built images from docker-hub. For `docker compose` to work along with the Postgres image it boots up, you'll want to point to a `-dev`-suffixed TAG, as in - `export TAG=4.0.0-dev` or `export TAG=3.0.0-dev`, `latest-dev` being the default. + `export TAG=4.0.0-dev` or `export TAG=3.0.0-dev`, with `latest-dev` being the default. That's because The `dev` builds happen to package the `psycopg2-binary` required to connect to the Postgres database launched as part of the `docker compose` builds. ``