Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip building frontend when APP_ENV=dev #4843

Closed
wants to merge 3 commits into from
Closed

Skip building frontend when APP_ENV=dev #4843

wants to merge 3 commits into from

Conversation

koooge
Copy link
Contributor

@koooge koooge commented Apr 28, 2020

What type of PR is this? (check all applicable)

  • Refactor
  • Feature
  • Bug Fix
  • New Query Runner (Data Source)
  • New Alert Destination
  • Other

Description

Skip building frontend materials when APP_ENV=dev with BuildKit

$ make compose_build
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose build
redis uses an image, skipping
postgres uses an image, skipping
email uses an image, skipping
WARNING: Native build is an experimental feature and could change at any time
Building server
[+] Building 325.0s (17/17) FINISHED                                                                                                                                    
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 35B                                                                                                                                   0.0s
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 2.62kB                                                                                                                             0.0s
 => [internal] load metadata for docker.io/library/python:3.7-slim                                                                                                 0.0s
 => CACHED [server 1/6] FROM docker.io/library/python:3.7-slim                                                                                                     0.0s
 => CACHED https://databricks.com/wp-content/uploads/2.6.10.1010-2/SimbaSparkODBC-2.6.10.1010-2-Debian-64bit.zip                                                   0.0s
 => [internal] load build context                                                                                                                                  0.3s
 => => transferring context: 94.95kB                                                                                                                               0.2s
 => [server 2/6] RUN useradd --create-home redash                                                                                                                  0.5s
 => [server 3/6] RUN apt-get update &&   apt-get install -y     curl     gnupg     build-essential     pwgen     libffi-dev     sudo     git-core     wget     l  55.5s
 => [server 4/6] ADD https://databricks.com/wp-content/uploads/2.6.10.1010-2/SimbaSparkODBC-2.6.10.1010-2-Debian-64bit.zip /tmp/simba_odbc.zip                     0.1s 
 => [server 5/6] RUN unzip /tmp/simba_odbc.zip -d /tmp/   && dpkg -i /tmp/SimbaSparkODBC-2.6.10.1010-2-Debian-64bit/simbaspark_2.6.10.1010-2_amd64.deb   && echo   1.2s 
 => [server 6/6] WORKDIR /app                                                                                                                                      0.0s 
 => [dev-server 1/4] COPY requirements.txt requirements_bundles.txt requirements_dev.txt requirements_all_ds.txt ./                                                0.1s 
 => [dev-server 2/4] RUN pip install -r requirements.txt -r requirements_dev.txt                                                                                 109.3s
 => [dev-server 3/4] RUN if [ "x$skip_ds_deps" = "x" ] ; then pip install -r requirements_all_ds.txt ; else echo "Skipping pip install -r requirements_all_ds.t  147.3s
 => [dev-server 4/4] COPY . /app                                                                                                                                   0.5s
 => [stage-4 1/1] RUN chown -R redash /app                                                                                                                         4.3s
 => exporting to image                                                                                                                                             6.0s
 => => exporting layers                                                                                                                                            6.0s
 => => writing image sha256:b74dea256833205521af4536f5b1e003839a0a62b4c887a12eabc070d02f81cf                                                                       0.0s
 => => naming to docker.io/library/redash_server                                                                                                                   0.0s
Successfully built b74dea256833205521af4536f5b1e003839a0a62b4c887a12eabc070d02f81cf

Requirements:
docker 18.09.3+
docker-compose 1.25.1+

⚠️ You can build redash without BuildKit. But the build may be slower without BuildKit because Docker tries to build both dev and prod steps in Dockerfile.

Related Tickets & Documents

Yet another #4292

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

@arikfr
Copy link
Member

arikfr commented Apr 29, 2020

Thank you for giving this another try! Can you elaborate a bit on what's build kit?

@koooge
Copy link
Contributor Author

koooge commented Apr 29, 2020

BuildKit is a feature of Docker that makes building convenient and fast.
https://docs.docker.com/develop/develop-images/build_enhancements/

I thought this way would be better than to separate Dockerfile #4292 in the view of maintainability.

Signed-off-by: koooge <koooooge@gmail.com>
Signed-off-by: koooge <koooooge@gmail.com>
@koooge koooge marked this pull request as ready for review May 1, 2020 00:42
@koooge koooge changed the title [wip] Skip building frontend when APP_ENV=dev Skip building frontend when APP_ENV=dev May 1, 2020
Signed-off-by: koooge <koooooge@gmail.com>
Comment on lines +42 to +49
- run:
name: Build Front End Assets for some testcases # TODO: Fix testcases and remove this step
command: |
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs
npm ci
npm run build
docker cp ./client/dist $(docker-compose ps -q redash):/app/client/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some backend unit testcases depend on frontend.

COPY . /app
COPY --from=frontend-builder /frontend/client/dist /app/client/dist

FROM ${APP_ENV}-server
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prod-server or dev-server

@arikfr
Copy link
Member

arikfr commented May 11, 2020

What do you think of #4879? I realized that we can do it with an ARG, which felt simpler as it's supported across more versions of Docker.

The main issue with BUILDKIT is if it's not supported, Docker will build more images than before resulting in slower performance. Also it required more changes. But curious about what you think.

Thanks.

@koooge
Copy link
Contributor Author

koooge commented May 12, 2020

Superseded by #4879

@koooge koooge closed this May 12, 2020
@koooge koooge deleted the dev_container branch May 12, 2020 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants