-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(docker): introduce src-imported
stage
#4712
Conversation
Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp>
We are waiting for:
|
no-cuda was succeeded but cuda was failed...
|
@xmfcx Do you remember if put the fix for "No space left on device" for the cuda workflow? |
This is something new.
There is 53GB space, which should be plenty. But the job also took |
https://github.com/autowarefoundation/autoware/actions/runs/8921431056 The last time this ran, it took about 2.5 hours. |
First of all I'm going to speed up the CI build with cache and then come back here. |
@youtalk If you |
@oguzkaganozt Oh, it's good point of view. #4656 might be better approach. The PR calls both of However we need to the difference of the packages between
I'm investigating. Please wait a while. |
Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp>
Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp>
This reverts commit ebff10f.
By generating the install package lists and copy the list only on the @oguzkaganozt Please review again. |
src-imported
stage
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change LGTM.
It would make the Dockerfile a bit complex as downside, but it think it's still manageable.
@oguzkaganozt You are also making big changes in another PR so I would like to know if you have any concerns before merging.
Thanks @youtalk now it's looking really good |
Why not adding This wouldn't be simpler? |
@doganulus It's not cache. We need to manage the source code change to rerun |
@youtalk If you run It is better to use the container's own mechanisms than rolling out our custom caches whenever possible. I think it is possible. Something like that: ARG AUTOWARE_VERSION=20240521
ENV AUTOWARE_VERSION=${AUTOWARE_VERSION}
...
RUN --mount=type=cache,target=/src/autoware,id=/src/autoware/${AUTOWARE_VERSION} \
vcs import ... |
* introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734) * reuse load-env.yaml to load env * update base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(docker): introduce `src-imported` stage (autowarefoundation#4712) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refine order and remove /autoware/src at last * fix hadolint * remove tar files Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * shell: bash Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734) * reuse load-env.yaml to load env * update base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * add --data-path option Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * run download_artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove todo Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp>
* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734) * reuse load-env.yaml to load env * update base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(docker): introduce `src-imported` stage (autowarefoundation#4712) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): install minimum packages in `base` stage (autowarefoundation#4751) * install ssh only and copy only once Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * install gosu Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix dir Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * revert ansible Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore: add discussion templates for general category (autowarefoundation#4764) Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> * feat(ci,docker): change `vcs import` to `COPY src` (autowarefoundation#4738) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refine order and remove /autoware/src at last * fix hadolint * remove tar files Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * shell: bash Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker,ansible): download artifacts at runtime (autowarefoundation#4771) * feat(ci): enable Docker build cache to 5 times faster container build (autowarefoundation#4730) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * enable docker build cache Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * add no-cuda/cuda suffix Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * add --data-path option Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * run download_artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove todo Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * revert run Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4772) * fix: add download url of yolox with semantic segmentation header (autowarefoundation#4012) Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add error handling for held packages during rmw installation (autowarefoundation#4781) * chore: add error handling for held packages during apt installation Signed-off-by: Naophis <nao1288stusj@gmail.com> * feat: Add check for held ROS packages during installation Signed-off-by: Naophis <nao1288stusj@gmail.com> --------- Signed-off-by: Naophis <nao1288stusj@gmail.com> * refactor(docker,ci): rename `autoware-openadk` to `autoware` and `adkit` to `openadkit` (autowarefoundation#4785) * rename adkit to openadkit Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * rename autoware-openadkit to autoware Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix hcl path Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(ci): disable `Upload Artifacts` on `docker-build-and-push` action (autowarefoundation#4789) * remove upload artifact Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove build and push Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778) * mount apt cache Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * reorder Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove upload artifact Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove build and push Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore: update code owners (autowarefoundation#4797) Signed-off-by: M. Fatih Cırıt <mfc@leodrive.ai> * ci(pre-commit-ansible): autoupdate (autowarefoundation#4676) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * ci: pre-commit autoupdate (autowarefoundation#4670) [pre-commit.ci] pre-commit autoupdate updates: - [github.com/igorshubovych/markdownlint-cli: v0.39.0 → v0.41.0](igorshubovych/markdownlint-cli@v0.39.0...v0.41.0) - [github.com/AleksaC/hadolint-py: v2.12.1-beta → v2.12.1b3](AleksaC/hadolint-py@v2.12.1-beta...v2.12.1b3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * ci(pre-commit): autoupdate (autowarefoundation#4662) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * ci(pre-commit-optional): autoupdate (autowarefoundation#4766) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * fix(ci): run `vsc import` on `build-main` workflows (autowarefoundation#4799) Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Naophis <nao1288stusj@gmail.com> Signed-off-by: M. Fatih Cırıt <mfc@leodrive.ai> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ryohsuke Mitsudome <43976834+mitsudome-r@users.noreply.github.com> Co-authored-by: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Co-authored-by: Naophis <nao1288stusj@gmail.com> Co-authored-by: M. Fatih Cırıt <xmfcx@users.noreply.github.com> Co-authored-by: awf-autoware-bot[bot] <94889083+awf-autoware-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734) * reuse load-env.yaml to load env * update base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix base_image Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(docker): introduce `src-imported` stage (autowarefoundation#4712) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): install minimum packages in `base` stage (autowarefoundation#4751) * install ssh only and copy only once Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * install gosu Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix dir Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * revert ansible Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore: add discussion templates for general category (autowarefoundation#4764) Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> * feat(ci,docker): change `vcs import` to `COPY src` (autowarefoundation#4738) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refine order and remove /autoware/src at last * fix hadolint * remove tar files Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * shell: bash Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker,ansible): download artifacts at runtime (autowarefoundation#4771) * feat(ci): enable Docker build cache to 5 times faster container build (autowarefoundation#4730) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update * vcs import outside dockerfile Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * enable docker build cache Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * add no-cuda/cuda suffix Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove --download-artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * add --data-path option Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * run download_artifacts Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove todo Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * revert run Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4772) * fix: add download url of yolox with semantic segmentation header (autowarefoundation#4012) Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> * chore: add error handling for held packages during rmw installation (autowarefoundation#4781) * chore: add error handling for held packages during apt installation Signed-off-by: Naophis <nao1288stusj@gmail.com> * feat: Add check for held ROS packages during installation Signed-off-by: Naophis <nao1288stusj@gmail.com> --------- Signed-off-by: Naophis <nao1288stusj@gmail.com> * refactor(docker,ci): rename `autoware-openadk` to `autoware` and `adkit` to `openadkit` (autowarefoundation#4785) * rename adkit to openadkit Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * rename autoware-openadkit to autoware Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix hcl path Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(ci): disable `Upload Artifacts` on `docker-build-and-push` action (autowarefoundation#4789) * remove upload artifact Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove build and push Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778) * mount apt cache Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * reorder Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove upload artifact Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * remove build and push Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore: update code owners (autowarefoundation#4797) Signed-off-by: M. Fatih Cırıt <mfc@leodrive.ai> * ci(pre-commit-ansible): autoupdate (autowarefoundation#4676) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * ci: pre-commit autoupdate (autowarefoundation#4670) [pre-commit.ci] pre-commit autoupdate updates: - [github.com/igorshubovych/markdownlint-cli: v0.39.0 → v0.41.0](igorshubovych/markdownlint-cli@v0.39.0...v0.41.0) - [github.com/AleksaC/hadolint-py: v2.12.1-beta → v2.12.1b3](AleksaC/hadolint-py@v2.12.1-beta...v2.12.1b3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * ci(pre-commit): autoupdate (autowarefoundation#4662) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * ci(pre-commit-optional): autoupdate (autowarefoundation#4766) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> * fix(ci): run `vsc import` on `build-main` workflows (autowarefoundation#4799) Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4800) cache ccache again Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore: hold check of plotjuggler-ros package and display warning if held (autowarefoundation#4805) * chore: Hold check of plotjuggler-ros package and display warning if held Signed-off-by: Naophis <nao1288stusj@gmail.com> * chore: fix condition for checking held ROS package in dev_tools role Signed-off-by: Naophis <nao1288stusj@gmail.com> * style(pre-commit): autofix * chore: Update dev_tools role to fix condition for checking held ROS package Signed-off-by: Naophis <nao1288stusj@gmail.com> --------- Signed-off-by: Naophis <nao1288stusj@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * feat(docker): revert "cache apt archives to the Docker build cache" (autowarefoundation#4793) Revert "feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)" Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(ci): add a workflow for DCO check to avoid DCO bot system failure (autowarefoundation#4825) Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> * feat(ci): add CPU architecture suffixes to the Docker build cache (autowarefoundation#4826) * add arch prefix Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * rename Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * use lib_dir Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * change suffix Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(simulator.repos): use autoware_msgs_support branch for scenario_simulator (autowarefoundation#4823) Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> Co-authored-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * feat(ansible): add lidar_transfusion artifacts (autowarefoundation#4835) * refactor(docker): fix colcon `--cmake-args` (autowarefoundation#4828) fix colcon cmake-args Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(ci): use platform arg to replace raw string (autowarefoundation#4844) use platform arg to replace raw string Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * refactor(ci): install `vcstool` and run `vcs import` on `docker-build-and-push.yaml` (autowarefoundation#4845) * refactor docker-build-and-push Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * fix shell Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> Co-authored-by: oguzkaganozt <oguzkaganozt@gmail.com> * fix platform Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> Signed-off-by: Ryohsuke Mitsudome <ryohsuke.mitsudome@tier4.jp> Signed-off-by: badai-nguyen <dai.nguyen@tier4.jp> Signed-off-by: Naophis <nao1288stusj@gmail.com> Signed-off-by: M. Fatih Cırıt <mfc@leodrive.ai> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ryohsuke Mitsudome <43976834+mitsudome-r@users.noreply.github.com> Co-authored-by: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Co-authored-by: Naophis <nao1288stusj@gmail.com> Co-authored-by: M. Fatih Cırıt <xmfcx@users.noreply.github.com> Co-authored-by: awf-autoware-bot[bot] <94889083+awf-autoware-bot[bot]@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Amadeusz Szymko <amadeuszszymko@gmail.com> Co-authored-by: oguzkaganozt <oguzkaganozt@gmail.com>
…4712) * introduce src-imported stage Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * empty Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * generate install package lists * append || true Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * cat packages Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719) * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * wip Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp> * Update Dockerfile * Update Dockerfile * install pacmod * rename * Revert "install pacmod" This reverts commit ebff10f. * install pacmod3 * register pamod3 repos * ignore hadolint * revert * run set-dev-env.sh * pip install * copy * rosdep update --------- Signed-off-by: Yutaka Kondo <yutaka.kondo@youtalk.jp>
Description
This is the resubmission of #4709 to push the upstream
src-imported
branch.This PR adds the
src-imported
stage to the top stage of theDockerfile
.This allows the
base
andsrc-imported
stages to be processed in parallel, reducing the download latency of thevcs import
command. It also eliminates the need to run thevcs import
command again in theruntime
stage.The trial in my local environment speeded up the
docker build
by about 5 minutes.This PR does not cut out
rosdep install
, based on the reflection of the previous PR:#4656 (comment)
Tests performed
Not applicable.
Effects on system behavior
Not applicable.
Interface changes
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.
In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.
Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.
After all checkboxes are checked, anyone who has write access can merge the PR.