Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] add workflow 'Containers' #116

Merged
merged 3 commits into from
Jul 14, 2021
Merged

Conversation

umarcor
Copy link
Collaborator

@umarcor umarcor commented Jul 10, 2021

Instead of installing the dependencies in each CI workflow and job, in this PR a 'Containers' workflow is added for building custom images to be used in this repo. Precisely, images ghcr.io/stnolting/neorv32/sim and ghcr.io/stnolting/neorv32/impl are built and pushed. Then, in other workflows, those images are used.

Workflow 'Containers' is not triggered by pushes or PRs. It is executed weekly, and it can be triggered manually.

Note that we are currently using container images already. Therefore, the only advantage of this PR for now is avoiding the installation of the RISCV toolchain. However, it sets the foundation for future enhancements, such as the ones in #110.

@umarcor
Copy link
Collaborator Author

umarcor commented Jul 10, 2021

Note that CI fails because images were not pushed to ghcr.io/stnolting/neorv32 (since I don't have permissions to do that, and a PR would never be allowed to do it). However, I tested it in my fork using ghcr.io/umarcor/neorv32:

@umarcor umarcor marked this pull request as draft July 11, 2021 02:11
@stnolting
Copy link
Owner

I like this concept! However, I actually don't line the thing with installing the GCC toolchain. There should be a better approach like using the "official" nightly releases from https://github.com/riscv/riscv-gnu-toolchain/releases. I know we have discussed this already and I know there are some multilib issues... Anyway, I just wanted to say again, that I don't like this "curl Stephan's GCC" concept 😄

However, the container workflow is a good idea and it might be very useful for future modules (even though I am still overwhelmed by this and I honestly cannot say that I understand all of it 😄). 👍

Honestly, I did not look into it thoroughly. It was surprising for me that this bug showed up when working on the tools, so I just did the minimal changes for working around it. Before fixing it, I think we should understand why it is not failing in the current CI, but it failed when I tried to use this PR.

Let's clear that before merging.

@umarcor
Copy link
Collaborator Author

umarcor commented Jul 13, 2021

I like this concept! However, I actually don't line the thing with installing the GCC toolchain. There should be a better approach like using the "official" nightly releases from https://github.com/riscv/riscv-gnu-toolchain/releases. I know we have discussed this already and I know there are some multilib issues... Anyway, I just wanted to say again, that I don't like this "curl Stephan's GCC" concept 😄

I agree, however, it is unrelated to this PR. Here, I moved the same code that is used currently into a container. I did not change the functionality/procedures at all. This is precisely meant to make modifications easier. After this PR is merged, changing it in the dockerfile will suffice for all the (non-windows) workflows to use it.

However, the container workflow is a good idea and it might be very useful for future modules (even though I am still overwhelmed by this and I honestly cannot say that I understand all of it 😄). 👍

The main idea is: instead of installing the dependencies each time, use containers for that. We create container images once a week only (so, we download and install the dependencies once a week). All other workflow runs will just retrieve the container image and execute the tasks inside. If you review the modifications to the workflows, I just removed all the setup/installation steps, and then used uses: docker:// for running the remaining steps inside an specific container.

Let's clear that before merging.

See #126.

@umarcor
Copy link
Collaborator Author

umarcor commented Jul 13, 2021

I rebased this PR, and I reorganised the content. There are now three commits:

  • Add workflow and dockerfiles.
  • Use container in workflow Implementation.
  • Use container in workflow Processor.

I did not use the container in workflow riscv-arch-test due to the serious performance issues discussed in #119.

@stnolting, I think this is ready to merge. It will allow to rebase the pydoit branch. Meanwhile, we can investigate what's happening with the performance of those tests.

@umarcor umarcor marked this pull request as ready for review July 13, 2021 16:37
@stnolting
Copy link
Owner

@stnolting, I think this is ready to merge. It will allow to rebase the pydoit branch. Meanwhile, we can investigate what's happening with the performance of those tests.

Sorry, somehow this got lost in my feed... 😅

However, thanks for the contribution! ❤️

@stnolting stnolting merged commit 245e701 into stnolting:master Jul 14, 2021
@umarcor umarcor deleted the ci-containers branch July 14, 2021 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants