Switch from CircleCI to GitHub Actions #1057

HAEKADI · 2021-10-01T12:38:47Z

Technical changes

Switch CI provider from CircleCI to GitHub Actions

To Do:

Test deploy
Add PyPI password and CircleCI token to GitHub Actions

Once Openfisca-doc has migrated to GitHub Actions as well, the use of CircleCI API to update the doc should be changed as well.

HAEKADI · 2021-10-01T13:00:23Z

The test-core job fails if the build is not incorporated in the same job. I have no idea why.
When running pytest alone, the yaml tests fail with a file not found error.
Even with the build cache restored. I tried caching the entire home directory and it still fails.

The work around was to add the make build to the test-core job after restoring the env cache. It takes on average 23 extra seconds.

bonjourmauko · 2021-10-01T21:11:32Z

The test-core job fails if the build is not incorporated in the same job. I have no idea why. When running pytest alone, the yaml tests fail with a file not found error. Even with the build cache restored. I tried caching the entire home directory and it still fails.

The work around was to add the make build to the test-core job after restoring the env cache. It takes on average 23 extra seconds.

#1048

HAEKADI · 2021-10-25T07:09:35Z

Hello @maukoquiroga! Since I rebased this PR the coverage went down to zero (for both make test-core and make test)with a CoverageWarning: No data was collected. (no-data-collected) warning. Which was not the case before the rebase 🤔 Do you have any idea why that might be the case?

MattiSG

Good!

.github/has-functional-changes.sh

.github/is-version-number-acceptable.sh

.github/workflows/workflow.yml

HAEKADI · 2021-10-26T14:16:48Z

I tested deploy and the package is indeed published. But the documentation update is still problematic due to permission issues.

.

MattiSG · 2021-10-26T14:35:16Z

Thanks @HAEKADI! 😊
I updated the token to an “admin” scope, does it work now? 🙂

HAEKADI · 2021-10-26T14:50:34Z

@MattiSG Permission denied comes up still 😕

MattiSG · 2021-10-26T20:46:48Z

Mmh, I understand that I'd need to issue a personal token to get support for the API v1… I'm not so keen on that considering there is no scope limitation and I am admin on over 20 organisations. Could we consider using API v2 if it is not too costly? If that takes you more than one hour to put together, you can instead issue a personal token for yourself and send it to me in two pieces through side channels (e.g. Slack and SMS), I will then set it up 🙂

MattiSG · 2021-10-26T20:48:08Z

(as a side note, the deploy job should fail if update-doc fails, so we know…)

.github/workflows/workflow.yml

bonjourmauko · 2021-10-27T09:46:03Z

Thanks for considering me for review @HAEKADI 😊 .

I barely know Github Actions so I did what I could.

HAEKADI · 2021-10-27T12:12:24Z

Thank you for the feedback @maukoquiroga ^^

HAEKADI · 2021-10-27T12:59:08Z

@MattiSG I upgraded to CircleCI API v2 and added a personal token that I sent you.

.github/workflows/workflow.yml

MattiSG · 2021-10-27T19:48:36Z

I upgraded to CircleCI API v2 and added a personal token that I sent you

I don't think we needed both 😉
The current CIRCLECI_OPENFISCADOC_TOKEN should be valid for API v2. Could you please trigger a build and check that deployment works for you under these conditions?
If not, I added the personal token you sent me under the CIRCLECI_V1_OPENFISCADOC_TOKEN secret.
I will delete whichever one we don't need.

HAEKADI · 2021-10-28T09:17:20Z

I used both because according to the doc project tokens are not supported on API v2.
I switched back to v1.1 because a trigger is needed to use v2 and I would have to modify the openfisca doc CircleCI config file in order to do so.
As of now, the update doc step is functional with a personal token 🥳
I think this should be documented for future use. Should I do it directly with a comment in the workflow file or somewhere else?

HAEKADI · 2021-10-28T15:44:37Z

All that's left is to:

- choose the Python patch version;
- decide whether or not we're going to ignore changes to the tasks files;
- update merge rules.

MattiSG

🤩 🚢

.github/workflows/workflow.yml

MattiSG · 2021-10-29T12:15:24Z

.github/workflows/workflow.yml

+          path: ${{ env.pythonLocation }}
+          key: build-${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}-${{ github.sha }}
+      - name: Check NumPy typing against latest 3 minor versions
+        run: for i in {1..3}; do VERSION=$(${GITHUB_WORKSPACE}/.github/get-numpy-version.py prev) && pip install numpy==$VERSION && make check-types; done


Why is this implemented with a shell script instead of with the matrix strategy? 🙂

I tried using the matrix strategy but it was much more complicated than I anticipated since it is a dynamic matrix build which is problematic.

To my understanding it would require, first, creating another job, with a for loop, to get the three Numpy versions to check, store them into a JSON format and then using a matrix strategy (in the job itself) go through the list and install one by one. I was not successful in doing so, but I did not try for very long.

In my opinion, it is a complicated implementation for a simple output that is executed much more elegantly with the existing job and in 30 seconds. If however you think there is a simpler way to do it, or if it's worth the time investment, I'm happy to do it 😉

I will merge this PR for the time being. And if need be address this issue in a new one :)

IMHO this script is / will be useless once the matrix strategy is in place —if not done already.

.github/workflows/workflow.yml

CHANGELOG.md

MattiSG · 2021-10-29T12:26:10Z

I switched off all requested checks. Please notify admins on Slack to set GitHub Actions checks as mandatory once this has been merged 😃

Co-authored-by: Matti Schneider <matti.schneider@beta.gouv.fr>

HAEKADI changed the title ~~GitHub actions~~ Switch from CircleCI to GitHub actions Oct 1, 2021

HAEKADI self-assigned this Oct 1, 2021

HAEKADI changed the title ~~Switch from CircleCI to GitHub actions~~ Switch from CircleCI to GitHub Actions Oct 1, 2021

MattiSG mentioned this pull request Oct 18, 2021

Allow complete group encapsulation #1070

Merged

HAEKADI force-pushed the github-actions branch 2 times, most recently from 38db8d1 to bb30986 Compare October 20, 2021 09:47

HAEKADI requested review from MattiSG and bonjourmauko October 26, 2021 10:26

MattiSG requested changes Oct 26, 2021

View reviewed changes

MattiSG mentioned this pull request Oct 26, 2021

Remove unused Coveralls #1073

Open

bonjourmauko reviewed Oct 27, 2021

View reviewed changes

.github/workflows/workflow.yml Show resolved Hide resolved

bonjourmauko reviewed Oct 27, 2021

View reviewed changes

.github/workflows/workflow.yml Show resolved Hide resolved

MattiSG reviewed Oct 27, 2021

View reviewed changes

.github/workflows/workflow.yml Outdated Show resolved Hide resolved

HAEKADI added 5 commits October 28, 2021 08:37

Add GitHub Actions config

70440fe

Correct indent

811b621

Correct indent

dc70f89

Change quotes

698f1a8

Correct job name

e12cf4a

HAEKADI added 6 commits October 28, 2021 08:37

Test deploy for this branch

8c8393d

Update CHANGELOG and setup

2b5d199

Test updating doc on this branch

41c5f8c

Update check-for-functional-changes job comments

3bac99e

Update CircleCI API version to v2

9f9e1ef

Add missing quotation

72334e5

HAEKADI force-pushed the github-actions branch from 3324650 to 72334e5 Compare October 28, 2021 06:45

HAEKADI added 6 commits October 28, 2021 09:12

Replace CircleCI token with personnal token

884929e

Update CHANGELOG and setup

4614b6d

Replace CircleCI API v2 with v1

63951be

Update branch name

1369f48

Change content type format

bfbe880

Use master branch for doc

57afac8

HAEKADI added 3 commits October 28, 2021 17:25

Update master branch condition to deploy

66afdb4

Add CircleCI API toekn comment

2f0e3dd

Update CHANGELOG

dcd7005

HAEKADI requested a review from MattiSG October 28, 2021 15:44

MattiSG added 2 commits October 29, 2021 09:22

Remove obsolete CircleCI badge

6de7779

Make CI mention vendor-neutral

8453bff

MattiSG approved these changes Oct 29, 2021

View reviewed changes

HAEKADI and others added 4 commits November 2, 2021 08:43

Replace env variable with direct mention

ec1c377

Co-authored-by: Matti Schneider <matti.schneider@beta.gouv.fr>

Bump patch version

9b67a3c

Delete patch bump

01ab4df

Bump patch version

ff80338

HAEKADI merged commit 976c8c7 into master Nov 2, 2021

HAEKADI deleted the github-actions branch November 2, 2021 11:21

MattiSG mentioned this pull request May 5, 2022

Switch from CircleCI to GitHub Actions #1030

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch from CircleCI to GitHub Actions #1057

Switch from CircleCI to GitHub Actions #1057

HAEKADI commented Oct 1, 2021 •

edited

Loading

HAEKADI commented Oct 1, 2021 •

edited

Loading

bonjourmauko commented Oct 1, 2021

HAEKADI commented Oct 25, 2021 •

edited

Loading

MattiSG left a comment

HAEKADI commented Oct 26, 2021

MattiSG commented Oct 26, 2021

HAEKADI commented Oct 26, 2021

MattiSG commented Oct 26, 2021

MattiSG commented Oct 26, 2021

bonjourmauko commented Oct 27, 2021

HAEKADI commented Oct 27, 2021

HAEKADI commented Oct 27, 2021

MattiSG commented Oct 27, 2021

HAEKADI commented Oct 28, 2021

HAEKADI commented Oct 28, 2021 •

edited by MattiSG

Loading

MattiSG left a comment

MattiSG Oct 29, 2021

HAEKADI Nov 2, 2021

HAEKADI Nov 2, 2021

bonjourmauko Nov 2, 2021

MattiSG commented Oct 29, 2021

Switch from CircleCI to GitHub Actions #1057

Switch from CircleCI to GitHub Actions #1057

Conversation

HAEKADI commented Oct 1, 2021 • edited Loading

Technical changes

HAEKADI commented Oct 1, 2021 • edited Loading

bonjourmauko commented Oct 1, 2021

HAEKADI commented Oct 25, 2021 • edited Loading

MattiSG left a comment

Choose a reason for hiding this comment

HAEKADI commented Oct 26, 2021

MattiSG commented Oct 26, 2021

HAEKADI commented Oct 26, 2021

MattiSG commented Oct 26, 2021

MattiSG commented Oct 26, 2021

bonjourmauko commented Oct 27, 2021

HAEKADI commented Oct 27, 2021

HAEKADI commented Oct 27, 2021

MattiSG commented Oct 27, 2021

HAEKADI commented Oct 28, 2021

HAEKADI commented Oct 28, 2021 • edited by MattiSG Loading

MattiSG left a comment

Choose a reason for hiding this comment

MattiSG Oct 29, 2021

Choose a reason for hiding this comment

HAEKADI Nov 2, 2021

Choose a reason for hiding this comment

HAEKADI Nov 2, 2021

Choose a reason for hiding this comment

bonjourmauko Nov 2, 2021

Choose a reason for hiding this comment

MattiSG commented Oct 29, 2021

HAEKADI commented Oct 1, 2021 •

edited

Loading

HAEKADI commented Oct 1, 2021 •

edited

Loading

HAEKADI commented Oct 25, 2021 •

edited

Loading

HAEKADI commented Oct 28, 2021 •

edited by MattiSG

Loading