Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: fix smoke shard total in CI #13844

Merged
merged 3 commits into from
Apr 14, 2022
Merged

tests: fix smoke shard total in CI #13844

merged 3 commits into from
Apr 14, 2022

Conversation

brendankenny
Copy link
Member

#13792 relied on ${{ strategy.job-total }} for the count of total shards, but in the smoke tests the job-total is 6 from 3 shards * 2 versions of chrome. So we were accidentally testing half the smoke tests (shards 1/6, 2/6, and 3/6). You had to dig into the logs to see the problem, though the three minute completion times were suspicious :)

github actions log showing `--shard=1/6` CLI flag

Also put the shard total in the job name so it's a little more prominent (though the name is still set manually, so there's no guarantee it'll be kept in sync with what's actually running).

@brendankenny brendankenny requested a review from a team as a code owner April 11, 2022 22:39
@brendankenny brendankenny requested review from adamraine and removed request for a team April 11, 2022 22:39
Comment on lines +21 to +22
# The total number of shards. Set dynamically when length of *single* matrix variable is
# computable. See https://github.community/t/get-length-of-strategy-matrix-or-get-all-matrix-options/18342
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to change the wording here, as I assume it read like there is a solution (since the linked github discussion has an accepted solution), when really it was meant for background of the slightly different problem (multiple matrix dimensions) that isn't solved.

@brendankenny
Copy link
Member Author

Also put the shard total in the job name so it's a little more prominent (though the name is still set manually, so there's no guarantee it'll be kept in sync with what's actually running).

I guess you aren't allowed to do that with env variables?

Copy link
Collaborator

@connorjclark connorjclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops! yeah, I had assumed this problem was fixed.

will need to ensure the "these jobs must pass" form is correct in our repo settings when this lands

@brendankenny
Copy link
Member Author

Also put the shard total in the job name so it's a little more prominent (though the name is still set manually, so there's no guarantee it'll be kept in sync with what's actually running).

I guess you aren't allowed to do that with env variables?

nope

https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants