-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: fix smoke shard total in CI #13844
Conversation
# The total number of shards. Set dynamically when length of *single* matrix variable is | ||
# computable. See https://github.community/t/get-length-of-strategy-matrix-or-get-all-matrix-options/18342 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy to change the wording here, as I assume it read like there is a solution (since the linked github discussion has an accepted solution), when really it was meant for background of the slightly different problem (multiple matrix dimensions) that isn't solved.
I guess you aren't allowed to do that with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oops! yeah, I had assumed this problem was fixed.
will need to ensure the "these jobs must pass" form is correct in our repo settings when this lands
nope https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability |
4f92c64
to
1f6a573
Compare
#13792 relied on
${{ strategy.job-total }}
for the count of total shards, but in the smoke tests thejob-total
is 6 from 3 shards * 2 versions of chrome. So we were accidentally testing half the smoke tests (shards 1/6, 2/6, and 3/6). You had to dig into the logs to see the problem, though the three minute completion times were suspicious :)Also put the shard total in the job name so it's a little more prominent (though the name is still set manually, so there's no guarantee it'll be kept in sync with what's actually running).