-
Notifications
You must be signed in to change notification settings - Fork 416
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bench: run branch_creation_many at 500, seeded #6959
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. I'd suggest adding a temporary pytest.mark.repeat()
on the test to check for flakiness before merging.
2570 tests run: 2437 passed, 0 failed, 133 skipped (full report)Flaky tests (2)Postgres 16Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
8e2913d at 2024-03-07T13:40:10.407Z :recycle: |
You mean the seeding could make it flaky or what? Note we don't even run these without |
Drafted to avoid the e2e while waiting the new token. |
Well, this PR took long enough to get going, might as well continue with the added metrics. |
Example run output for me:
|
it is unnecessary because we know we are on the second run already.
We have a benchmark for creating a lot of branches, but it does random things, and the branch count is not what we is the largest maximum we aim to support. If this PR would stabilize the benchmark total duration it means that there are some structures which are very much slower than others. Then we should add a seed-outputting variant to help find and reproduce such cases.
Additionally, record for the benchmark: