Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(meta): make complete barrier non-async in worker loop #15109

Merged
merged 25 commits into from
Mar 5, 2024
Merged

feat(meta): make complete barrier non-async in worker loop #15109

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ffeeb4e
feat(meta): make complete barrier non-async in worker loop
wenym1 Feb 9, 2024
9ec1655
make dylint happy
wenym1 Feb 18, 2024
232a28a
Merge branch 'main' into yiming/non-async-complete-barrier
wenym1 Feb 18, 2024
6b85b4f
refactor(meta): track finished create mv job in tracker
wenym1 Feb 18, 2024
5b0c7c1
Merge branch 'main' into yiming/non-async-complete-barrier
wenym1 Feb 18, 2024
c7dfcb2
Merge branch 'main' into yiming/finished-mv-in-tracker
wenym1 Feb 19, 2024
3e885e9
move code
wenym1 Feb 19, 2024
b02a7b4
minor
wenym1 Feb 19, 2024
9055c21
Merge branch 'main' into yiming/non-async-complete-barrier
wenym1 Feb 20, 2024
2f28b42
Merge branch 'main' into yiming/finished-mv-in-tracker
wenym1 Feb 20, 2024
abff967
Merge branch 'yiming/finished-mv-in-tracker' into yiming/non-async-co…
wenym1 Feb 20, 2024
5dd19ff
Merge branch 'main' into yiming/finished-mv-in-tracker
wenym1 Feb 23, 2024
800b7fb
Merge branch 'yiming/finished-mv-in-tracker' into yiming/non-async-co…
wenym1 Feb 23, 2024
77e14b8
add comment
wenym1 Feb 23, 2024
5cf1b1e
reduce diff
wenym1 Feb 23, 2024
a00a500
continue to complete collected barrier in recovery
wenym1 Feb 26, 2024
81d85e3
use btreemap as queue
wenym1 Feb 26, 2024
b4fc60a
increase timeout
wenym1 Feb 26, 2024
b501998
fix assert
wenym1 Feb 26, 2024
78aa155
fix assert
wenym1 Feb 26, 2024
34db788
reduce kill rate
wenym1 Feb 27, 2024
4e45e43
Merge branch 'main' into yiming/non-async-complete-barrier
wenym1 Feb 29, 2024
f97e6a9
fix comment
wenym1 Mar 3, 2024
fbdfc28
Merge branch 'main' into yiming/non-async-complete-barrier
wenym1 Mar 4, 2024
66f916e
break on join err
wenym1 Mar 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/workflows/pull-request.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,7 +519,7 @@ steps:
retry: *auto-retry

- label: "recovery test (deterministic simulation)"
command: "TEST_NUM=8 KILL_RATE=0.5 BACKGROUND_DDL_RATE=0.0 ci/scripts/deterministic-recovery-test.sh"
command: "TEST_NUM=8 KILL_RATE=0.4 BACKGROUND_DDL_RATE=0.0 ci/scripts/deterministic-recovery-test.sh"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask why we need to change this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this PR, handling new command will not be blocked by called on the async complete_barrier, which makes the command be handled more timely. Once a command is started to be handled, a failure will fail the command, and then the test will retry the ddl. This increases the time taken to finish the test. Reduce the kill rate to reduce the time taken to finish the test.

if: |
!(build.pull_request.labels includes "ci/skip-ci") && build.env("CI_STEPS") == null
|| build.pull_request.labels includes "ci/run-recovery-test-deterministic-simulation"
Expand Down
Loading
Loading