Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](query) prefer to chose tablet on alive disk #39467

Merged
merged 3 commits into from
Aug 20, 2024
Merged

[improvement](query) prefer to chose tablet on alive disk #39467

merged 3 commits into from
Aug 20, 2024

Conversation

yujun777
Copy link
Collaborator

@yujun777 yujun777 commented Aug 16, 2024 •
edited
Loading

improvement:

  1. when query, prefer to chose tablets on alive disks;
  2. when be report tablets, if report version fall behind, try report again;
  3. when be restart, it report its tablets and disks immedidately, no wait 1min;
  4. when fe handle tablet report, even if this report is stale, but if there exists other health tablets and this tablet is on bad disk, still process this tablet;

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777 yujun777 marked this pull request as draft August 16, 2024 04:40
@yujun777
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yujun777
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yujun777
Copy link
Collaborator Author

run buildall

1 similar comment
@yujun777
Copy link
Collaborator Author

run buildall

@yujun777 yujun777 marked this pull request as ready for review August 19, 2024 08:34
@yujun777
Copy link
Collaborator Author

run performance

@yujun777
Copy link
Collaborator Author

run buildall

@yujun777
Copy link
Collaborator Author

run performance

dataroaring
dataroaring approved these changes Aug 19, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yujun777
Copy link
Collaborator Author

run buildall

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38258 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f509608f04ea00205394beb6f66a8a2c8671d8ff, data reload: false

------ Round 1 ----------------------------------
q1	18160	4412	4280	4280
q2	2064	213	203	203
q3	10470	1149	1119	1119
q4	10166	827	766	766
q5	7788	2832	2826	2826
q6	265	162	160	160
q7	1022	698	649	649
q8	9389	2108	2091	2091
q9	7251	6545	6544	6544
q10	7107	2228	2223	2223
q11	474	274	277	274
q12	455	257	258	257
q13	17793	3016	3030	3016
q14	298	265	263	263
q15	580	541	528	528
q16	531	422	435	422
q17	1001	768	685	685
q18	7405	6725	6653	6653
q19	7244	1069	1059	1059
q20	716	358	343	343
q21	3930	2971	2876	2876
q22	1137	1053	1021	1021
Total cold run time: 115246 ms
Total hot run time: 38258 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4477	4301	4324	4301
q2	403	318	296	296
q3	2891	2603	2651	2603
q4	1996	1720	1737	1720
q5	5705	5673	5697	5673
q6	244	158	158	158
q7	2244	1845	1854	1845
q8	3345	3464	3473	3464
q9	8713	8598	8753	8598
q10	3562	3351	3315	3315
q11	623	533	535	533
q12	832	681	689	681
q13	16440	3202	3149	3149
q14	344	321	307	307
q15	562	529	514	514
q16	506	485	463	463
q17	1818	1558	1567	1558
q18	8019	7948	7747	7747
q19	2888	1563	1571	1563
q20	2144	1896	1876	1876
q21	14119	5259	5428	5259
q22	1215	1055	1061	1055
Total cold run time: 83090 ms
Total hot run time: 56678 ms

@yujun777
Copy link
Collaborator Author

run performance

deardeng
deardeng approved these changes Aug 20, 2024
View reviewed changes
Copy link
Contributor

@deardeng deardeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 5280c18 into apache:master Aug 20, 2024
25 of 28 checks passed
This was referenced Aug 20, 2024
Merged
Merged
dataroaring pushed a commit that referenced this pull request Aug 23, 2024
@yujun777
[improvement](query) prefer to chose tablet on alive disk #39467 (#39654
0934fbe 
)

cherry pick from #39467
dataroaring pushed a commit that referenced this pull request Aug 23, 2024
@yujun777
[improvement](query) prefer to chose tablet on alive disk #39467 (#39657
6f8b949 
)

cherry pick from #39467
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@yujun777 @dataroaring
[improvement](query) prefer to chose tablet on alive disk (#39467)
ee7786a 
improvement:
1.  when query, prefer to chose tablets on alive disks;
2. when be report tablets, if report version fall behind, try report
again;
3. when be restart, it report its tablets and disks immedidately, no
wait 1min;
4. when fe handle tablet report, even if this report is stale, but if
there exists other health tablets and this tablet is on bad disk, still
process this tablet;
@xiaokang xiaokang mentioned this pull request Sep 14, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@deardeng deardeng deardeng approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yujun777 @doris-robot @deardeng @dataroaring