Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](mv) Catch exception when init sync materialized view context #37701

Merged
merged 1 commit into from
Jul 16, 2024
Merged

[improvement](mv) Catch exception when init sync materialized view context #37701

merged 1 commit into from
Jul 16, 2024

Conversation

seawinde
Copy link
Contributor

Proposed changes

Init sync materialized view context happend after analyze, If fail would cause query or insert into fail.
This is inacceptable, so catch the exception and recored the warn log.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@seawinde
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40286 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bc54f390cfbd406e26fe41c9adbbbb1f16d39489, data reload: false

------ Round 1 ----------------------------------
q1	17605	4293	4305	4293
q2	2021	193	186	186
q3	10442	1144	1098	1098
q4	10198	758	731	731
q5	7566	2702	2666	2666
q6	222	137	136	136
q7	952	600	597	597
q8	9203	2079	2067	2067
q9	8755	6532	6551	6532
q10	8882	3800	3785	3785
q11	455	239	234	234
q12	452	231	228	228
q13	17777	3001	2998	2998
q14	274	230	235	230
q15	517	480	494	480
q16	487	374	377	374
q17	952	686	721	686
q18	8026	7632	7595	7595
q19	2889	1526	1476	1476
q20	668	322	331	322
q21	4916	3230	3298	3230
q22	410	342	345	342
Total cold run time: 113669 ms
Total hot run time: 40286 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4347	4244	4215	4215
q2	383	259	256	256
q3	3036	2753	2795	2753
q4	1965	1701	1723	1701
q5	5578	5706	5536	5536
q6	239	142	138	138
q7	2231	1828	1865	1828
q8	3258	3442	3394	3394
q9	8779	8738	8915	8738
q10	4129	3936	3828	3828
q11	596	480	503	480
q12	815	641	669	641
q13	15866	3209	3164	3164
q14	323	302	276	276
q15	541	508	495	495
q16	472	454	432	432
q17	1797	1537	1529	1529
q18	8158	7946	7791	7791
q19	3237	1707	1514	1514
q20	2121	1954	1832	1832
q21	5136	5016	4804	4804
q22	599	556	543	543
Total cold run time: 73606 ms
Total hot run time: 55888 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174889 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bc54f390cfbd406e26fe41c9adbbbb1f16d39489, data reload: false

query1	930	378	362	362
query2	6463	2503	2340	2340
query3	6635	211	225	211
query4	28404	17541	17434	17434
query5	3646	494	486	486
query6	288	165	161	161
query7	4583	295	288	288
query8	324	308	299	299
query9	8667	2492	2462	2462
query10	430	300	279	279
query11	11763	10063	10044	10044
query12	114	90	83	83
query13	1653	378	375	375
query14	10288	7762	7663	7663
query15	246	191	187	187
query16	7639	331	331	331
query17	1769	594	559	559
query18	1693	298	294	294
query19	206	150	159	150
query20	89	83	83	83
query21	208	127	123	123
query22	4279	4163	4045	4045
query23	34000	34699	33942	33942
query24	11171	2898	2887	2887
query25	544	429	439	429
query26	699	153	151	151
query27	2218	281	290	281
query28	6512	2164	2184	2164
query29	902	688	710	688
query30	251	150	148	148
query31	968	763	772	763
query32	97	57	57	57
query33	750	299	314	299
query34	1011	496	504	496
query35	688	598	580	580
query36	1141	977	963	963
query37	144	87	90	87
query38	2985	2782	2792	2782
query39	918	820	824	820
query40	199	125	118	118
query41	52	50	52	50
query42	113	103	99	99
query43	589	538	543	538
query44	1151	730	737	730
query45	193	161	164	161
query46	1113	766	727	727
query47	1852	1747	1767	1747
query48	386	292	301	292
query49	833	404	416	404
query50	780	394	404	394
query51	6910	6811	6802	6802
query52	114	88	102	88
query53	362	298	287	287
query54	853	456	453	453
query55	74	73	75	73
query56	293	273	275	273
query57	1118	1013	1060	1013
query58	244	243	243	243
query59	3486	3282	3063	3063
query60	294	280	271	271
query61	98	93	98	93
query62	814	672	618	618
query63	329	293	298	293
query64	9112	2242	1625	1625
query65	3153	3091	3118	3091
query66	750	330	339	330
query67	15561	14972	15000	14972
query68	6958	598	564	564
query69	743	498	366	366
query70	1176	1146	1088	1088
query71	547	273	276	273
query72	8629	5362	5013	5013
query73	814	337	332	332
query74	6152	5533	5522	5522
query75	4821	2718	2647	2647
query76	4831	962	920	920
query77	789	312	303	303
query78	9783	9013	8987	8987
query79	2895	523	537	523
query80	1058	479	476	476
query81	589	223	221	221
query82	747	134	134	134
query83	306	175	167	167
query84	281	92	91	91
query85	1296	313	296	296
query86	453	336	321	321
query87	3279	3113	3070	3070
query88	4818	2484	2450	2450
query89	481	382	400	382
query90	1866	191	193	191
query91	131	102	106	102
query92	62	93	47	47
query93	4299	517	521	517
query94	1099	212	210	210
query95	418	324	315	315
query96	624	277	274	274
query97	3161	3003	3062	3003
query98	215	201	199	199
query99	1486	1241	1278	1241
Total cold run time: 290949 ms
Total hot run time: 174889 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.32 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bc54f390cfbd406e26fe41c9adbbbb1f16d39489, data reload: false

query1	0.04	0.04	0.04
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.50	0.48
query6	1.14	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.56	0.54	0.56
query11	0.15	0.12	0.12
query12	0.15	0.12	0.13
query13	0.59	0.58	0.58
query14	0.77	0.79	0.82
query15	0.85	0.82	0.80
query16	0.34	0.36	0.36
query17	0.96	1.00	0.99
query18	0.23	0.21	0.21
query19	1.75	1.74	1.72
query20	0.01	0.00	0.01
query21	15.39	0.81	0.68
query22	4.26	7.33	1.55
query23	18.28	1.43	1.30
query24	2.10	0.23	0.24
query25	0.16	0.08	0.09
query26	0.29	0.22	0.21
query27	0.46	0.23	0.24
query28	13.24	1.01	1.01
query29	12.58	3.27	3.30
query30	0.26	0.05	0.06
query31	2.88	0.39	0.38
query32	3.29	0.48	0.49
query33	2.86	2.91	2.89
query34	16.96	4.33	4.33
query35	4.46	4.41	4.42
query36	0.64	0.48	0.50
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.14	0.12	0.12
query41	0.10	0.05	0.04
query42	0.05	0.05	0.05
query43	0.05	0.04	0.03
Total cold run time: 109.49 s
Total hot run time: 30.32 s

@dataroaring dataroaring merged commit e478570 into apache:master Jul 16, 2024
27 of 30 checks passed
seawinde added a commit to seawinde/doris that referenced this pull request Jul 16, 2024
@seawinde
[improvement](mv) Catch exception when init sync materialized view co…
c9c8711 
…ntext (apache#37701)

## Proposed changes
Init sync materialized view context happend after analyze, If fail would
cause query or insert into fail.
This is inacceptable, so catch the exception and recored the warn log.
# Conflicts:
#	fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/InitMaterializationContextHook.java
seawinde added a commit to seawinde/doris that referenced this pull request Jul 16, 2024
@seawinde
[improvement](mv) Catch exception when init sync materialized view co…
382f8b9 
…ntext (apache#37701)

## Proposed changes
Init sync materialized view context happend after analyze, If fail would
cause query or insert into fail.
This is inacceptable, so catch the exception and recored the warn log.
@seawinde seawinde mentioned this pull request Jul 16, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jul 16, 2024
@seawinde @starocean999
[improvement](mv) Catch exception when init sync materialized view co…
2d7acd0 
…ntext #37701 (#37872)

## Proposed changes

cherry-pick 3.0
pr: #37701
commit: e478570


pr: #37154
commit: 841e39a

---------

Co-authored-by: starocean999 <40539150+starocean999@users.noreply.github.com>
seawinde added a commit to seawinde/doris that referenced this pull request Jul 17, 2024
@seawinde
[improvement](mv) Catch exception when init sync materialized view co…
04afaa6 
…ntext (apache#37701)

## Proposed changes
Init sync materialized view context happend after analyze, If fail would
cause query or insert into fail.
This is inacceptable, so catch the exception and recored the warn log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged not-merge/2.1 p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@seawinde @doris-robot @starocean999 @morrySnow @dataroaring