Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids) fix union all instance number #40100

Merged
merged 2 commits into from
Sep 2, 2024

Conversation

xzj7019
Copy link
Contributor

@xzj7019 xzj7019 commented Aug 29, 2024

pick from #39999

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xzj7019
Copy link
Contributor Author

xzj7019 commented Aug 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49510 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f73b44fa3d5693f80a63b1fe20fcf430ec70bc7a, data reload: false

------ Round 1 ----------------------------------
q1	17661	4381	4345	4345
q2	2031	188	141	141
q3	10236	1914	1908	1908
q4	10367	1255	1302	1255
q5	8768	3895	3922	3895
q6	233	125	123	123
q7	2052	1632	1604	1604
q8	9259	2712	2683	2683
q9	10676	10229	10170	10170
q10	8640	3502	3494	3494
q11	421	237	251	237
q12	469	292	305	292
q13	18358	3940	3979	3940
q14	361	329	333	329
q15	509	457	468	457
q16	680	567	568	567
q17	1117	970	974	970
q18	7264	6892	6822	6822
q19	1670	1542	1487	1487
q20	528	307	292	292
q21	4459	4145	4117	4117
q22	495	400	382	382
Total cold run time: 116254 ms
Total hot run time: 49510 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4290	4259	4307	4259
q2	319	221	221	221
q3	4189	4152	4140	4140
q4	2732	2724	2766	2724
q5	7125	7066	7080	7066
q6	234	116	121	116
q7	3198	2921	2876	2876
q8	4385	4465	4486	4465
q9	16892	16717	16591	16591
q10	4266	4287	4244	4244
q11	758	674	683	674
q12	1019	849	862	849
q13	7470	3731	3738	3731
q14	455	432	426	426
q15	497	462	454	454
q16	749	683	670	670
q17	3756	3787	3821	3787
q18	8745	8752	8748	8748
q19	1700	1671	1614	1614
q20	2361	2139	2107	2107
q21	8414	8424	8351	8351
q22	1035	920	944	920
Total cold run time: 84589 ms
Total hot run time: 79033 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201939 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f73b44fa3d5693f80a63b1fe20fcf430ec70bc7a, data reload: false

query1	954	390	406	390
query2	6531	2169	2176	2169
query3	6920	210	197	197
query4	20812	18111	18141	18111
query5	19737	6548	6457	6457
query6	296	225	239	225
query7	4162	301	313	301
query8	271	253	263	253
query9	3183	2762	2675	2675
query10	444	305	304	304
query11	11345	10742	10722	10722
query12	120	91	73	73
query13	5593	651	660	651
query14	17850	13309	13552	13309
query15	380	219	237	219
query16	6471	282	268	268
query17	1743	1452	903	903
query18	2276	408	405	405
query19	220	146	151	146
query20	81	78	80	78
query21	187	96	98	96
query22	5196	5050	5056	5050
query23	32718	32032	31941	31941
query24	6944	6550	6484	6484
query25	532	426	419	419
query26	527	166	160	160
query27	1880	297	298	297
query28	6103	2295	2261	2261
query29	2935	2786	2793	2786
query30	242	168	170	168
query31	908	744	795	744
query32	68	61	58	58
query33	408	254	266	254
query34	856	482	467	467
query35	1115	929	919	919
query36	1215	1246	1408	1246
query37	90	61	61	61
query38	3052	2918	2883	2883
query39	1381	1316	1313	1313
query40	208	100	98	98
query41	39	38	37	37
query42	82	89	79	79
query43	691	521	650	521
query44	1150	719	722	719
query45	243	238	229	229
query46	1243	963	994	963
query47	1765	1748	1759	1748
query48	954	674	681	674
query49	619	381	371	371
query50	888	626	616	616
query51	4802	4655	4664	4655
query52	95	77	91	77
query53	436	317	325	317
query54	2667	2449	2425	2425
query55	87	83	81	81
query56	231	228	209	209
query57	1248	1058	1189	1058
query58	221	220	204	204
query59	3428	3257	3089	3089
query60	226	200	197	197
query61	99	96	97	96
query62	843	513	488	488
query63	486	336	345	336
query64	2597	1551	1543	1543
query65	3616	3558	3561	3558
query66	778	374	383	374
query67	15755	17399	15354	15354
query68	8970	643	629	629
query69	592	365	349	349
query70	1717	1587	1418	1418
query71	419	312	309	309
query72	6508	3478	3458	3458
query73	722	331	326	326
query74	6302	5834	5856	5834
query75	5288	3632	3715	3632
query76	5588	1142	1179	1142
query77	920	259	256	256
query78	12514	11879	11456	11456
query79	8870	633	651	633
query80	1610	410	391	391
query81	498	244	231	231
query82	1644	98	94	94
query83	171	134	133	133
query84	261	70	71	70
query85	883	321	316	316
query86	338	291	288	288
query87	3274	3010	3064	3010
query88	4996	2329	2304	2304
query89	393	279	311	279
query90	1966	216	217	216
query91	152	131	120	120
query92	58	53	52	52
query93	6710	547	539	539
query94	705	225	208	208
query95	1121	1084	1053	1053
query96	648	327	323	323
query97	6426	6371	6304	6304
query98	200	186	173	173
query99	2996	814	876	814
Total cold run time: 314703 ms
Total hot run time: 201939 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.33 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f73b44fa3d5693f80a63b1fe20fcf430ec70bc7a, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.02	0.02
query3	0.26	0.05	0.05
query4	1.78	0.08	0.08
query5	0.53	0.52	0.53
query6	1.30	0.66	0.61
query7	0.01	0.01	0.00
query8	0.04	0.03	0.03
query9	0.53	0.47	0.46
query10	0.53	0.54	0.53
query11	0.11	0.09	0.09
query12	0.11	0.09	0.09
query13	0.61	0.62	0.61
query14	0.78	0.79	0.80
query15	0.79	0.76	0.76
query16	0.35	0.36	0.36
query17	1.03	1.01	1.02
query18	0.19	0.27	0.24
query19	1.91	1.84	1.85
query20	0.02	0.01	0.01
query21	15.44	0.57	0.56
query22	2.05	2.54	1.43
query23	16.77	0.97	1.12
query24	8.40	0.90	0.78
query25	0.40	0.07	0.05
query26	0.87	0.18	0.15
query27	0.04	0.04	0.04
query28	5.16	0.72	0.77
query29	12.71	2.30	2.22
query30	0.60	0.53	0.52
query31	2.80	0.38	0.38
query32	3.39	0.50	0.49
query33	3.09	3.08	3.06
query34	15.26	4.78	4.79
query35	4.87	4.81	4.84
query36	1.04	1.01	1.01
query37	0.06	0.05	0.05
query38	0.04	0.02	0.02
query39	0.03	0.01	0.02
query40	0.16	0.13	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.02
query43	0.03	0.01	0.01
Total cold run time: 104.26 s
Total hot run time: 30.33 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f73b44fa3d5693f80a63b1fe20fcf430ec70bc7a with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@xzj7019
Copy link
Contributor Author

xzj7019 commented Aug 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49343 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc2b44974ef30525984f56e2644a783aaa9bc4ed, data reload: false

------ Round 1 ----------------------------------
q1	17622	4389	4339	4339
q2	2104	157	144	144
q3	10462	1873	1918	1873
q4	10361	1260	1322	1260
q5	8448	3904	3883	3883
q6	236	121	123	121
q7	1999	1623	1603	1603
q8	9563	2711	2691	2691
q9	13727	10243	10071	10071
q10	8665	3514	3514	3514
q11	411	260	250	250
q12	466	297	309	297
q13	18332	3950	4031	3950
q14	359	338	320	320
q15	514	456	473	456
q16	660	568	565	565
q17	1120	942	903	903
q18	7203	6963	6798	6798
q19	1674	1575	1526	1526
q20	542	316	295	295
q21	4398	4139	4093	4093
q22	488	391	424	391
Total cold run time: 119354 ms
Total hot run time: 49343 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4280	4312	4266	4266
q2	317	236	223	223
q3	4187	4320	4165	4165
q4	2745	2747	2745	2745
q5	7222	7200	7128	7128
q6	239	117	119	117
q7	3215	2790	2816	2790
q8	4372	4450	4472	4450
q9	16819	16701	16716	16701
q10	4257	4271	4261	4261
q11	767	695	690	690
q12	1035	870	871	870
q13	6791	3755	3743	3743
q14	455	418	428	418
q15	503	450	457	450
q16	732	690	685	685
q17	3777	3838	3784	3784
q18	8690	8702	8789	8702
q19	1708	1675	1647	1647
q20	2338	2112	2095	2095
q21	8493	8530	8484	8484
q22	1026	926	947	926
Total cold run time: 83968 ms
Total hot run time: 79340 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201753 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc2b44974ef30525984f56e2644a783aaa9bc4ed, data reload: false

query1	948	392	432	392
query2	6538	2243	2100	2100
query3	6926	206	199	199
query4	21156	18180	18007	18007
query5	19744	6524	6546	6524
query6	278	218	245	218
query7	4172	300	299	299
query8	260	237	227	227
query9	3109	2709	2649	2649
query10	419	296	301	296
query11	11339	10737	10817	10737
query12	120	80	77	77
query13	5610	657	646	646
query14	17866	13466	13394	13394
query15	358	234	237	234
query16	6441	284	259	259
query17	1701	1441	864	864
query18	2335	425	404	404
query19	209	147	157	147
query20	79	81	80	80
query21	187	97	101	97
query22	5153	5093	4994	4994
query23	32435	31836	31966	31836
query24	6962	6552	6560	6552
query25	517	437	431	431
query26	530	162	159	159
query27	1893	295	289	289
query28	6208	2303	2256	2256
query29	2973	2761	2686	2686
query30	237	168	165	165
query31	928	715	742	715
query32	67	63	57	57
query33	402	255	251	251
query34	831	472	483	472
query35	1117	899	849	849
query36	1269	1095	1187	1095
query37	88	63	61	61
query38	3072	2961	2940	2940
query39	1389	1326	1313	1313
query40	208	96	94	94
query41	39	37	38	37
query42	84	84	85	84
query43	558	546	553	546
query44	1140	719	724	719
query45	244	236	235	235
query46	1245	963	981	963
query47	1876	1726	1770	1726
query48	978	663	658	658
query49	619	375	373	373
query50	872	645	617	617
query51	4783	4686	4706	4686
query52	88	87	80	80
query53	444	324	323	323
query54	2657	2470	2432	2432
query55	91	82	91	82
query56	213	229	212	212
query57	1394	1115	1080	1080
query58	200	217	208	208
query59	3443	3179	3277	3179
query60	216	203	195	195
query61	97	103	95	95
query62	831	511	470	470
query63	480	341	332	332
query64	2631	1556	1482	1482
query65	3623	3566	3564	3564
query66	812	378	371	371
query67	18827	17482	14960	14960
query68	9140	694	649	649
query69	571	357	351	351
query70	1716	1442	1450	1442
query71	414	305	305	305
query72	6654	3445	3450	3445
query73	728	322	314	314
query74	6318	5951	5838	5838
query75	5259	3757	3733	3733
query76	5632	1080	1073	1073
query77	936	257	260	257
query78	12599	11846	12848	11846
query79	13650	630	631	630
query80	1379	397	392	392
query81	496	240	235	235
query82	1131	100	95	95
query83	186	134	129	129
query84	259	70	72	70
query85	892	317	314	314
query86	332	297	295	295
query87	3238	3031	3031	3031
query88	5737	2295	2290	2290
query89	422	308	301	301
query90	2005	207	212	207
query91	156	124	118	118
query92	58	52	52	52
query93	5630	618	579	579
query94	755	210	206	206
query95	1107	1083	1058	1058
query96	630	320	318	318
query97	6499	6455	6382	6382
query98	206	174	176	174
query99	2807	930	850	850
Total cold run time: 322101 ms
Total hot run time: 201753 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.78 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc2b44974ef30525984f56e2644a783aaa9bc4ed, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.05	0.06
query4	1.78	0.10	0.08
query5	0.55	0.53	0.52
query6	1.24	0.62	0.61
query7	0.01	0.00	0.00
query8	0.03	0.02	0.03
query9	0.51	0.48	0.48
query10	0.53	0.54	0.54
query11	0.12	0.08	0.08
query12	0.11	0.10	0.09
query13	0.64	0.62	0.62
query14	0.80	0.78	0.78
query15	0.77	0.76	0.75
query16	0.36	0.37	0.36
query17	1.04	1.02	1.00
query18	0.21	0.25	0.23
query19	1.94	1.87	1.81
query20	0.01	0.01	0.01
query21	15.47	0.58	0.56
query22	1.87	1.84	1.42
query23	17.35	1.00	0.95
query24	4.28	1.24	1.38
query25	0.38	0.06	0.06
query26	0.60	0.16	0.15
query27	0.04	0.03	0.04
query28	8.28	0.72	0.75
query29	12.81	2.35	2.26
query30	0.57	0.54	0.52
query31	2.80	0.39	0.37
query32	3.40	0.49	0.50
query33	3.08	3.05	3.04
query34	15.26	4.82	4.81
query35	4.86	4.86	4.82
query36	1.04	1.01	1.02
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.06	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.45 s
Total hot run time: 30.78 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit dc2b44974ef30525984f56e2644a783aaa9bc4ed with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.0 seconds inserted 10000000 Rows, about 476K ops/s

@morrySnow morrySnow merged commit a4d124e into apache:branch-2.0 Sep 2, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants