Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement](multi-catalog)support max compute complex type parse #39259

Merged
merged 6 commits into from
Aug 16, 2024
Merged

[Improvement](multi-catalog)support max compute complex type parse #39259

merged 6 commits into from
Aug 16, 2024

Conversation

wsjz
Copy link
Contributor

@wsjz wsjz commented Aug 12, 2024 •
edited
Loading

Proposed changes

support max compute array/map/struct type and add test case

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 12, 2024
@wsjz wsjz force-pushed the mc_array_map branch 4 times, most recently from 6b76101 to 2425b11 Compare August 12, 2024 17:01
@wsjz
Copy link
Contributor Author

wsjz commented Aug 12, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39043 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7e0b07a0777bdb1b5ec3f646d898bac9bbb11f78, data reload: false

------ Round 1 ----------------------------------
q1	17619	4474	4260	4260
q2	2008	173	171	171
q3	10516	1155	1078	1078
q4	10147	736	771	736
q5	7484	2539	2511	2511
q6	219	137	137	137
q7	970	585	573	573
q8	9226	1892	1925	1892
q9	8877	6547	6541	6541
q10	6998	2232	2162	2162
q11	502	245	246	245
q12	392	217	217	217
q13	17771	2963	2957	2957
q14	286	240	236	236
q15	516	484	489	484
q16	486	393	384	384
q17	945	667	632	632
q18	8102	7484	7445	7445
q19	7607	1049	1084	1049
q20	705	329	333	329
q21	5261	4381	4058	4058
q22	1099	978	946	946
Total cold run time: 117736 ms
Total hot run time: 39043 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4465	4241	4248	4241
q2	383	260	267	260
q3	2822	2628	2607	2607
q4	1854	1644	1587	1587
q5	5251	5213	5238	5213
q6	216	130	130	130
q7	2052	1685	1704	1685
q8	3147	3282	3315	3282
q9	8393	8332	8340	8332
q10	3396	3170	3122	3122
q11	574	509	514	509
q12	779	616	605	605
q13	17495	2991	2995	2991
q14	302	267	274	267
q15	531	480	473	473
q16	487	414	406	406
q17	1778	1503	1461	1461
q18	7639	7443	7338	7338
q19	1699	1504	1556	1504
q20	2035	1769	1790	1769
q21	5249	4996	5080	4996
q22	1078	1004	1008	1004
Total cold run time: 71625 ms
Total hot run time: 53782 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 200067 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7e0b07a0777bdb1b5ec3f646d898bac9bbb11f78, data reload: false

query1	901	365	378	365
query2	6457	1915	1873	1873
query3	6644	219	215	215
query4	34099	23116	22979	22979
query5	4202	488	488	488
query6	270	158	161	158
query7	4582	290	301	290
query8	259	197	194	194
query9	8708	2507	2473	2473
query10	543	466	447	447
query11	17660	14926	15099	14926
query12	145	96	93	93
query13	1619	371	358	358
query14	10213	7639	6866	6866
query15	259	214	218	214
query16	7693	460	477	460
query17	1622	554	532	532
query18	1925	274	269	269
query19	194	143	139	139
query20	112	108	106	106
query21	207	98	95	95
query22	4199	3921	3875	3875
query23	33643	33038	32999	32999
query24	12093	2610	2546	2546
query25	648	362	365	362
query26	1824	146	151	146
query27	2989	279	288	279
query28	7791	2042	2037	2037
query29	1116	409	405	405
query30	306	145	150	145
query31	958	797	737	737
query32	89	55	53	53
query33	743	283	277	277
query34	976	466	472	466
query35	944	821	816	816
query36	1090	938	922	922
query37	287	81	81	81
query38	4290	4133	4194	4133
query39	1432	1382	1369	1369
query40	285	112	118	112
query41	45	44	43	43
query42	110	97	97	97
query43	513	464	459	459
query44	1225	722	741	722
query45	232	206	209	206
query46	1087	734	725	725
query47	1840	1754	1767	1754
query48	363	298	295	295
query49	1204	420	446	420
query50	823	405	411	405
query51	6745	6668	6710	6668
query52	97	87	92	87
query53	247	187	178	178
query54	960	442	455	442
query55	75	74	78	74
query56	264	233	236	233
query57	1176	1048	1062	1048
query58	238	215	226	215
query59	2872	2934	2811	2811
query60	291	261	280	261
query61	101	92	96	92
query62	816	661	678	661
query63	212	178	180	178
query64	10644	2224	1760	1760
query65	3208	3146	3139	3139
query66	1358	344	328	328
query67	15364	14701	14812	14701
query68	4567	538	546	538
query69	499	366	398	366
query70	1137	1108	1061	1061
query71	403	278	270	270
query72	18133	16946	16928	16928
query73	758	332	335	332
query74	9102	8740	8725	8725
query75	3411	2652	2671	2652
query76	2733	1015	986	986
query77	501	315	301	301
query78	9602	9240	9101	9101
query79	2075	528	511	511
query80	1091	475	486	475
query81	594	218	223	218
query82	764	132	138	132
query83	222	147	147	147
query84	246	83	80	80
query85	2047	272	326	272
query86	414	310	299	299
query87	4692	4464	4574	4464
query88	3889	2490	2480	2480
query89	387	279	284	279
query90	1778	195	197	195
query91	126	92	91	91
query92	61	52	49	49
query93	2316	543	525	525
query94	748	287	309	287
query95	354	263	258	258
query96	621	278	280	278
query97	3233	2996	3073	2996
query98	218	198	193	193
query99	1564	1316	1276	1276
Total cold run time: 314413 ms
Total hot run time: 200067 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.59 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7e0b07a0777bdb1b5ec3f646d898bac9bbb11f78, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.49	0.50
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.48	0.49
query10	0.56	0.55	0.53
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.58	0.59
query14	0.76	0.78	0.78
query15	0.90	0.82	0.82
query16	0.38	0.35	0.36
query17	0.93	1.01	1.00
query18	0.22	0.22	0.21
query19	1.80	1.64	1.65
query20	0.02	0.01	0.02
query21	15.42	0.77	0.67
query22	4.26	7.49	2.07
query23	18.28	1.37	1.21
query24	2.15	0.22	0.22
query25	0.15	0.08	0.09
query26	0.30	0.22	0.21
query27	0.45	0.23	0.22
query28	13.29	1.02	1.00
query29	12.59	3.27	3.28
query30	0.24	0.05	0.05
query31	2.89	0.38	0.37
query32	3.30	0.47	0.46
query33	2.86	2.93	2.90
query34	17.11	4.32	4.36
query35	4.42	4.44	4.42
query36	0.65	0.46	0.49
query37	0.20	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.12	0.12
query41	0.09	0.04	0.04
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.85 s
Total hot run time: 30.59 s

@wsjz wsjz changed the title [Improvement](multi-catalog)support array and map type parse [Improvement](multi-catalog)support max compute complex type parse Aug 13, 2024
@wsjz
Copy link
Contributor Author

wsjz commented Aug 13, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40055 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14af7170e2486f92d4fe3a0834458bda20c6e9c9, data reload: false

------ Round 1 ----------------------------------
q1	17627	4382	4334	4334
q2	2012	189	178	178
q3	11903	1018	1063	1018
q4	10518	754	751	751
q5	7766	2812	2831	2812
q6	219	136	136	136
q7	956	587	589	587
q8	9516	2046	2039	2039
q9	10167	6578	6525	6525
q10	7034	2202	2180	2180
q11	453	251	244	244
q12	390	228	225	225
q13	17751	2988	2986	2986
q14	281	235	234	234
q15	518	484	492	484
q16	508	391	388	388
q17	980	747	675	675
q18	8018	7478	7516	7478
q19	6922	1081	1066	1066
q20	690	328	352	328
q21	5473	4555	4388	4388
q22	1128	1019	999	999
Total cold run time: 120830 ms
Total hot run time: 40055 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4517	4286	4260	4260
q2	362	285	272	272
q3	3009	2784	2732	2732
q4	2003	1789	1670	1670
q5	5625	5560	5608	5560
q6	242	136	140	136
q7	2179	1808	1761	1761
q8	3314	3413	3382	3382
q9	8734	8968	8789	8789
q10	3457	3254	3280	3254
q11	613	517	512	512
q12	816	650	620	620
q13	17286	3142	3171	3142
q14	339	302	299	299
q15	547	498	477	477
q16	519	438	452	438
q17	1812	1529	1512	1512
q18	8159	8019	8032	8019
q19	1784	1694	1588	1588
q20	2964	1924	1906	1906
q21	7795	5306	5332	5306
q22	1146	1086	1052	1052
Total cold run time: 77222 ms
Total hot run time: 56687 ms

@wsjz
Copy link
Contributor Author

wsjz commented Aug 13, 2024

run external

@wsjz
Copy link
Contributor Author

wsjz commented Aug 13, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39957 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da6cbed0341f2ae1b3356a21894bee5c0ae607eb, data reload: false

------ Round 1 ----------------------------------
q1	18087	4492	4394	4394
q2	2832	179	175	175
q3	11135	1167	1132	1132
q4	11478	792	785	785
q5	7914	2774	2814	2774
q6	223	139	140	139
q7	959	591	577	577
q8	9323	1985	2020	1985
q9	8610	6545	6553	6545
q10	7051	2201	2147	2147
q11	482	246	250	246
q12	389	225	219	219
q13	18723	2947	2963	2947
q14	284	235	248	235
q15	520	476	489	476
q16	500	398	382	382
q17	951	629	700	629
q18	8041	7525	7419	7419
q19	5326	1079	1076	1076
q20	697	320	338	320
q21	5317	4333	4641	4333
q22	1129	1026	1022	1022
Total cold run time: 119971 ms
Total hot run time: 39957 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4493	4223	4247	4223
q2	382	267	274	267
q3	2832	2631	2622	2622
q4	1835	1628	1540	1540
q5	5352	5334	5341	5334
q6	216	134	133	133
q7	2028	1658	1662	1658
q8	3135	3335	3308	3308
q9	8422	8321	8352	8321
q10	3393	3114	3125	3114
q11	583	503	512	503
q12	778	603	604	603
q13	17376	2987	3005	2987
q14	302	278	268	268
q15	530	474	482	474
q16	473	417	415	415
q17	1774	1480	1463	1463
q18	7700	7678	7318	7318
q19	1643	1465	1589	1465
q20	1975	1781	1805	1781
q21	5019	5128	5129	5128
q22	1111	1057	1051	1051
Total cold run time: 71352 ms
Total hot run time: 53976 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185805 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da6cbed0341f2ae1b3356a21894bee5c0ae607eb, data reload: false

query1	909	374	366	366
query2	6460	1966	1925	1925
query3	6647	203	217	203
query4	34118	23288	23149	23149
query5	4206	481	506	481
query6	281	163	160	160
query7	4580	289	296	289
query8	244	214	208	208
query9	8482	2432	2410	2410
query10	451	286	265	265
query11	17002	14840	14930	14840
query12	149	101	96	96
query13	1649	370	361	361
query14	9802	7786	7743	7743
query15	233	163	164	163
query16	7805	474	485	474
query17	1587	595	549	549
query18	2008	296	290	290
query19	199	149	149	149
query20	126	106	109	106
query21	212	101	98	98
query22	4370	3971	4096	3971
query23	34055	33108	33340	33108
query24	12008	2814	2812	2812
query25	690	395	400	395
query26	1779	160	159	159
query27	2877	272	279	272
query28	7730	2021	2001	2001
query29	1110	440	427	427
query30	309	154	155	154
query31	1010	732	760	732
query32	98	56	59	56
query33	769	289	297	289
query34	915	475	493	475
query35	845	717	727	717
query36	1106	924	945	924
query37	207	89	86	86
query38	3894	3825	3965	3825
query39	1439	1380	1384	1380
query40	279	118	116	116
query41	49	47	48	47
query42	116	96	100	96
query43	510	467	485	467
query44	1163	742	733	733
query45	195	164	166	164
query46	1121	725	739	725
query47	1835	1748	1731	1731
query48	367	299	316	299
query49	1214	436	431	431
query50	800	409	414	409
query51	6895	6716	6650	6650
query52	102	98	88	88
query53	261	182	191	182
query54	965	456	449	449
query55	84	77	76	76
query56	281	253	256	253
query57	1168	1057	1090	1057
query58	242	238	227	227
query59	2955	2725	2689	2689
query60	301	275	274	274
query61	119	118	229	118
query62	828	643	666	643
query63	223	183	187	183
query64	10623	2276	1729	1729
query65	3255	3197	3144	3144
query66	1364	344	329	329
query67	15470	14716	14917	14716
query68	8882	553	572	553
query69	680	397	282	282
query70	1351	1085	1057	1057
query71	524	274	278	274
query72	8007	2260	2040	2040
query73	2212	330	335	330
query74	9112	8796	8649	8649
query75	5344	2693	2711	2693
query76	5647	961	953	953
query77	780	316	313	313
query78	10086	9155	9635	9155
query79	9786	540	540	540
query80	1103	495	483	483
query81	590	223	219	219
query82	459	136	131	131
query83	328	143	141	141
query84	267	77	76	76
query85	1039	271	277	271
query86	336	299	295	295
query87	4469	4324	4372	4324
query88	4184	2401	2424	2401
query89	484	286	326	286
query90	2088	201	195	195
query91	128	96	95	95
query92	64	51	51	51
query93	6354	544	543	543
query94	1108	271	286	271
query95	354	258	263	258
query96	615	272	271	271
query97	3202	3048	3078	3048
query98	242	200	197	197
query99	1545	1291	1277	1277
Total cold run time: 326516 ms
Total hot run time: 185805 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.46 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da6cbed0341f2ae1b3356a21894bee5c0ae607eb, data reload: false

query1	0.05	0.04	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.08	0.07
query5	0.50	0.48	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.04	0.04
query9	0.54	0.48	0.48
query10	0.54	0.54	0.54
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.60	0.59
query14	0.76	0.78	0.80
query15	0.86	0.81	0.80
query16	0.37	0.36	0.38
query17	1.03	0.98	0.95
query18	0.23	0.22	0.21
query19	1.79	1.70	1.70
query20	0.01	0.01	0.03
query21	15.41	0.76	0.67
query22	3.57	8.03	1.78
query23	18.29	1.50	1.37
query24	2.13	0.22	0.22
query25	0.16	0.09	0.08
query26	0.30	0.21	0.21
query27	0.46	0.22	0.22
query28	13.29	1.01	1.00
query29	12.64	3.21	3.27
query30	0.24	0.05	0.04
query31	2.91	0.40	0.39
query32	3.27	0.48	0.48
query33	2.93	2.95	2.99
query34	16.95	4.33	4.34
query35	4.39	4.42	4.37
query36	0.65	0.50	0.46
query37	0.19	0.15	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.03
query40	0.14	0.12	0.12
query41	0.09	0.05	0.05
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.08 s
Total hot run time: 30.46 s

@wsjz
Copy link
Contributor Author

wsjz commented Aug 13, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40041 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5458a210972554f4b987f0e9ca420ff673a294b6, data reload: false

------ Round 1 ----------------------------------
q1	17631	4429	4310	4310
q2	2017	180	175	175
q3	11859	1299	1061	1061
q4	10972	838	802	802
q5	7725	2755	2821	2755
q6	226	145	143	143
q7	961	598	604	598
q8	9540	2057	2119	2057
q9	10223	6519	6541	6519
q10	7008	2213	2193	2193
q11	449	251	244	244
q12	393	229	225	225
q13	17772	2985	2968	2968
q14	270	254	235	235
q15	538	491	504	491
q16	494	378	397	378
q17	971	642	654	642
q18	7976	7468	7389	7389
q19	6444	999	1103	999
q20	661	351	338	338
q21	5288	4520	4518	4518
q22	1093	1001	1002	1001
Total cold run time: 120511 ms
Total hot run time: 40041 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4523	4313	4314	4313
q2	381	277	275	275
q3	2974	2840	2749	2749
q4	2036	1770	1765	1765
q5	5593	5554	5584	5554
q6	230	139	146	139
q7	2169	1778	1790	1778
q8	3289	3428	3392	3392
q9	8753	8898	8806	8806
q10	3400	3161	3317	3161
q11	638	518	523	518
q12	819	641	648	641
q13	16103	3206	3215	3206
q14	329	292	294	292
q15	553	510	496	496
q16	503	451	438	438
q17	1827	1534	1492	1492
q18	8175	8182	7901	7901
q19	4266	1591	1615	1591
q20	2187	1920	1903	1903
q21	6256	5274	5298	5274
q22	1115	1041	1056	1041
Total cold run time: 76119 ms
Total hot run time: 56725 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190766 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5458a210972554f4b987f0e9ca420ff673a294b6, data reload: false

query1	1257	889	867	867
query2	6454	1942	1885	1885
query3	10616	3831	3834	3831
query4	58452	25141	23299	23299
query5	5772	488	504	488
query6	513	190	172	172
query7	6490	295	284	284
query8	315	194	192	192
query9	9034	2427	2412	2412
query10	543	257	267	257
query11	16830	15039	15171	15039
query12	162	104	99	99
query13	1633	365	381	365
query14	12847	7787	7916	7787
query15	252	177	168	168
query16	7913	513	515	513
query17	1134	562	569	562
query18	2203	330	301	301
query19	274	151	157	151
query20	127	115	115	115
query21	217	106	99	99
query22	4602	4477	4576	4477
query23	34383	33358	33269	33269
query24	5514	2827	2844	2827
query25	527	388	379	379
query26	701	155	157	155
query27	1819	267	276	267
query28	3768	2026	2014	2014
query29	724	408	404	404
query30	237	154	149	149
query31	945	739	714	714
query32	103	56	55	55
query33	511	295	299	295
query34	851	464	476	464
query35	813	699	744	699
query36	1060	944	910	910
query37	166	82	80	80
query38	4002	3875	3838	3838
query39	1425	1367	1368	1367
query40	194	112	115	112
query41	44	43	46	43
query42	116	96	94	94
query43	489	455	454	454
query44	1064	724	727	724
query45	192	161	163	161
query46	1082	729	728	728
query47	1878	1764	1749	1749
query48	374	286	290	286
query49	744	407	413	407
query50	804	405	400	400
query51	6818	6697	6590	6590
query52	99	91	87	87
query53	257	179	180	179
query54	557	443	449	443
query55	74	75	75	75
query56	270	252	249	249
query57	1144	1052	1068	1052
query58	226	260	241	241
query59	2989	2765	2697	2697
query60	294	266	257	257
query61	93	95	94	94
query62	751	639	646	639
query63	210	180	179	179
query64	9218	2258	1735	1735
query65	3316	3160	3211	3160
query66	684	330	325	325
query67	15264	14943	14810	14810
query68	4283	541	539	539
query69	394	276	265	265
query70	1131	1132	1105	1105
query71	384	273	282	273
query72	6839	2289	1990	1990
query73	732	324	324	324
query74	9083	8683	8668	8668
query75	3363	2706	2734	2706
query76	2599	1044	991	991
query77	568	308	310	308
query78	9611	9074	9209	9074
query79	1835	536	525	525
query80	1042	479	484	479
query81	548	219	223	219
query82	568	137	133	133
query83	254	143	145	143
query84	258	76	83	76
query85	1086	328	334	328
query86	469	301	276	276
query87	4395	4193	4229	4193
query88	4510	2422	2433	2422
query89	384	290	288	288
query90	1908	195	198	195
query91	131	112	106	106
query92	61	49	53	49
query93	2155	533	531	531
query94	938	294	306	294
query95	366	276	271	271
query96	585	277	275	275
query97	3192	3061	3040	3040
query98	223	210	218	210
query99	1511	1290	1251	1251
Total cold run time: 319840 ms
Total hot run time: 190766 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.12 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5458a210972554f4b987f0e9ca420ff673a294b6, data reload: false

query1	0.05	0.05	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.49	0.49	0.48
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.55	0.54	0.54
query11	0.14	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.62	0.58
query14	0.76	0.78	0.79
query15	0.85	0.81	0.82
query16	0.37	0.36	0.35
query17	1.02	0.98	1.05
query18	0.24	0.22	0.23
query19	1.83	1.69	1.68
query20	0.01	0.01	0.01
query21	15.38	0.77	0.67
query22	4.18	6.97	2.34
query23	18.28	1.42	1.24
query24	2.12	0.23	0.22
query25	0.15	0.09	0.09
query26	0.30	0.21	0.21
query27	0.46	0.23	0.23
query28	13.19	1.03	1.00
query29	12.67	3.30	3.30
query30	0.24	0.06	0.05
query31	2.88	0.40	0.38
query32	3.28	0.48	0.47
query33	2.93	2.99	2.99
query34	16.95	4.32	4.38
query35	4.40	4.46	4.42
query36	0.66	0.47	0.49
query37	0.19	0.16	0.16
query38	0.16	0.16	0.16
query39	0.05	0.03	0.04
query40	0.15	0.12	0.13
query41	0.10	0.04	0.04
query42	0.05	0.06	0.06
query43	0.04	0.04	0.04
Total cold run time: 109.59 s
Total hot run time: 31.12 s

@wsjz
Copy link
Contributor Author

wsjz commented Aug 13, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37485 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 689773f69b62aca855f78bfeb4068a663257abde, data reload: false

------ Round 1 ----------------------------------
q1	17623	4348	4283	4283
q2	2046	179	185	179
q3	11851	1024	1197	1024
q4	10517	728	732	728
q5	7756	2804	2778	2778
q6	226	139	139	139
q7	950	597	606	597
q8	9511	2019	2070	2019
q9	8638	6530	6504	6504
q10	6986	2168	2250	2168
q11	435	244	244	244
q12	395	219	219	219
q13	18916	2976	2966	2966
q14	281	238	240	238
q15	541	489	487	487
q16	505	394	400	394
q17	974	662	759	662
q18	7319	6740	6671	6671
q19	7165	1043	1036	1036
q20	687	330	336	330
q21	4054	2894	2839	2839
q22	1083	980	1005	980
Total cold run time: 118459 ms
Total hot run time: 37485 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4460	4222	4294	4222
q2	375	286	268	268
q3	2849	2626	2576	2576
q4	2008	1734	1737	1734
q5	5584	5701	5540	5540
q6	221	153	142	142
q7	2106	1733	1779	1733
q8	3321	3453	3395	3395
q9	8707	8729	8890	8729
q10	3509	3234	3310	3234
q11	596	495	493	493
q12	773	603	631	603
q13	15909	3214	3137	3137
q14	317	294	292	292
q15	519	487	494	487
q16	491	450	452	450
q17	1846	1528	1506	1506
q18	8095	7919	7876	7876
q19	2743	1629	1612	1612
q20	2151	1902	1910	1902
q21	5372	5257	5250	5250
q22	1108	1037	1020	1020
Total cold run time: 73060 ms
Total hot run time: 56201 ms

@wsjz
Copy link
Contributor Author

wsjz commented Aug 15, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37471 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 53ad32c047a7f9bc898cb5c9a69f79d9c7362538, data reload: false

------ Round 1 ----------------------------------
q1	17914	4373	4275	4275
q2	2032	173	166	166
q3	10524	1171	1136	1136
q4	10145	637	778	637
q5	7729	2811	2718	2718
q6	221	137	137	137
q7	951	596	591	591
q8	9327	2006	2034	2006
q9	7017	6480	6487	6480
q10	7041	2192	2239	2192
q11	448	243	244	243
q12	393	218	213	213
q13	17755	2965	2962	2962
q14	287	236	239	236
q15	528	484	488	484
q16	517	383	385	383
q17	960	670	677	670
q18	7466	6858	6768	6768
q19	6904	1106	1021	1021
q20	668	332	335	332
q21	3999	2805	2828	2805
q22	1084	1017	1016	1016
Total cold run time: 113910 ms
Total hot run time: 37471 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4571	4244	4257	4244
q2	398	263	268	263
q3	2791	2625	2654	2625
q4	1899	1625	1644	1625
q5	5403	5541	5587	5541
q6	227	132	132	132
q7	2170	1774	1783	1774
q8	3291	3409	3390	3390
q9	8686	8756	8920	8756
q10	3518	3249	3281	3249
q11	599	571	501	501
q12	803	615	596	596
q13	17311	3021	3143	3021
q14	326	294	291	291
q15	517	500	486	486
q16	499	441	449	441
q17	1802	1566	1503	1503
q18	8121	7808	7365	7365
q19	1721	1502	1533	1502
q20	2677	1823	1791	1791
q21	7954	5121	5108	5108
q22	1110	1026	1023	1023
Total cold run time: 76394 ms
Total hot run time: 55227 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183737 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 53ad32c047a7f9bc898cb5c9a69f79d9c7362538, data reload: false

query1	919	369	375	369
query2	6440	1935	1823	1823
query3	6637	204	216	204
query4	34061	23234	23024	23024
query5	4177	491	501	491
query6	271	172	173	172
query7	4588	290	295	290
query8	248	194	203	194
query9	8522	2448	2415	2415
query10	431	279	252	252
query11	17849	14972	14986	14972
query12	140	107	96	96
query13	1622	371	354	354
query14	9840	7021	6476	6476
query15	213	163	171	163
query16	7585	490	436	436
query17	1496	541	543	541
query18	1839	278	285	278
query19	180	136	137	136
query20	107	105	105	105
query21	210	98	103	98
query22	4350	3969	3932	3932
query23	33771	33181	33291	33181
query24	11378	2902	2888	2888
query25	642	372	373	372
query26	1349	153	152	152
query27	2936	274	274	274
query28	7410	2056	2067	2056
query29	895	396	407	396
query30	285	148	149	148
query31	963	747	764	747
query32	95	51	56	51
query33	734	295	296	295
query34	925	460	473	460
query35	820	708	706	706
query36	1063	908	890	890
query37	151	80	79	79
query38	4011	3795	3817	3795
query39	1419	1380	1414	1380
query40	273	123	114	114
query41	55	48	44	44
query42	114	95	99	95
query43	508	458	440	440
query44	1245	723	728	723
query45	193	164	163	163
query46	1090	732	740	732
query47	1859	1766	1770	1766
query48	357	285	287	285
query49	1076	443	413	413
query50	808	402	400	400
query51	6928	6769	6659	6659
query52	95	90	87	87
query53	252	181	185	181
query54	836	461	453	453
query55	76	76	76	76
query56	270	239	250	239
query57	1129	1043	1075	1043
query58	250	228	217	217
query59	2989	2783	2670	2670
query60	300	262	261	261
query61	96	93	94	93
query62	819	613	638	613
query63	210	178	182	178
query64	5433	2263	1750	1750
query65	3182	3118	3199	3118
query66	958	323	322	322
query67	15465	14941	14846	14846
query68	4570	545	547	545
query69	455	301	276	276
query70	1085	1087	1159	1087
query71	410	284	281	281
query72	6701	2285	2015	2015
query73	766	325	323	323
query74	9068	8740	8716	8716
query75	3477	2677	2650	2650
query76	2562	926	907	907
query77	506	305	298	298
query78	9488	9187	8989	8989
query79	2109	518	521	518
query80	1161	482	478	478
query81	597	219	217	217
query82	801	133	130	130
query83	246	145	146	145
query84	244	76	75	75
query85	2091	271	268	268
query86	468	318	306	306
query87	4281	4272	4164	4164
query88	3791	2293	2290	2290
query89	404	285	290	285
query90	1870	196	195	195
query91	132	177	96	96
query92	57	49	51	49
query93	2505	530	523	523
query94	915	304	293	293
query95	356	250	250	250
query96	600	267	266	266
query97	3210	2981	3069	2981
query98	223	206	201	201
query99	1694	1256	1248	1248
Total cold run time: 294281 ms
Total hot run time: 183737 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.29 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 53ad32c047a7f9bc898cb5c9a69f79d9c7362538, data reload: false

query1	0.05	0.04	0.03
query2	0.07	0.04	0.03
query3	0.22	0.05	0.05
query4	1.66	0.10	0.10
query5	0.50	0.50	0.50
query6	1.13	0.73	0.74
query7	0.02	0.02	0.01
query8	0.06	0.05	0.05
query9	0.54	0.48	0.47
query10	0.55	0.55	0.54
query11	0.16	0.12	0.12
query12	0.15	0.11	0.12
query13	0.60	0.60	0.58
query14	0.76	0.78	0.79
query15	0.87	0.80	0.80
query16	0.38	0.37	0.37
query17	0.97	0.98	0.98
query18	0.24	0.22	0.23
query19	1.78	1.71	1.69
query20	0.01	0.01	0.01
query21	15.40	0.78	0.66
query22	3.96	7.84	1.55
query23	18.29	1.35	1.28
query24	2.10	0.22	0.21
query25	0.17	0.08	0.07
query26	0.29	0.21	0.21
query27	0.46	0.23	0.22
query28	13.30	1.03	0.99
query29	12.68	3.28	3.30
query30	0.24	0.05	0.05
query31	2.89	0.39	0.38
query32	3.28	0.48	0.48
query33	2.94	2.93	2.93
query34	17.10	4.33	4.41
query35	4.50	4.44	4.44
query36	0.65	0.46	0.47
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.04
query40	0.15	0.12	0.13
query41	0.09	0.05	0.04
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.7 s
Total hot run time: 30.29 s

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morningman morningman merged commit e350ff5 into apache:master Aug 16, 2024
29 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@wsjz @dataroaring
[Improvement](multi-catalog)support max compute complex type parse (#…
2458804 
…39259)

## Proposed changes

support max compute array/map/struct type and add test case
@wsjz wsjz deleted the mc_array_map branch August 21, 2024 03:46
@wsjz wsjz mentioned this pull request Aug 21, 2024
Merged
@wsjz
Copy link
Contributor Author

wsjz commented Aug 21, 2024

add fix #39680

morningman pushed a commit that referenced this pull request Aug 22, 2024
@wsjz
[fix](multi-catalog)fix max compute array and map type read offset (#…
ab56888 
…39680)

fix for #39259
wsjz added a commit to wsjz/incubator-doris that referenced this pull request Aug 23, 2024
@wsjz
[fix](multi-catalog)fix max compute array and map type read offset (a…
a594245 
…pache#39680)

fix for apache#39259

(cherry picked from commit ab56888)
@wsjz wsjz mentioned this pull request Aug 23, 2024
Merged
morningman pushed a commit to wsjz/incubator-doris that referenced this pull request Aug 23, 2024
@wsjz @morningman
[fix](multi-catalog)fix max compute array and map type read offset (a…
397389d 
…pache#39680)

fix for apache#39259

(cherry picked from commit ab56888)
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@wsjz @dataroaring
[fix](multi-catalog)fix max compute array and map type read offset (#…
2511ea0 
…39680)

fix for #39259
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged doing reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wsjz @doris-robot @morningman @CalvinKirs @dataroaring