Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](func)fix array_with_const with larger than max_array_size #38150

Merged
merged 2 commits into from
Jul 28, 2024

Conversation

amorynan
Copy link
Contributor

Proposed changes

backport: #37495
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@amorynan
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466)
Line Coverage: 29.48% (66535/225696)
Region Coverage: 28.95% (34287/118447)
Branch Coverage: 24.83% (17615/70952)
Coverage Report: http://coverage.selectdb-in.cc/coverage/98d4e700e19d1dec2a20b33c4284dacbf09dde83_98d4e700e19d1dec2a20b33c4284dacbf09dde83/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49504 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98d4e700e19d1dec2a20b33c4284dacbf09dde83, data reload: false

------ Round 1 ----------------------------------
q1	17695	4419	4324	4324
q2	2087	155	147	147
q3	10267	1870	1948	1870
q4	10383	1215	1290	1215
q5	8794	3819	3919	3819
q6	235	147	127	127
q7	2036	1612	1583	1583
q8	9317	2725	2700	2700
q9	10580	10374	10187	10187
q10	8672	3496	3527	3496
q11	414	260	240	240
q12	467	305	308	305
q13	18334	3941	3972	3941
q14	360	334	328	328
q15	497	454	457	454
q16	677	576	573	573
q17	1127	948	959	948
q18	7382	6881	6838	6838
q19	1765	1668	1586	1586
q20	543	312	294	294
q21	4464	4099	4099	4099
q22	526	463	430	430
Total cold run time: 116622 ms
Total hot run time: 49504 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4298	4327	4295	4295
q2	321	228	221	221
q3	4167	4139	4166	4139
q4	2745	2747	2760	2747
q5	7157	7127	7119	7119
q6	239	119	117	117
q7	3266	2843	2807	2807
q8	4346	4475	4468	4468
q9	16922	16839	16736	16736
q10	4232	4262	4255	4255
q11	779	703	674	674
q12	1029	874	855	855
q13	6995	3758	3735	3735
q14	452	433	422	422
q15	504	466	459	459
q16	740	687	708	687
q17	3864	3905	3878	3878
q18	8758	8832	8670	8670
q19	1779	1703	1672	1672
q20	2393	2132	2132	2132
q21	8435	8495	8487	8487
q22	1069	980	978	978
Total cold run time: 84490 ms
Total hot run time: 79553 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203584 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98d4e700e19d1dec2a20b33c4284dacbf09dde83, data reload: false

query1	927	418	376	376
query2	6559	2711	2576	2576
query3	6922	212	205	205
query4	20088	17989	17930	17930
query5	19718	6501	6496	6496
query6	284	219	237	219
query7	4154	307	307	307
query8	406	397	393	393
query9	3137	2664	2593	2593
query10	427	313	296	296
query11	11240	10769	10837	10769
query12	128	80	80	80
query13	5608	685	695	685
query14	17673	13252	13716	13252
query15	366	254	243	243
query16	6475	282	264	264
query17	1722	1443	894	894
query18	2307	418	413	413
query19	209	153	158	153
query20	84	78	81	78
query21	191	95	91	91
query22	5303	4980	4934	4934
query23	32498	31826	31936	31826
query24	6993	6657	6602	6602
query25	534	444	429	429
query26	536	166	171	166
query27	1864	303	298	298
query28	6185	2358	2300	2300
query29	2977	2663	2773	2663
query30	239	169	169	169
query31	903	742	774	742
query32	63	62	62	62
query33	402	257	245	245
query34	843	488	494	488
query35	1132	953	948	948
query36	1277	1232	1296	1232
query37	89	62	63	62
query38	3056	2940	2959	2940
query39	1382	1356	1307	1307
query40	212	98	99	98
query41	47	44	52	44
query42	91	82	85	82
query43	691	736	649	649
query44	1139	720	723	720
query45	251	242	236	236
query46	1226	950	974	950
query47	1895	1682	1726	1682
query48	1005	744	723	723
query49	630	378	365	365
query50	858	612	613	612
query51	4748	4723	4685	4685
query52	87	86	91	86
query53	457	321	331	321
query54	2657	2456	2474	2456
query55	90	90	86	86
query56	247	220	221	220
query57	1277	1119	1049	1049
query58	232	203	199	199
query59	4074	4035	4064	4035
query60	217	208	227	208
query61	100	100	101	100
query62	835	479	474	474
query63	489	342	350	342
query64	2579	1575	1447	1447
query65	3634	3557	3545	3545
query66	777	393	382	382
query67	16755	15277	15445	15277
query68	8084	658	653	653
query69	566	368	377	368
query70	1730	1309	1552	1309
query71	428	306	312	306
query72	6554	3492	3502	3492
query73	726	323	319	319
query74	6240	5821	5802	5802
query75	4654	3716	3737	3716
query76	4598	1177	1170	1170
query77	575	256	245	245
query78	12475	11643	12034	11643
query79	7007	627	660	627
query80	1069	402	405	402
query81	502	232	234	232
query82	1506	100	98	98
query83	175	134	129	129
query84	262	73	70	70
query85	898	338	343	338
query86	335	297	291	291
query87	3280	3067	3106	3067
query88	4905	2280	2289	2280
query89	349	292	328	292
query90	1932	204	213	204
query91	175	139	138	138
query92	60	55	60	55
query93	4101	612	569	569
query94	750	219	209	209
query95	1114	1065	1055	1055
query96	639	319	329	319
query97	6475	6343	6324	6324
query98	193	178	161	161
query99	2983	902	916	902
Total cold run time: 307541 ms
Total hot run time: 203584 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.85 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98d4e700e19d1dec2a20b33c4284dacbf09dde83, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.78	0.06	0.07
query5	0.55	0.52	0.52
query6	1.22	0.63	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.51	0.50	0.49
query10	0.54	0.54	0.54
query11	0.12	0.09	0.09
query12	0.12	0.09	0.10
query13	0.63	0.61	0.62
query14	0.78	0.78	0.78
query15	0.79	0.76	0.77
query16	0.38	0.37	0.38
query17	1.01	1.02	1.01
query18	0.21	0.25	0.24
query19	1.90	1.82	1.85
query20	0.02	0.01	0.01
query21	15.46	0.56	0.54
query22	2.13	1.97	1.58
query23	17.24	0.97	1.01
query24	4.68	1.04	1.35
query25	0.37	0.12	0.03
query26	0.57	0.16	0.16
query27	0.05	0.04	0.04
query28	8.01	0.72	0.73
query29	12.61	2.33	2.31
query30	0.52	0.57	0.55
query31	2.81	0.39	0.39
query32	3.38	0.50	0.49
query33	3.05	3.04	3.04
query34	15.28	4.82	4.76
query35	4.86	4.86	4.84
query36	1.06	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.41 s
Total hot run time: 30.85 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 98d4e700e19d1dec2a20b33c4284dacbf09dde83 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@amorynan
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49917 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false

------ Round 1 ----------------------------------
q1	18138	4443	4329	4329
q2	2074	156	157	156
q3	10298	1876	1905	1876
q4	10102	1250	1301	1250
q5	8420	3830	3935	3830
q6	239	125	150	125
q7	2072	1653	1619	1619
q8	9594	2732	2740	2732
q9	12916	10304	10308	10304
q10	8612	3523	3556	3523
q11	413	239	258	239
q12	473	303	303	303
q13	18360	3964	4020	3964
q14	346	339	322	322
q15	513	456	456	456
q16	660	579	577	577
q17	1138	948	945	945
q18	7306	6960	7000	6960
q19	1797	1662	1602	1602
q20	526	318	314	314
q21	4404	4142	4050	4050
q22	530	441	449	441
Total cold run time: 118931 ms
Total hot run time: 49917 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4375	4329	4303	4303
q2	322	230	222	222
q3	4221	4150	4146	4146
q4	2754	2735	2745	2735
q5	7202	7129	7160	7129
q6	243	125	123	123
q7	3191	2784	2805	2784
q8	4381	4485	4434	4434
q9	16830	16761	16750	16750
q10	4221	4278	4259	4259
q11	751	697	667	667
q12	1032	882	864	864
q13	6709	3791	3756	3756
q14	446	424	425	424
q15	512	454	459	454
q16	729	681	674	674
q17	3920	3918	3870	3870
q18	8805	8783	8781	8781
q19	1761	1668	1706	1668
q20	2382	2172	2084	2084
q21	8546	8409	8524	8409
q22	1046	1010	1008	1008
Total cold run time: 84379 ms
Total hot run time: 79544 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8117/21466)
Line Coverage: 29.48% (66552/225741)
Region Coverage: 28.96% (34303/118467)
Branch Coverage: 24.84% (17625/70960)
Coverage Report: http://coverage.selectdb-in.cc/coverage/cb1808a3924f62b57926e08509bb31647acff685_cb1808a3924f62b57926e08509bb31647acff685/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 203766 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false

query1	940	424	380	380
query2	6550	2720	2827	2720
query3	6930	219	218	218
query4	20233	18161	17912	17912
query5	19727	6559	6646	6559
query6	290	213	223	213
query7	4160	308	325	308
query8	411	443	442	442
query9	3186	2722	2654	2654
query10	420	304	284	284
query11	11293	10754	10712	10712
query12	123	84	73	73
query13	5602	723	701	701
query14	17634	13373	13444	13373
query15	360	238	242	238
query16	6484	287	261	261
query17	1714	1441	872	872
query18	2327	436	415	415
query19	213	146	149	146
query20	83	79	80	79
query21	193	99	94	94
query22	5228	5015	4984	4984
query23	32643	32120	32126	32120
query24	6938	6475	6474	6474
query25	523	436	439	436
query26	534	165	160	160
query27	1884	308	307	307
query28	6177	2384	2363	2363
query29	2785	2755	2696	2696
query30	241	166	175	166
query31	937	732	764	732
query32	69	67	63	63
query33	411	256	262	256
query34	847	500	494	494
query35	1125	934	934	934
query36	1332	1116	1080	1080
query37	89	65	60	60
query38	3111	2907	2958	2907
query39	1364	1321	1339	1321
query40	201	90	97	90
query41	48	45	44	44
query42	97	89	86	86
query43	713	706	658	658
query44	1149	717	727	717
query45	244	236	242	236
query46	1221	975	966	966
query47	1817	1839	1741	1741
query48	1024	719	719	719
query49	643	387	378	378
query50	862	613	589	589
query51	4729	4714	4661	4661
query52	96	87	91	87
query53	445	325	318	318
query54	2639	2494	2475	2475
query55	90	83	84	83
query56	248	230	221	221
query57	1248	1106	1132	1106
query58	209	215	208	208
query59	4446	4130	3898	3898
query60	225	231	212	212
query61	99	93	102	93
query62	835	464	444	444
query63	492	344	344	344
query64	2507	1538	1401	1401
query65	3672	3562	3573	3562
query66	830	392	389	389
query67	17329	16153	15017	15017
query68	8609	645	659	645
query69	568	356	357	356
query70	1707	1388	1435	1388
query71	394	303	328	303
query72	6604	3449	3497	3449
query73	737	328	324	324
query74	6356	5863	5970	5863
query75	4978	3735	3743	3735
query76	5026	1161	1150	1150
query77	756	272	259	259
query78	12583	11632	11684	11632
query79	7157	633	634	633
query80	1202	416	412	412
query81	501	237	232	232
query82	1496	101	99	99
query83	178	140	134	134
query84	258	73	70	70
query85	1021	331	335	331
query86	342	291	297	291
query87	3240	3024	2981	2981
query88	5076	2373	2362	2362
query89	353	285	315	285
query90	1880	207	210	207
query91	176	140	141	140
query92	58	54	53	53
query93	3047	580	539	539
query94	729	216	213	213
query95	1138	1068	1056	1056
query96	645	335	331	331
query97	6557	6417	6365	6365
query98	187	175	168	168
query99	2863	810	860	810
Total cold run time: 309691 ms
Total hot run time: 203766 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cb1808a3924f62b57926e08509bb31647acff685, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.03	0.03
query3	0.25	0.05	0.04
query4	1.80	0.07	0.07
query5	0.54	0.52	0.52
query6	1.23	0.88	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.48
query10	0.54	0.54	0.54
query11	0.12	0.09	0.08
query12	0.12	0.09	0.10
query13	0.62	0.62	0.60
query14	0.79	0.79	0.79
query15	0.76	0.75	0.76
query16	0.37	0.36	0.38
query17	1.00	1.03	1.02
query18	0.23	0.24	0.25
query19	1.85	1.85	1.85
query20	0.01	0.02	0.01
query21	15.48	0.56	0.56
query22	2.28	2.16	1.73
query23	17.11	1.13	0.95
query24	7.97	0.72	0.54
query25	0.38	0.09	0.07
query26	0.72	0.16	0.14
query27	0.04	0.04	0.05
query28	5.97	0.73	0.75
query29	12.68	2.29	2.31
query30	0.55	0.53	0.52
query31	2.82	0.38	0.38
query32	3.38	0.49	0.49
query33	3.07	3.04	3.07
query34	15.26	4.79	4.76
query35	4.83	4.83	4.85
query36	1.05	1.01	1.01
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.03	0.02	0.02
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.89 s
Total hot run time: 30.47 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit cb1808a3924f62b57926e08509bb31647acff685 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@xiaokang xiaokang merged commit 1079af3 into apache:branch-2.0 Jul 28, 2024
21 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants