Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement](partition) Forbid create table with null partition item which relative column is not null #39449

Merged
merged 3 commits into from
Aug 19, 2024

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Aug 15, 2024

Proposed changes

Issue Number: close #xxx

before:

CREATE TABLE `test_null` (
`k0` BIGINT NOT NULL,
`k1` BIGINT NOT NULL
)
partition by list (k0, k1) (
PARTITION `pX` values in ((NULL, 1))
)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);

may core in local exchange for inserting.

now:

mysql [test]>CREATE TABLE `test_null` (
    -> `k0` BIGINT NOT NULL,
    -> `k1` BIGINT NOT NULL
    -> )
    -> partition by list (k0, k1) (
    -> PARTITION `pX` values in ((NULL, 1))
    -> )
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Can't have null partition is for NOT NULL partition column in partition expr's index 0

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb zclllyybb marked this pull request as ready for review August 15, 2024 14:40
@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38355 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 58cb1b3827cf9718259403884b86dd3006f11ec7, data reload: false

------ Round 1 ----------------------------------
q1	18464	4519	4389	4389
q2	2684	218	231	218
q3	11344	1298	1131	1131
q4	10756	745	771	745
q5	8070	2865	2796	2796
q6	265	156	157	156
q7	993	646	647	646
q8	9380	2078	2044	2044
q9	7358	6547	6525	6525
q10	7062	2178	2235	2178
q11	497	266	269	266
q12	422	255	244	244
q13	17796	2983	2976	2976
q14	296	250	248	248
q15	559	518	517	517
q16	522	403	404	403
q17	979	661	737	661
q18	7386	6839	6828	6828
q19	5411	1110	1069	1069
q20	705	358	366	358
q21	3861	3038	2920	2920
q22	1154	1037	1051	1037
Total cold run time: 115964 ms
Total hot run time: 38355 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4479	4287	4268	4268
q2	412	295	294	294
q3	2862	2647	2623	2623
q4	1863	1625	1648	1625
q5	5409	5408	5376	5376
q6	239	151	150	150
q7	2090	1691	1698	1691
q8	3223	3388	3355	3355
q9	8463	8385	8395	8385
q10	3426	3143	3158	3143
q11	647	547	533	533
q12	805	639	662	639
q13	16616	2974	3027	2974
q14	330	306	301	301
q15	572	523	516	516
q16	510	455	464	455
q17	1796	1504	1477	1477
q18	7694	7690	7381	7381
q19	1744	1627	1563	1563
q20	2104	1853	1827	1827
q21	13499	5140	5122	5122
q22	1094	1043	1025	1025
Total cold run time: 79877 ms
Total hot run time: 54723 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190112 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 58cb1b3827cf9718259403884b86dd3006f11ec7, data reload: false

query1	1002	405	392	392
query2	6741	1929	1903	1903
query3	6667	231	235	231
query4	34116	23528	23259	23259
query5	4414	642	639	639
query6	307	200	200	200
query7	4602	312	313	312
query8	416	357	362	357
query9	8629	2489	2453	2453
query10	481	307	315	307
query11	16073	14952	14944	14944
query12	173	145	125	125
query13	1681	421	412	412
query14	10079	6673	6764	6673
query15	254	184	189	184
query16	7709	473	504	473
query17	1626	590	587	587
query18	1921	335	338	335
query19	219	166	163	163
query20	140	130	132	130
query21	254	143	141	141
query22	4255	4125	4006	4006
query23	34148	33397	33549	33397
query24	11324	2942	2860	2860
query25	664	410	416	410
query26	1274	178	174	174
query27	2343	295	298	295
query28	7199	2115	2111	2111
query29	893	441	441	441
query30	334	186	185	185
query31	1037	850	834	834
query32	126	75	77	75
query33	808	341	336	336
query34	921	491	511	491
query35	931	752	756	752
query36	1145	953	950	950
query37	169	102	101	101
query38	3995	3872	3848	3848
query39	1515	1457	1440	1440
query40	235	152	149	149
query41	139	134	177	134
query42	135	115	118	115
query43	525	501	483	483
query44	1215	776	781	776
query45	218	192	193	192
query46	1139	769	757	757
query47	1904	1838	1851	1838
query48	402	342	338	338
query49	1220	602	573	573
query50	865	456	455	455
query51	6966	6761	6685	6685
query52	118	113	109	109
query53	296	223	227	223
query54	939	501	495	495
query55	95	90	90	90
query56	320	307	322	307
query57	1222	1133	1132	1132
query58	308	346	332	332
query59	3032	2842	2812	2812
query60	350	331	326	326
query61	150	145	149	145
query62	863	687	703	687
query63	254	228	232	228
query64	5348	2450	1848	1848
query65	3224	3205	3178	3178
query66	1101	663	712	663
query67	15640	14891	15043	14891
query68	7901	592	592	592
query69	704	421	351	351
query70	1246	1159	1154	1154
query71	562	311	311	311
query72	7545	2353	2079	2079
query73	858	365	355	355
query74	9142	8798	8831	8798
query75	4833	2765	2715	2715
query76	5004	1089	1107	1089
query77	905	504	434	434
query78	9906	9231	9424	9231
query79	6871	553	551	551
query80	1161	630	623	623
query81	632	266	252	252
query82	654	154	152	152
query83	370	213	207	207
query84	292	97	101	97
query85	1486	357	377	357
query86	410	324	319	319
query87	4492	4246	4329	4246
query88	4509	2515	2529	2515
query89	446	343	326	326
query90	2058	226	269	226
query91	148	126	125	125
query92	94	75	74	74
query93	4767	550	548	548
query94	969	321	323	321
query95	385	294	294	294
query96	625	287	281	281
query97	3203	3077	3127	3077
query98	252	240	237	237
query99	1595	1288	1315	1288
Total cold run time: 312833 ms
Total hot run time: 190112 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.82 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 58cb1b3827cf9718259403884b86dd3006f11ec7, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.10	0.10
query5	0.50	0.49	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.04
query9	0.56	0.49	0.50
query10	0.55	0.55	0.55
query11	0.15	0.12	0.12
query12	0.16	0.12	0.14
query13	0.61	0.59	0.58
query14	0.78	0.78	0.78
query15	0.84	0.83	0.82
query16	0.39	0.34	0.36
query17	1.03	1.07	1.01
query18	0.24	0.23	0.24
query19	1.86	1.69	1.80
query20	0.02	0.01	0.01
query21	15.94	0.84	0.65
query22	4.14	6.67	2.48
query23	18.29	1.35	1.29
query24	2.16	0.23	0.22
query25	0.15	0.09	0.09
query26	0.30	0.22	0.21
query27	0.46	0.23	0.22
query28	13.30	1.03	1.00
query29	12.70	3.33	3.31
query30	0.44	0.24	0.24
query31	2.82	0.40	0.41
query32	3.24	0.49	0.49
query33	2.96	2.98	2.99
query34	17.13	4.36	4.37
query35	4.46	4.47	4.39
query36	0.68	0.51	0.49
query37	0.20	0.17	0.18
query38	0.17	0.17	0.17
query39	0.06	0.06	0.06
query40	0.19	0.16	0.15
query41	0.11	0.07	0.07
query42	0.07	0.07	0.07
query43	0.07	0.07	0.06
Total cold run time: 110.97 s
Total hot run time: 31.82 s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38502 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4fcb1f5b0371478b48e6db64f242fdc2515fa11a, data reload: false

------ Round 1 ----------------------------------
q1	15480	5993	4315	4315
q2	1392	223	228	223
q3	10316	1203	1073	1073
q4	9962	776	752	752
q5	6568	2856	2849	2849
q6	268	161	158	158
q7	1007	690	654	654
q8	7562	2130	2085	2085
q9	6654	6544	6533	6533
q10	5987	2309	2339	2309
q11	426	269	279	269
q12	435	264	257	257
q13	18493	3043	3003	3003
q14	304	264	261	261
q15	561	539	533	533
q16	524	417	424	417
q17	985	690	675	675
q18	7570	7019	6827	6827
q19	6623	1129	1062	1062
q20	686	353	362	353
q21	3871	2992	2846	2846
q22	1136	1064	1048	1048
Total cold run time: 106810 ms
Total hot run time: 38502 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4512	4316	4323	4316
q2	422	315	313	313
q3	2879	2673	2644	2644
q4	1949	1707	1671	1671
q5	5367	5400	5414	5400
q6	235	149	151	149
q7	2110	1718	1714	1714
q8	3221	3400	3364	3364
q9	8457	8422	8438	8422
q10	3439	3217	3182	3182
q11	625	525	513	513
q12	827	648	640	640
q13	17649	3007	3001	3001
q14	318	283	299	283
q15	550	512	518	512
q16	505	440	434	434
q17	1779	1501	1500	1500
q18	7892	7510	7386	7386
q19	4105	1715	1527	1527
q20	2097	1815	1822	1815
q21	13850	5013	5081	5013
q22	1113	1047	1048	1047
Total cold run time: 83901 ms
Total hot run time: 54846 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190879 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4fcb1f5b0371478b48e6db64f242fdc2515fa11a, data reload: false

query1	975	393	393	393
query2	6738	2050	1956	1956
query3	6668	228	250	228
query4	33729	23175	23185	23175
query5	4335	652	624	624
query6	305	190	192	190
query7	4611	325	326	325
query8	474	434	425	425
query9	8721	2549	2533	2533
query10	501	347	340	340
query11	15794	15034	14967	14967
query12	173	125	121	121
query13	1691	430	423	423
query14	9283	7177	7131	7131
query15	245	193	186	186
query16	7785	546	549	546
query17	1615	594	580	580
query18	1989	338	342	338
query19	216	162	168	162
query20	145	137	140	137
query21	245	141	139	139
query22	4246	3945	3972	3945
query23	34069	33345	33471	33345
query24	11597	2987	2943	2943
query25	706	467	461	461
query26	1209	191	193	191
query27	2885	332	322	322
query28	7745	2207	2181	2181
query29	946	519	447	447
query30	345	186	184	184
query31	1044	832	817	817
query32	122	80	84	80
query33	815	343	329	329
query34	899	520	493	493
query35	865	784	754	754
query36	1109	972	960	960
query37	173	107	104	104
query38	3868	3863	3832	3832
query39	1526	1465	1456	1456
query40	237	155	152	152
query41	139	136	163	136
query42	141	117	117	117
query43	561	511	531	511
query44	1270	788	809	788
query45	223	194	194	194
query46	1128	746	782	746
query47	1940	1833	1857	1833
query48	415	337	338	337
query49	1247	589	580	580
query50	863	461	469	461
query51	6733	6742	6781	6742
query52	122	113	107	107
query53	301	236	233	233
query54	967	501	496	496
query55	88	89	92	89
query56	336	306	314	306
query57	1224	1127	1093	1093
query58	315	304	307	304
query59	3135	2944	2914	2914
query60	352	328	326	326
query61	150	147	145	145
query62	896	693	704	693
query63	256	227	227	227
query64	5561	2372	1870	1870
query65	3216	3198	3172	3172
query66	1301	672	664	664
query67	15637	14894	14836	14836
query68	6395	597	593	593
query69	796	416	326	326
query70	1215	1147	1162	1147
query71	562	313	307	307
query72	7601	2337	2113	2113
query73	870	361	372	361
query74	9306	8758	8817	8758
query75	4709	2779	2782	2779
query76	4457	1016	1079	1016
query77	927	442	434	434
query78	9981	9131	9157	9131
query79	7964	562	557	557
query80	1130	621	622	621
query81	614	259	259	259
query82	803	156	158	156
query83	391	212	217	212
query84	292	98	95	95
query85	1472	361	393	361
query86	462	330	334	330
query87	4392	4173	4178	4173
query88	4785	2567	2533	2533
query89	457	329	318	318
query90	2071	233	236	233
query91	156	130	131	130
query92	88	76	74	74
query93	5801	580	562	562
query94	1029	326	316	316
query95	395	298	299	298
query96	632	290	290	290
query97	3272	3044	3108	3044
query98	244	232	226	226
query99	1579	1308	1291	1291
Total cold run time: 314009 ms
Total hot run time: 190879 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4fcb1f5b0371478b48e6db64f242fdc2515fa11a, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.08	0.07
query5	0.48	0.51	0.50
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.05	0.05
query9	0.55	0.50	0.48
query10	0.56	0.53	0.55
query11	0.16	0.12	0.12
query12	0.16	0.13	0.13
query13	0.63	0.59	0.59
query14	0.79	0.77	0.79
query15	0.86	0.83	0.81
query16	0.38	0.36	0.36
query17	1.00	0.99	1.00
query18	0.23	0.22	0.23
query19	1.86	1.70	1.71
query20	0.01	0.01	0.01
query21	15.40	0.69	0.69
query22	3.97	8.20	1.68
query23	18.30	1.37	1.28
query24	2.09	0.22	0.22
query25	0.16	0.09	0.08
query26	0.31	0.24	0.23
query27	0.46	0.22	0.22
query28	13.30	1.02	1.02
query29	12.61	3.36	3.37
query30	0.44	0.24	0.24
query31	2.83	0.40	0.39
query32	3.25	0.49	0.49
query33	2.93	2.96	2.95
query34	17.14	4.39	4.36
query35	4.40	4.38	4.45
query36	0.68	0.49	0.49
query37	0.21	0.17	0.18
query38	0.18	0.17	0.18
query39	0.06	0.05	0.05
query40	0.18	0.15	0.16
query41	0.12	0.07	0.07
query42	0.08	0.07	0.07
query43	0.07	0.06	0.06
Total cold run time: 110.07 s
Total hot run time: 31.01 s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37878 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a121d6d7254e7118f6856d284ef9ca7089297520, data reload: false

------ Round 1 ----------------------------------
q1	17866	4339	4257	4257
q2	2059	206	209	206
q3	11779	1003	1070	1003
q4	10522	809	800	800
q5	7755	2806	2762	2762
q6	264	156	154	154
q7	1018	657	655	655
q8	9590	2089	2128	2089
q9	8773	6528	6530	6528
q10	7055	2282	2234	2234
q11	506	270	266	266
q12	438	243	245	243
q13	17774	3029	3012	3012
q14	287	257	253	253
q15	537	511	527	511
q16	517	435	400	400
q17	984	772	681	681
q18	7412	6770	6798	6770
q19	7410	1045	1017	1017
q20	851	359	351	351
q21	3813	2928	2658	2658
q22	1142	1034	1028	1028
Total cold run time: 118352 ms
Total hot run time: 37878 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4543	4285	4320	4285
q2	418	304	298	298
q3	2843	2650	2639	2639
q4	1994	1733	1695	1695
q5	5705	5706	5693	5693
q6	243	152	161	152
q7	2249	1784	1803	1784
q8	3316	3493	3466	3466
q9	8846	8795	8768	8768
q10	3585	3355	3277	3277
q11	617	540	535	535
q12	831	685	676	676
q13	17027	3184	3185	3184
q14	329	297	284	284
q15	557	506	511	506
q16	503	451	465	451
q17	1829	1539	1546	1539
q18	8151	7904	7700	7700
q19	8056	1601	1612	1601
q20	2167	1891	1930	1891
q21	13841	5383	5494	5383
q22	1176	1084	1082	1082
Total cold run time: 88826 ms
Total hot run time: 56889 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196944 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a121d6d7254e7118f6856d284ef9ca7089297520, data reload: false

query1	1299	892	879	879
query2	6567	2064	2016	2016
query3	10635	3897	3753	3753
query4	59215	24692	23314	23314
query5	5772	751	718	718
query6	554	222	234	222
query7	6405	341	335	335
query8	545	448	443	443
query9	9363	2553	2533	2533
query10	587	354	343	343
query11	18232	14995	15483	14995
query12	207	152	144	144
query13	1704	448	445	445
query14	11683	7556	7514	7514
query15	336	204	195	195
query16	7426	553	518	518
query17	1186	614	610	610
query18	2026	362	346	346
query19	310	182	173	173
query20	154	141	142	141
query21	254	150	162	150
query22	4530	4375	4576	4375
query23	34437	34407	33933	33933
query24	5737	3010	3066	3010
query25	578	434	425	425
query26	711	183	193	183
query27	1715	309	317	309
query28	3708	2212	2178	2178
query29	713	460	450	450
query30	243	200	191	191
query31	1032	896	821	821
query32	104	82	83	82
query33	563	344	341	341
query34	917	537	507	507
query35	886	792	774	774
query36	1099	973	977	973
query37	157	103	103	103
query38	3890	3806	3857	3806
query39	1518	1477	1490	1477
query40	234	164	155	155
query41	143	139	141	139
query42	133	118	120	118
query43	558	530	526	526
query44	1143	795	792	792
query45	229	200	203	200
query46	1135	791	783	783
query47	1908	1839	1834	1834
query48	415	336	335	335
query49	942	591	596	591
query50	873	475	512	475
query51	6902	6840	6695	6695
query52	122	110	113	110
query53	300	228	235	228
query54	630	511	508	508
query55	93	90	92	90
query56	332	309	310	309
query57	1239	1135	1154	1135
query58	298	312	302	302
query59	3120	2824	2867	2824
query60	355	324	331	324
query61	152	152	146	146
query62	759	672	705	672
query63	264	228	227	227
query64	3420	1845	1873	1845
query65	3262	3186	3240	3186
query66	1010	684	681	681
query67	15301	14857	14708	14708
query68	5339	585	589	585
query69	580	324	314	314
query70	1200	1200	1220	1200
query71	504	312	315	312
query72	7009	2300	2145	2145
query73	833	358	361	358
query74	9355	8821	8982	8821
query75	3797	2783	2761	2761
query76	3470	1132	989	989
query77	696	461	451	451
query78	9995	9254	9488	9254
query79	3200	559	557	557
query80	2103	619	623	619
query81	586	269	261	261
query82	777	166	162	162
query83	352	225	225	225
query84	290	97	105	97
query85	1234	409	358	358
query86	362	341	322	322
query87	4383	4143	4264	4143
query88	4653	2499	2512	2499
query89	452	327	323	323
query90	2017	240	240	240
query91	160	136	132	132
query92	91	79	79	79
query93	5353	551	553	551
query94	826	335	325	325
query95	402	299	306	299
query96	636	295	294	294
query97	3297	3105	3044	3044
query98	251	236	232	232
query99	1666	1329	1364	1329
Total cold run time: 328139 ms
Total hot run time: 196944 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a121d6d7254e7118f6856d284ef9ca7089297520, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.05
query3	0.22	0.06	0.06
query4	1.66	0.08	0.08
query5	0.51	0.50	0.49
query6	1.14	0.73	0.74
query7	0.02	0.02	0.02
query8	0.06	0.05	0.05
query9	0.55	0.49	0.47
query10	0.54	0.55	0.57
query11	0.16	0.13	0.12
query12	0.16	0.14	0.13
query13	0.61	0.59	0.59
query14	0.76	0.78	0.80
query15	0.85	0.82	0.83
query16	0.37	0.36	0.38
query17	0.97	1.00	1.03
query18	0.22	0.21	0.22
query19	1.83	1.69	1.68
query20	0.01	0.01	0.02
query21	15.39	0.66	0.66
query22	3.50	8.69	1.83
query23	18.30	1.35	1.24
query24	2.18	0.21	0.22
query25	0.16	0.09	0.09
query26	0.31	0.23	0.23
query27	0.45	0.22	0.23
query28	13.27	1.03	1.03
query29	12.66	3.34	3.28
query30	0.42	0.25	0.25
query31	2.81	0.40	0.39
query32	3.24	0.49	0.49
query33	2.95	2.98	2.99
query34	17.13	4.32	4.38
query35	4.43	4.43	4.41
query36	0.69	0.48	0.47
query37	0.22	0.17	0.17
query38	0.16	0.16	0.18
query39	0.07	0.06	0.05
query40	0.19	0.16	0.15
query41	0.12	0.07	0.06
query42	0.08	0.07	0.07
query43	0.07	0.06	0.06
Total cold run time: 109.58 s
Total hot run time: 31.06 s

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37985 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8f2b82b07ef71b034891ad557c59b71cde101d60, data reload: false

------ Round 1 ----------------------------------
q1	17847	4313	4295	4295
q2	2042	230	210	210
q3	11851	1002	1135	1002
q4	10530	808	708	708
q5	7798	2845	2768	2768
q6	263	158	155	155
q7	1013	653	657	653
q8	9591	2093	2082	2082
q9	8637	6533	6532	6532
q10	7071	2219	2205	2205
q11	482	276	271	271
q12	431	255	262	255
q13	17777	2985	2996	2985
q14	291	258	257	257
q15	551	533	547	533
q16	521	430	410	410
q17	967	651	752	651
q18	7411	6827	6650	6650
q19	6911	1081	1009	1009
q20	686	359	362	359
q21	3991	2942	3091	2942
q22	1136	1053	1069	1053
Total cold run time: 117798 ms
Total hot run time: 37985 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4559	4317	4326	4317
q2	417	314	311	311
q3	2821	2679	2668	2668
q4	1982	1700	1706	1700
q5	5665	5726	5667	5667
q6	245	147	145	145
q7	2177	1825	1761	1761
q8	3285	3479	3439	3439
q9	8729	8674	8746	8674
q10	3603	3380	3318	3318
q11	620	516	522	516
q12	826	655	681	655
q13	16699	3173	3082	3082
q14	315	295	292	292
q15	551	520	525	520
q16	494	450	459	450
q17	1804	1581	1508	1508
q18	8305	7921	7658	7658
q19	5606	1526	1727	1526
q20	2164	1911	1891	1891
q21	13881	5310	5364	5310
q22	1207	1130	1070	1070
Total cold run time: 85955 ms
Total hot run time: 56478 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195999 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8f2b82b07ef71b034891ad557c59b71cde101d60, data reload: false

query1	1310	893	897	893
query2	6609	2061	1972	1972
query3	10647	3909	3672	3672
query4	57560	24075	23197	23197
query5	6019	660	659	659
query6	518	196	199	196
query7	6409	333	321	321
query8	532	437	451	437
query9	9375	2537	2511	2511
query10	596	369	335	335
query11	18479	15123	15290	15123
query12	220	148	125	125
query13	1665	445	467	445
query14	12326	7491	7356	7356
query15	264	204	211	204
query16	7527	504	496	496
query17	1162	629	591	591
query18	2041	344	346	344
query19	320	169	165	165
query20	152	140	154	140
query21	257	145	144	144
query22	4680	4301	4320	4301
query23	34541	33796	33992	33796
query24	5737	2955	3023	2955
query25	575	426	450	426
query26	740	180	182	180
query27	1943	308	318	308
query28	3983	2186	2155	2155
query29	710	448	441	441
query30	226	191	190	190
query31	1018	791	826	791
query32	96	75	76	75
query33	518	339	338	338
query34	918	505	489	489
query35	852	782	761	761
query36	1101	981	942	942
query37	155	102	99	99
query38	4069	3850	3901	3850
query39	1549	1451	1470	1451
query40	236	153	150	150
query41	137	141	136	136
query42	141	118	118	118
query43	550	533	529	529
query44	1112	784	791	784
query45	223	191	192	191
query46	1108	773	788	773
query47	1905	1797	1838	1797
query48	402	339	330	330
query49	903	580	571	571
query50	875	454	456	454
query51	6869	6832	6828	6828
query52	122	106	109	106
query53	317	223	221	221
query54	597	508	497	497
query55	96	91	86	86
query56	340	306	305	305
query57	1231	1128	1134	1128
query58	296	317	297	297
query59	3239	3033	3154	3033
query60	349	332	336	332
query61	159	167	141	141
query62	775	685	697	685
query63	255	226	224	224
query64	3300	1880	1854	1854
query65	3250	3199	3180	3180
query66	1084	673	655	655
query67	15382	14923	14851	14851
query68	6013	573	592	573
query69	514	317	312	312
query70	1211	1150	1186	1150
query71	541	315	311	311
query72	6715	2331	2129	2129
query73	818	359	357	357
query74	9383	8815	8846	8815
query75	3810	2761	2699	2699
query76	3357	1000	1034	1000
query77	672	437	430	430
query78	9953	9336	9097	9097
query79	2293	555	566	555
query80	2629	598	606	598
query81	621	261	263	261
query82	809	163	162	162
query83	355	222	211	211
query84	300	96	106	96
query85	1251	361	354	354
query86	498	335	314	314
query87	4402	4240	4206	4206
query88	3925	2516	2487	2487
query89	439	324	325	324
query90	1981	238	232	232
query91	154	129	127	127
query92	89	74	75	74
query93	3275	554	560	554
query94	958	309	330	309
query95	383	288	286	286
query96	611	285	284	284
query97	3264	3034	3069	3034
query98	253	247	230	230
query99	1609	1298	1326	1298
Total cold run time: 325738 ms
Total hot run time: 195999 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.26 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8f2b82b07ef71b034891ad557c59b71cde101d60, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.05	0.05
query3	0.23	0.05	0.05
query4	1.67	0.07	0.08
query5	0.51	0.48	0.50
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.56	0.50	0.50
query10	0.57	0.54	0.55
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.63	0.60	0.59
query14	0.76	0.78	0.79
query15	0.86	0.82	0.83
query16	0.38	0.36	0.38
query17	1.00	0.95	1.05
query18	0.22	0.22	0.22
query19	1.91	1.76	1.84
query20	0.01	0.02	0.01
query21	15.40	0.67	0.66
query22	4.40	7.07	2.07
query23	18.25	1.44	1.30
query24	2.14	0.23	0.22
query25	0.16	0.08	0.08
query26	0.31	0.24	0.23
query27	0.45	0.24	0.23
query28	13.27	1.02	1.00
query29	12.65	3.29	3.27
query30	0.42	0.19	0.19
query31	2.80	0.40	0.42
query32	3.23	0.48	0.48
query33	2.96	2.94	2.94
query34	16.99	4.38	4.33
query35	4.40	4.45	4.37
query36	0.67	0.52	0.49
query37	0.21	0.18	0.18
query38	0.18	0.16	0.16
query39	0.07	0.06	0.06
query40	0.18	0.15	0.16
query41	0.12	0.07	0.07
query42	0.08	0.08	0.07
query43	0.07	0.07	0.07
Total cold run time: 110.38 s
Total hot run time: 31.26 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 19, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@BiteTheDDDDt BiteTheDDDDt merged commit c26076e into apache:master Aug 19, 2024
28 of 30 checks passed
@zclllyybb zclllyybb deleted the check_partition_null branch August 19, 2024 12:21
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Aug 23, 2024
… which relative column is not null (apache#39449)

Issue Number: close #xxx

before:
```sql
CREATE TABLE `test_null` (
`k0` BIGINT NOT NULL,
`k1` BIGINT NOT NULL
)
partition by list (k0, k1) (
PARTITION `pX` values in ((NULL, 1))
)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
```
may core in local exchange for inserting.

now:
```sql
mysql [test]>CREATE TABLE `test_null` (
    -> `k0` BIGINT NOT NULL,
    -> `k1` BIGINT NOT NULL
    -> )
    -> partition by list (k0, k1) (
    -> PARTITION `pX` values in ((NULL, 1))
    -> )
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Can't have null partition is for NOT NULL partition column in partition expr's index 0
```
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Aug 23, 2024
… which relative column is not null (apache#39449)

Issue Number: close #xxx

before:
```sql
CREATE TABLE `test_null` (
`k0` BIGINT NOT NULL,
`k1` BIGINT NOT NULL
)
partition by list (k0, k1) (
PARTITION `pX` values in ((NULL, 1))
)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
```
may core in local exchange for inserting.

now:
```sql
mysql [test]>CREATE TABLE `test_null` (
    -> `k0` BIGINT NOT NULL,
    -> `k1` BIGINT NOT NULL
    -> )
    -> partition by list (k0, k1) (
    -> PARTITION `pX` values in ((NULL, 1))
    -> )
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Can't have null partition is for NOT NULL partition column in partition expr's index 0
```
yiguolei pushed a commit that referenced this pull request Aug 24, 2024
## Proposed changes

Issue Number: close #xxx

#38385 optimize parsing datetime
#38978 make stream load failure
message more clear and disable some error's stacktrace by default
#39255 fix random function coredump
#39324 fix function corr
inconsistency with doc
#39449 check auto partitoin nullity
when creating partition
#39695 make
DynamicPartitionScheduler immediately know interval's change
#39754 Add some partition expr check
on creating table
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
… which relative column is not null (#39449)

## Proposed changes

Issue Number: close #xxx

before:
```sql
CREATE TABLE `test_null` (
`k0` BIGINT NOT NULL,
`k1` BIGINT NOT NULL
)
partition by list (k0, k1) (
PARTITION `pX` values in ((NULL, 1))
)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
```
may core in local exchange for inserting.

now:
```sql
mysql [test]>CREATE TABLE `test_null` (
    -> `k0` BIGINT NOT NULL,
    -> `k1` BIGINT NOT NULL
    -> )
    -> partition by list (k0, k1) (
    -> PARTITION `pX` values in ((NULL, 1))
    -> )
    -> PROPERTIES (
    -> "replication_allocation" = "tag.location.default: 1"
    -> );
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Can't have null partition is for NOT NULL partition column in partition expr's index 0
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants