Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](schema-change) Fix potential data race when a schema change jobs is set to cancelled but the table state is still SCHEMA_CHANGE #39164

Merged

Conversation

TangSiyang2001
Copy link
Collaborator

Proposed changes

Set job cancel state after table state changed to normal.

…s set to cancelled but table state is still schema change
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@TangSiyang2001
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot added the doing label Aug 9, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 38924 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ac034efabc78a8cf6a9ff2ac990c43cfdb5b272, data reload: false

------ Round 1 ----------------------------------
q1	17633	4312	4239	4239
q2	2018	172	174	172
q3	10498	1158	1078	1078
q4	10148	695	686	686
q5	7515	2544	2446	2446
q6	219	140	139	139
q7	963	592	598	592
q8	9220	1886	1924	1886
q9	8755	6511	6541	6511
q10	7059	2223	2205	2205
q11	494	245	255	245
q12	462	221	218	218
q13	17770	3001	2983	2983
q14	284	240	232	232
q15	523	463	488	463
q16	508	383	389	383
q17	959	698	730	698
q18	8026	7458	7357	7357
q19	6253	1051	944	944
q20	660	336	345	336
q21	5707	4101	4332	4101
q22	1087	1028	1010	1010
Total cold run time: 116761 ms
Total hot run time: 38924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4511	4290	4257	4257
q2	379	273	265	265
q3	2813	2535	2675	2535
q4	1994	1669	1746	1669
q5	5525	5499	5424	5424
q6	224	131	138	131
q7	2079	1731	1781	1731
q8	3273	3457	3436	3436
q9	8736	8708	9016	8708
q10	3531	3318	3332	3318
q11	610	513	497	497
q12	788	630	628	628
q13	16970	3179	3164	3164
q14	319	275	305	275
q15	537	494	479	479
q16	501	442	428	428
q17	1856	1572	1544	1544
q18	8053	7960	7899	7899
q19	1744	1550	1596	1550
q20	2135	1893	1942	1893
q21	8881	5217	5322	5217
q22	1121	1026	994	994
Total cold run time: 76580 ms
Total hot run time: 56042 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202151 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ac034efabc78a8cf6a9ff2ac990c43cfdb5b272, data reload: false

query1	905	366	356	356
query2	6447	1932	1822	1822
query3	6638	213	221	213
query4	34103	23202	23093	23093
query5	3603	491	483	483
query6	279	166	166	166
query7	4584	297	284	284
query8	236	205	206	205
query9	8557	2495	2451	2451
query10	542	482	451	451
query11	15630	14916	14962	14916
query12	136	101	98	98
query13	1635	383	382	382
query14	10239	7723	7812	7723
query15	267	204	230	204
query16	7678	483	483	483
query17	1743	555	552	552
query18	1936	288	299	288
query19	222	155	156	155
query20	120	116	171	116
query21	205	106	106	106
query22	4603	4442	4359	4359
query23	34413	33597	34129	33597
query24	10599	2662	2633	2633
query25	611	401	400	400
query26	704	161	162	161
query27	2201	280	280	280
query28	5866	2046	2013	2013
query29	807	423	431	423
query30	250	157	150	150
query31	940	776	740	740
query32	93	55	53	53
query33	637	291	296	291
query34	861	480	470	470
query35	918	814	839	814
query36	1069	918	918	918
query37	131	80	83	80
query38	4222	4177	4053	4053
query39	1437	1522	1375	1375
query40	208	115	115	115
query41	48	45	45	45
query42	112	96	97	96
query43	502	460	461	460
query44	1082	730	731	730
query45	239	208	198	198
query46	1086	741	721	721
query47	1843	1753	1767	1753
query48	360	315	305	305
query49	847	422	435	422
query50	795	410	417	410
query51	6839	6776	6917	6776
query52	102	88	91	88
query53	253	185	175	175
query54	859	444	446	444
query55	75	77	78	77
query56	269	252	251	251
query57	1137	1087	1063	1063
query58	214	224	227	224
query59	2806	2747	2668	2668
query60	284	261	267	261
query61	111	92	91	91
query62	790	650	656	650
query63	208	176	179	176
query64	9188	2263	1767	1767
query65	3183	3171	3169	3169
query66	690	321	318	318
query67	15466	14782	14785	14782
query68	8491	565	565	565
query69	464	424	387	387
query70	1429	1124	1063	1063
query71	544	268	299	268
query72	20215	16529	16372	16372
query73	2346	331	331	331
query74	9133	8686	8876	8686
query75	4427	2654	2670	2654
query76	4847	1050	1009	1009
query77	746	300	313	300
query78	11179	9285	9606	9285
query79	12102	527	520	520
query80	1235	505	487	487
query81	584	230	224	224
query82	333	132	128	128
query83	314	141	146	141
query84	265	75	77	75
query85	723	273	320	273
query86	299	298	292	292
query87	4741	4520	4523	4520
query88	5273	2486	2519	2486
query89	436	279	288	279
query90	1839	192	194	192
query91	122	92	96	92
query92	61	48	49	48
query93	5390	548	536	536
query94	648	292	289	289
query95	340	258	262	258
query96	617	283	279	279
query97	3221	3069	3055	3055
query98	223	196	199	196
query99	1500	1269	1290	1269
Total cold run time: 329750 ms
Total hot run time: 202151 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.53 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ac034efabc78a8cf6a9ff2ac990c43cfdb5b272, data reload: false

query1	0.05	0.04	0.04
query2	0.09	0.03	0.04
query3	0.23	0.05	0.06
query4	1.67	0.07	0.07
query5	0.48	0.49	0.49
query6	1.14	0.72	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.13
query13	0.62	0.61	0.59
query14	0.78	0.78	0.78
query15	0.85	0.82	0.81
query16	0.34	0.36	0.36
query17	0.96	1.02	1.01
query18	0.23	0.22	0.22
query19	1.90	1.81	1.68
query20	0.01	0.00	0.01
query21	15.39	0.74	0.66
query22	4.13	7.76	1.89
query23	18.32	1.44	1.24
query24	2.10	0.23	0.22
query25	0.17	0.08	0.08
query26	0.29	0.21	0.22
query27	0.45	0.24	0.22
query28	13.25	1.02	0.99
query29	12.64	3.26	3.28
query30	0.24	0.05	0.05
query31	2.89	0.40	0.39
query32	3.26	0.48	0.47
query33	2.87	2.91	2.99
query34	17.03	4.32	4.39
query35	4.44	4.41	4.44
query36	0.66	0.47	0.49
query37	0.18	0.15	0.15
query38	0.15	0.16	0.15
query39	0.04	0.03	0.04
query40	0.17	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.68 s
Total hot run time: 30.53 s

@TangSiyang2001 TangSiyang2001 changed the title [fix](schema-change) fix potential data race when schema change job is set to cancelled but table state is still schema change [fix](schema-change) Fix potential data race when a schema change jobs is set to cancelled but the table state is still SCHEMA_CHANGE Aug 9, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 9, 2024
Copy link
Contributor

github-actions bot commented Aug 9, 2024

PR approved by at least one committer and no changes requested.

Copy link
Contributor

github-actions bot commented Aug 9, 2024

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit de31d99 into apache:master Aug 10, 2024
31 of 32 checks passed
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
…s is set to cancelled but the table state is still SCHEMA_CHANGE (#39164)

## Proposed changes

Set job cancel state after table state changed to normal.
TangSiyang2001 added a commit to TangSiyang2001/doris that referenced this pull request Aug 14, 2024
…s is set to cancelled but the table state is still SCHEMA_CHANGE (apache#39164)

Set job cancel state after table state changed to normal.
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
…s is set to cancelled but the table state is still SCHEMA_CHANGE (apache#39164)

## Proposed changes

Set job cancel state after table state changed to normal.
yiguolei pushed a commit that referenced this pull request Aug 15, 2024
…s is set to cancelled but the table state is still SCHEMA_CHANGE (#39164) (#39327)

## Proposed changes

Set job cancel state after table state changed to normal.
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
…s is set to cancelled but the table state is still SCHEMA_CHANGE (#39164)

## Proposed changes

Set job cancel state after table state changed to normal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.2-merged doing reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants