Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore](explain) Add algorithm item to VSORT explainition and modify dump_data of Block #38543

Merged
merged 1 commit into from
Aug 1, 2024

Conversation

zclllyybb
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

  1. add algorithm record in VSORT's explain string:

before:

|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |

after:

|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  algorithm: full sort      |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
  1. add a new parameter to make Block::dump_data() could work when the nullity of the type of data is different from that of column. it's useful when in some function call under default_implementation_for_nulls()

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 42193 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6efd02da5470e5fa59077340358a0f3c71d59a15, data reload: false

------ Round 1 ----------------------------------
q1	18152	4338	4165	4165
q2	2236	207	208	207
q3	10473	1293	1340	1293
q4	10163	850	973	850
q5	7746	3068	3045	3045
q6	225	140	143	140
q7	1069	630	627	627
q8	9486	1914	2000	1914
q9	8646	6705	6745	6705
q10	8738	3936	3866	3866
q11	430	261	268	261
q12	410	231	233	231
q13	17763	2977	2948	2948
q14	272	246	241	241
q15	528	493	499	493
q16	492	409	387	387
q17	998	924	917	917
q18	8091	7436	7307	7307
q19	1395	1231	1210	1210
q20	566	321	341	321
q21	5335	4773	4904	4773
q22	364	292	292	292
Total cold run time: 113578 ms
Total hot run time: 42193 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4027	4019	4012	4012
q2	327	225	217	217
q3	3027	3003	2995	2995
q4	1915	1894	1903	1894
q5	5280	5279	5232	5232
q6	215	130	134	130
q7	2061	1658	1718	1658
q8	3239	3333	3299	3299
q9	8353	8379	8386	8379
q10	3778	3910	3826	3826
q11	554	451	459	451
q12	739	544	552	544
q13	14627	2963	2967	2963
q14	290	252	253	252
q15	522	484	480	480
q16	445	390	401	390
q17	1724	1730	1706	1706
q18	7728	7379	7446	7379
q19	1678	1669	1664	1664
q20	1971	1758	1774	1758
q21	5612	5340	5246	5246
q22	520	447	479	447
Total cold run time: 68632 ms
Total hot run time: 54922 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169050 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6efd02da5470e5fa59077340358a0f3c71d59a15, data reload: false

query1	917	379	367	367
query2	6490	1689	1651	1651
query3	6695	214	223	214
query4	19554	17505	17535	17505
query5	4250	517	510	510
query6	286	184	186	184
query7	4602	308	300	300
query8	254	216	218	216
query9	8505	2382	2357	2357
query10	445	278	271	271
query11	10567	9967	10077	9967
query12	132	82	86	82
query13	1640	375	370	370
query14	10188	7867	7027	7027
query15	205	158	164	158
query16	7125	443	429	429
query17	944	572	536	536
query18	1934	276	282	276
query19	194	150	151	150
query20	93	89	86	86
query21	204	101	100	100
query22	4198	3907	3940	3907
query23	33644	32995	33253	32995
query24	10367	3079	3011	3011
query25	699	404	414	404
query26	1750	152	152	152
query27	2923	268	281	268
query28	6978	1967	1956	1956
query29	1367	437	428	428
query30	295	153	153	153
query31	921	759	751	751
query32	103	59	59	59
query33	715	310	323	310
query34	895	467	483	467
query35	872	729	732	729
query36	1004	846	872	846
query37	298	81	84	81
query38	2897	2757	2752	2752
query39	880	816	802	802
query40	286	115	119	115
query41	47	46	47	46
query42	127	107	101	101
query43	462	422	414	414
query44	1217	734	738	734
query45	206	182	182	182
query46	1080	796	775	775
query47	1813	1685	1703	1685
query48	363	293	288	288
query49	1206	452	436	436
query50	915	435	446	435
query51	6679	6733	6634	6634
query52	112	90	93	90
query53	253	183	179	179
query54	640	464	452	452
query55	80	78	79	78
query56	299	262	274	262
query57	1152	1044	1047	1044
query58	291	279	276	276
query59	2576	2425	2392	2392
query60	302	280	373	280
query61	105	97	96	96
query62	900	660	646	646
query63	212	189	184	184
query64	5930	1918	1921	1918
query65	3207	3085	3095	3085
query66	1439	343	370	343
query67	15175	14837	14918	14837
query68	4527	546	576	546
query69	644	380	314	314
query70	1105	1083	1108	1083
query71	456	275	279	275
query72	7806	2742	2513	2513
query73	786	327	329	327
query74	6034	5644	5688	5644
query75	3632	2718	2751	2718
query76	2887	1302	1420	1302
query77	623	315	307	307
query78	9482	9025	8916	8916
query79	2462	593	527	527
query80	1536	524	506	506
query81	554	224	227	224
query82	1040	135	130	130
query83	241	176	180	176
query84	261	80	79	79
query85	1478	338	315	315
query86	471	323	305	305
query87	3230	3088	3143	3088
query88	3663	2383	2391	2383
query89	410	290	290	290
query90	1895	237	195	195
query91	127	103	101	101
query92	64	48	54	48
query93	2223	606	611	606
query94	890	302	289	289
query95	390	262	273	262
query96	598	280	278	278
query97	3201	3025	2975	2975
query98	213	209	194	194
query99	1587	1332	1300	1300
Total cold run time: 267620 ms
Total hot run time: 169050 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6efd02da5470e5fa59077340358a0f3c71d59a15, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.05	0.04
query4	1.69	0.08	0.07
query5	0.48	0.48	0.48
query6	1.15	0.71	0.70
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.56	0.51	0.54
query10	0.57	0.54	0.56
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.62	0.59	0.60
query14	0.77	0.81	0.80
query15	0.89	0.86	0.86
query16	0.35	0.36	0.35
query17	1.02	1.01	0.98
query18	0.22	0.21	0.21
query19	1.86	1.73	1.74
query20	0.01	0.01	0.01
query21	15.43	0.77	0.68
query22	4.06	8.92	1.02
query23	17.88	1.33	1.35
query24	2.28	0.23	0.21
query25	0.17	0.08	0.08
query26	0.33	0.21	0.21
query27	0.46	0.23	0.23
query28	13.18	1.00	0.97
query29	12.51	3.34	3.30
query30	0.26	0.05	0.05
query31	2.87	0.40	0.40
query32	3.22	0.49	0.49
query33	2.96	2.93	2.93
query34	15.46	4.24	4.26
query35	4.29	4.26	4.25
query36	0.67	0.48	0.49
query37	0.18	0.16	0.16
query38	0.17	0.16	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.10	0.05	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 107.65 s
Total hot run time: 29.7 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 31, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BiteTheDDDDt BiteTheDDDDt merged commit 8ee24cb into apache:master Aug 1, 2024
28 of 31 checks passed
feiniaofeiafei pushed a commit to feiniaofeiafei/doris that referenced this pull request Aug 9, 2024
…dump_data of Block (apache#38543)

## Proposed changes

Issue Number: close #xxx

1.  add algorithm record in `VSORT`'s explain string:

before:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
after:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  algorithm: full sort      |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
2. add a new parameter to make `Block::dump_data()` could work when the
nullity of the type of data is different from that of column. it's
useful when in some function call under
`default_implementation_for_nulls()`
dataroaring pushed a commit that referenced this pull request Aug 11, 2024
…dump_data of Block (#38543)

## Proposed changes

Issue Number: close #xxx

1.  add algorithm record in `VSORT`'s explain string:

before:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
after:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  algorithm: full sort      |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
2. add a new parameter to make `Block::dump_data()` could work when the
nullity of the type of data is different from that of column. it's
useful when in some function call under
`default_implementation_for_nulls()`
dataroaring pushed a commit that referenced this pull request Aug 16, 2024
…dump_data of Block (#38543)

## Proposed changes

Issue Number: close #xxx

1.  add algorithm record in `VSORT`'s explain string:

before:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
after:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  algorithm: full sort      |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
2. add a new parameter to make `Block::dump_data()` could work when the
nullity of the type of data is different from that of column. it's
useful when in some function call under
`default_implementation_for_nulls()`
@zclllyybb zclllyybb deleted the sort_chore branch August 23, 2024 09:55
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Aug 23, 2024
…dump_data of Block (apache#38543)

## Proposed changes

Issue Number: close #xxx

1.  add algorithm record in `VSORT`'s explain string:

before:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
after:
```sql
|   1:VSORT(101)                 |
|   |  order by: xxx             |
|   |  algorithm: full sort      |
|   |  offset: 0                 |
|   |  distribute expr lists:    |
|   |  tuple ids: 2              |
```
2. add a new parameter to make `Block::dump_data()` could work when the
nullity of the type of data is different from that of column. it's
useful when in some function call under
`default_implementation_for_nulls()`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants