Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable key range iter #8084

Merged
merged 1 commit into from
Jul 18, 2024
Merged

enable key range iter #8084

merged 1 commit into from
Jul 18, 2024

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Jun 27, 2024

There was an issue merging #8025 ontop of a revert. Enable the key iteration optimization.

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

Copy link

Additional work is required for integration with DoltgreSQL.

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
db954ca ok 5937457
version total_tests
db954ca 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 is_faster
tpcc-scale-factor-1 75.82 74.46 0
test_name server_name server_version tps test_name server_name server_version tps is_faster
tpcc-scale-factor-1 dolt 605b4de 32.87 tpcc-scale-factor-1 dolt db954ca 32.9 0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency_median to_latency_median is_faster
covering_index_scan 2.91 2.81 0
groupby_scan 17.01 17.32 0
index_join 5.28 5.28 0
index_join_scan 2.57 2.57 0
index_scan 53.85 53.85 0
oltp_point_select 0.45 0.46 0
oltp_read_only 7.43 7.56 0
select_random_points 0.73 0.78 0
select_random_ranges 0.87 0.92 0
table_scan 55.82 55.82 0
types_table_scan 142.39 142.39 0
write_tests from_latency_median to_latency_median is_faster
oltp_delete_insert 5.99 6.09 0
oltp_insert 2.97 3.02 0
oltp_read_write 13.7 13.95 0
oltp_update_index 3.07 3.07 0
oltp_update_non_index 2.97 3.02 0
oltp_write_only 6.32 6.43 0
types_delete_insert 6.55 6.67 0

@max-hoffman max-hoffman merged commit 81e45ba into main Jul 18, 2024
35 of 36 checks passed
@max-hoffman max-hoffman deleted the max/key-range branch July 18, 2024 23:36
Copy link

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.06 1.33
batching batch sql 10000 1 0.08 1.75
batching by line sql 10000 1 0.08 1.88
blob 1 blob 200000 1 0.91 3.8 3.65
blob 2 blobs 200000 1 0.89 4.4 4.4
blob no blob 200000 1 0.91 2.42 2.04
col type datetime 200000 1 0.88 2.81 2.58
col type varchar 200000 1 0.7 3.33 2.69
config width 2 cols 200000 1 0.79 2.61 2.11
config width 32 cols 200000 1 1.83 1.99 2.5
config width 8 cols 200000 1 1.02 2.28 2.1
pk type float 200000 1 0.88 2.48 1.99
pk type int 200000 1 0.8 2.53 2.24
pk type varchar 200000 1 1.5 1.92 1.44
row count 1.6mm 1600000 1 5.6 3 2.51
row count 400k 400000 1 1.46 2.82 2.34
row count 800k 800000 1 2.78 3 2.49
secondary index four index 200000 1 3.51 1.4 1.09
secondary index no secondary 200000 1 0.88 2.49 2.11
secondary index one index 200000 1 1.16 2.34 2.06
secondary index two index 200000 1 1.96 1.77 1.44
sorting shuffled 1mm 1000000 0 4.86 2.92 2.55
sorting sorted 1mm 1000000 1 4.87 2.9 2.63

Copy link

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.27
dolt_blame_commit_filter system table 3.4
dolt_commit_ancestors_commit_filter system table 0.82
dolt_commits_commit_filter system table 0.89
dolt_diff_log_join_from_commit system table 2.01
dolt_diff_log_join_to_commit system table 2.02
dolt_diff_table_from_commit_filter system table 1.12
dolt_diff_table_to_commit_filter system table 1.1
dolt_diffs_commit_filter system table 0.97
dolt_history_commit_filter system table 1.21
dolt_log_commit_filter system table 0.94

Copy link

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 1.33
adds_updates_deletes 60000 60000 60000 4.43
deletes_only 0 60000 0 2.41
updates_only 0 0 60000 3.02

Copy link

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.06 1.5
batching batch sql 10000 1 0.08 1.75
batching by line sql 10000 1 0.08 1.75
blob 1 blob 200000 1 0.89 3.98 3.79
blob 2 blobs 200000 1 0.88 4.45 4.44
blob no blob 200000 1 0.88 2.55 2.11
col type datetime 200000 1 0.82 3.02 2.79
col type varchar 200000 1 0.68 3.51 3.16
config width 2 cols 200000 1 0.79 2.54 2.1
config width 32 cols 200000 1 1.87 1.97 2.47
config width 8 cols 200000 1 0.96 2.4 2.2
pk type float 200000 1 0.83 2.51 2.06
pk type int 200000 1 0.8 2.93 2.11
pk type varchar 200000 1 1.5 1.75 1.47
row count 1.6mm 1600000 1 5.68 2.94 2.47
row count 400k 400000 1 1.42 2.87 2.39
row count 800k 800000 1 2.76 3.01 2.49
secondary index four index 200000 1 3.54 1.39 1.08
secondary index no secondary 200000 1 0.87 2.54 2.13
secondary index one index 200000 1 1.12 2.42 2.13
secondary index two index 200000 1 1.95 1.77 1.42
sorting shuffled 1mm 1000000 0 5.25 2.81 2.51
sorting sorted 1mm 1000000 1 5.18 2.84 2.54

Copy link

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.25
dolt_blame_commit_filter system table 3.38
dolt_commit_ancestors_commit_filter system table 0.84
dolt_commits_commit_filter system table 0.89
dolt_diff_log_join_from_commit system table 2.01
dolt_diff_log_join_to_commit system table 2.01
dolt_diff_table_from_commit_filter system table 1.1
dolt_diff_table_to_commit_filter system table 1.12
dolt_diffs_commit_filter system table 0.97
dolt_history_commit_filter system table 1.2
dolt_log_commit_filter system table 0.91

Copy link

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 1.34
adds_updates_deletes 60000 60000 60000 4.39
deletes_only 0 60000 0 2.41
updates_only 0 0 60000 2.98

Copy link

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.05 1.8
batching batch sql 10000 1 0.06 2.17
batching by line sql 10000 1 0.06 2.33
blob 1 blob 200000 1 0.87 3.92 3.71
blob 2 blobs 200000 1 0.85 4.51 4.58
blob no blob 200000 1 0.88 2.5 2.1
col type datetime 200000 1 0.79 3.05 2.86
col type varchar 200000 1 0.66 3.47 2.86
config width 2 cols 200000 1 0.79 2.52 2.11
config width 32 cols 200000 1 1.76 2.06 2.58
config width 8 cols 200000 1 0.99 2.34 2.27
pk type float 200000 1 0.88 2.31 1.91
pk type int 200000 1 0.8 2.49 2.07
pk type varchar 200000 1 1.48 1.79 1.45
row count 1.6mm 1600000 1 5.47 3 2.53
row count 400k 400000 1 1.41 2.85 2.38
row count 800k 800000 1 2.75 2.94 2.48
secondary index four index 200000 1 3.41 1.43 1.12
secondary index no secondary 200000 1 0.88 2.45 2.1
secondary index one index 200000 1 1.08 2.5 2.17
secondary index two index 200000 1 1.97 1.75 1.41
sorting shuffled 1mm 1000000 0 4.81 2.93 2.54
sorting sorted 1mm 1000000 1 4.77 2.94 2.53

Copy link

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.25
dolt_blame_commit_filter system table 3.35
dolt_commit_ancestors_commit_filter system table 0.81
dolt_commits_commit_filter system table 0.89
dolt_diff_log_join_from_commit system table 1.98
dolt_diff_log_join_to_commit system table 2.04
dolt_diff_table_from_commit_filter system table 1.08
dolt_diff_table_to_commit_filter system table 1.1
dolt_diffs_commit_filter system table 0.97
dolt_history_commit_filter system table 1.2
dolt_log_commit_filter system table 0.91

Copy link

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 1.33
adds_updates_deletes 60000 60000 60000 4.56
deletes_only 0 60000 0 2.4
updates_only 0 0 60000 2.98

Copy link

@coffeegoddd DOLT

test_name detail row_cnt sorted mysql_time sql_mult cli_mult
batching LOAD DATA 10000 1 0.06 1.5
batching batch sql 10000 1 0.08 1.88
batching by line sql 10000 1 0.08 1.75
blob 1 blob 200000 1 0.89 3.88 3.7
blob 2 blobs 200000 1 0.88 4.41 4.39
blob no blob 200000 1 0.9 2.44 2.07
col type datetime 200000 1 0.82 3 2.82
col type varchar 200000 1 0.66 3.53 2.91
config width 2 cols 200000 1 0.78 2.59 2.29
config width 32 cols 200000 1 1.85 1.99 2.48
config width 8 cols 200000 1 0.97 2.39 3.35
pk type float 200000 1 0.86 2.36 2
pk type int 200000 1 0.82 2.48 2.05
pk type varchar 200000 1 1.61 1.62 1.35
row count 1.6mm 1600000 1 5.6 2.99 2.5
row count 400k 400000 1 1.44 2.83 2.37
row count 800k 800000 1 2.83 2.92 2.43
secondary index four index 200000 1 3.54 1.38 1.06
secondary index no secondary 200000 1 0.92 2.36 2.01
secondary index one index 200000 1 1.12 2.44 2.12
secondary index two index 200000 1 1.98 1.77 1.4
sorting shuffled 1mm 1000000 0 4.82 2.98 2.73
sorting sorted 1mm 1000000 1 4.95 2.88 2.57

Copy link

@coffeegoddd DOLT

name detail mean_mult
dolt_blame_basic system table 1.24
dolt_blame_commit_filter system table 3.35
dolt_commit_ancestors_commit_filter system table 0.86
dolt_commits_commit_filter system table 0.86
dolt_diff_log_join_from_commit system table 2.04
dolt_diff_log_join_to_commit system table 2.05
dolt_diff_table_from_commit_filter system table 1.12
dolt_diff_table_to_commit_filter system table 1.08
dolt_diffs_commit_filter system table 1
dolt_history_commit_filter system table 1.17
dolt_log_commit_filter system table 0.94

Copy link

@coffeegoddd DOLT

name add_cnt delete_cnt update_cnt latency
adds_only 60000 0 0 1.33
adds_updates_deletes 60000 60000 60000 4.43
deletes_only 0 60000 0 2.41
updates_only 0 0 60000 2.98

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants