Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db_bench: many default values are not the default in Speedb #61

Closed
Yuval-Ariel opened this issue Jul 20, 2022 · 4 comments · Fixed by #322
Closed

db_bench: many default values are not the default in Speedb #61

Yuval-Ariel opened this issue Jul 20, 2022 · 4 comments · Fixed by #322
Assignees
Labels
performance Upstreamable can be upstreamed to RocksDB
Milestone

Comments

@Yuval-Ariel
Copy link
Contributor

these params are:

table_cache_numshardbits is 4 while default is 6.
new table reader for comp inputs
block cache 8mb
index shortening mode
enable_pipelined_write - true in db bench and false otherwise.
delayed_write_rate = 8Mb and default value is 0.

and possibly more.

fix by making db_bench take the default value

@isaac-io isaac-io added this to the v2.2.0 milestone Oct 26, 2022
@Yuval-Ariel
Copy link
Contributor Author

changes:
index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
enable_pipelined_write - from true to false

Yuval-Ariel added a commit that referenced this issue Jan 1, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
@Yuval-Ariel Yuval-Ariel linked a pull request Jan 1, 2023 that will close this issue
@erez-speedb
Copy link

@Yuval-Ariel
Copy link
Contributor Author

@erez-speedb , what do you mean outlier? the trend looks completely different
image
can you point to previous outliers that behaved like this? if you run two exact hashes, can there be a chance for such a difference?
why not run it again and verify whether its an outlier or not?

@erez-speedb
Copy link

Running the small db test again
The results are back to normal
https://admin.speedb.io/performance?items=MTnDOa9oyhGa33CyQgci&items=Swqg6xZSgqnVYc71L1mA&colors=%2300796b&colors=%23ec407a

Passed perf tests.

@Guyme Guyme added performance Upstreamable can be upstreamed to RocksDB labels Jan 18, 2023
Yuval-Ariel added a commit that referenced this issue Jan 18, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
ayulas pushed a commit that referenced this issue Feb 26, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
Yuval-Ariel added a commit that referenced this issue May 1, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
Yuval-Ariel added a commit that referenced this issue May 4, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
udi-speedb pushed a commit that referenced this issue Nov 13, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
udi-speedb pushed a commit that referenced this issue Nov 15, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
udi-speedb pushed a commit that referenced this issue Dec 4, 2023
actual values that have changed:
1. index_shortening_mode - from 2 (kShortenSeparatorsAndSuccessor) to 1 (kShortenSeparators)
2. hard_pending_compaction_bytes_limit - from 128 Gb to 256 Gb
3. delayed_write_rate - from 8Mb to 0 (0 is sanitized to 16Mb)
4. enable_pipelined_write - from true to false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Upstreamable can be upstreamed to RocksDB
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants