-
Notifications
You must be signed in to change notification settings - Fork 434
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(pageserver): generate image layers for sparse keyspace (#7567)
Part of #7462 Sparse keyspace does not generate image layers for now. This pull request adds support for generating image layers for sparse keyspace. ## Summary of changes * Use the scan interface to generate compaction data for sparse keyspace. * Track num of delta layers reads during scan. * Read-trigger compaction: when a scan on the keyspace touches too many delta files, generate an image layer. There are one hard-coded threshold for now: max delta layers we want to touch for a scan. * L0 compaction does not need to compute holes for metadata keyspace. Know issue: the scan interface currently reads past the image layer, which causes `delta_layer_accessed` keeps increasing even if image layers are generated. The pull request to fix that will be separate, and orthogonal to this one. --------- Signed-off-by: Alex Chi Z <chi@neon.tech>
- Loading branch information
Showing
5 changed files
with
362 additions
and
140 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
7701ca4
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3166 tests run: 3023 passed, 3 failed, 140 skipped (full report)
Failures on Postgres 14
test_storage_controller_many_tenants[github-actions-selfhosted]
: releasetest_basebackup_with_high_slru_count[github-actions-selfhosted-sequential-10-13-30]
: releasetest_basebackup_with_high_slru_count[github-actions-selfhosted-vectored-10-13-30]
: releaseFlaky tests (1)
Postgres 15
test_hot_standby_gc
: debugCode coverage* (full report)
functions
:31.3% (6396 of 20411 functions)
lines
:47.9% (48879 of 102126 lines)
* collected from Rust tests only
7701ca4 at 2024-05-20T17:28:05.001Z :recycle: