-
Notifications
You must be signed in to change notification settings - Fork 396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable zstd in tests #8368
Enable zstd in tests #8368
Conversation
3126 tests run: 3005 passed, 0 failed, 121 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
2f675d0 at 2024-07-18T18:14:04.899Z :recycle: |
Apparently it still uses a pageserver from commit 939b595. |
Quoting the test description:
I suppose this is precisely what is going on here. When planning compaction, we use the uncompressed sizes. Then, we compress them, obviously resulting in smaller images. Chi's work might be useful here, to cut an image layer in an on-demand fashion instead of the current planning based approach. |
Here we are apparently surpassing the limit. I suppose one needs to adjust the test in some way to account for smaller layer sizes, not sure which method is best. # in this test case both relative_spare and relative_equal produce
# the same outcomes; this must be a quantization effect of similar
# sizes (-s4 and -s6) and small (5MB) layer size.
# for pg15 and pg16 the absdiff is < 0.01, for pg14 it is closer to 0.02
assert abs_diff < expectation |
b9ae709
to
10377ea
Compare
The |
## Problem We didn't have a way to see if compression was really working. ## Summary of changes - Add `pageserver_compression_image_in_bytes_total`, `pageserver_compression_image_out_bytes_total` - Validate these from test_image_layer_compression - Run that test with & without compression to validate not just that compression works, but that switching it off also works.
Co-authored-by: Joonas Koivunen <joonas@neon.tech>
Addressing John's feedback on Slack:
comparing CI runtime of enabling zstd link and not enabling zstd link:
I'd say if there is an impact, it's very tiny and much smaller than the noise. If we see it slow down CI, we can add such a limitation later on as well in a followup. |
Successor of #8288 , just enable zstd in tests. Also adds a test that creates easily compressable data.
Part of #5431