test(pageserver): quantify compaction outcome

[skyzh]

Problem

A simple API to collect some statistics after compaction to easily understand the result.

The tool reads the layer map, and analyze range by range instead of doing single-key operations, which is more efficient than doing a benchmark to collect the result. It currently computes two key metrics:

• Latest data access efficiency, which finds how many delta layers / image layers the system needs to iterate before returning any key in a key range.
• (Approximate) PiTR efficiency, as in quantification of compaction algorithms #7770, which is simply the number of delta files in the range. The reason behind that is, assume no image layer is created, PiTR efficiency is simply the cost of collect records from the delta layers, and the replay time. Number of delta files (or in the future, estimated size of reads) is a simple yet efficient way of estimating how much effort the page server needs to reconstruct a page.

Summary of changes

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

3268 tests run: 3116 passed, 0 failed, 152 skipped (full report)

---

Flaky tests (1)

Postgres 15

• test_vm_bit_clear_on_heap_lock: debug

Code coverage* (full report)

• functions: 31.5% (6596 of 20943 functions)
• lines: 48.5% (51038 of 105301 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
45cad40 at 2024-06-05T19:26:39.506Z :recycle:}

[skyzh]

Updated the tool to become an HTTP interface so that the Python tests can read it.

[skyzh]

I believe CI will pass this time and ready for review :) Thanks! @arpad-m @problame

[problame]

This PR adds code to determine, for a given layer map snapshot, the amount of delta layers that need to be visited before we hit an image layer when reconstructing any key in the layer map.

That metric is what I'd laxely call delta layer stack height.

It is a rough proxy metric for random getpage@lsn IO amplification under the assumption of uniform density on key & LSN dimension among all delta layers in the layer map. I.e., the probability of finding X amount of information about a random (key,lsn) \in key-lsn-range(L) of a given layer L is the same for all layers L.

While this is useful, we wanted the point-in-time & total space efficiency metrics.

I suppose to calculate worst-case point-in-time space usage, we'd need a similar analysis but along the LSN dimension.

---

In addition to the missing metrics, I suggest to move the analysis code into a sub-module of mod timeline that extends impl Timeline.

E.g., like we did for the compaction code:

neon/pageserver/src/tenant/timeline/compaction.rs

Line 46 in 3860bc9

impl Timeline {

---

Lastly, what about branching? Not covered in this PR.

---

I suggest the way forward:

1. Apply renaming to submodule in this PR, then let's get it merged.
2. Another PR to add branching support (just build a temporary LayerMap instance that contains all the layers from all recusrive ancestors)
3. Another PR to extend the analysis for point-in-time space efficiency.

[problame]

Hm, and one more thought: The max_num_of_deltas_above_image is only for @latest LSN, but what we (also) care about is worst-case max_num_of_deltas_above_image at random LSN.

[skyzh]

but what we (also) care about is worst-case max_num_of_deltas_above_image at random LSN.

Yep, that makes sense. I will submit a separate pull request for that.

[skyzh]

Ready for review again :) Hopefully I've resolved all the concerns and we have quite some future works for this analysis code to be very useful.

[problame]

but what we (also) care about is worst-case max_num_of_deltas_above_image at random LSN.

Yep, that makes sense. I will submit a separate pull request for that.

I'll continue to work on the quantification efforts, implementing my asks above, while @skyzh will work on #7948 and follow-ups.