test(pageserver): add test interface to create artificial layers

[skyzh]

Problem

This pull request adds necessary interfaces to deterministically create scenarios we want to test. Simplify some test cases to use this interface to make it stable + reproducible.

Compaction test will be able to use this interface. Also the upcoming delete tombstone tests will use this interface to make test reproducible.

Summary of changes

• force_create_image_layer
• force_create_delta_layer
• force_advance_lsn
• create_test_timeline_with_states
• branch_timeline_test_with_states

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

3156 tests run: 3017 passed, 0 failed, 139 skipped (full report)

---

Flaky tests (1)

Postgres 15

• test_statvfs_pressure_usage: debug

Code coverage* (full report)

• functions: 31.4% (6503 of 20707 functions)
• lines: 48.3% (50223 of 103881 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
b12c705 at 2024-05-31T15:53:31.681Z :recycle:}

[koivunej]

I think this would move our tests into a direction which currently does what the previously invoked abstractions were doing, but also bypasses everything. I dislike this "alternative reality" and think this will incur great tech debt right away; instead we should investigate wrapping the TimelineWriter.

Discussion about above higher level test-api problem:
https://github.com/neondatabase/neon/pull/7899/files#r1620672536

[skyzh]

I don't feel timeline writer is the right thing to wrap for this test API. Timeline writer represents a normal+legal write to the timeline. The layers written by the timeline writer API goes through L0->compaction->L1. However, in my case, I want to construct a timeline state that is exactly what I want to test, instead of using high-level APIs to do a series of operations that may or may not end up in the state I want to test. As you see, the test cases are much simpler than before, and I don't see it would be a tech debt if this is only used in test cases.

Anyways, I feel what we lack is to describe a timeline state and directly test it in the test cases. For example, if I want to write a test "ensure vectored get does not go pass image layer", I will need to create two layers: an image layer above a delta layer. Using the current API, it's hard to achieve. It's hard to guarantee that after a series of compaction and data writing, the end state of the timeline is what I want. The interface to directly place a layer into the layer map is simple and reliable to recreate this scenario.

[koivunej]

I think this is looking much better. We could tune the declarativeness aspect with more builder-y structures and not using the layer creation but going through TimelineWriter, but I think this would already work for vectored layer tests just fine. Now I think it's much more clear that we aim to have only single path for "timeline growing" methods, or alternatively these should be used instead.