feat(pageserver): add an optional lease to the get_lsn_by_timestamp API

[yliang412]

Part of #7497, closes #8072.

Problem

Currently the get_lsn_by_timestamp and branch creation pageserver APIs do not provide a pleasant client experience where the looked-up LSN might be GC-ed between the two API calls.

This PR attempts to prevent common races between GC and branch creation by making use of LSN leases provided in #8084. A lease can be optionally granted to a looked-up LSN. With the lease, GC will not touch layers needed to reconstruct all pages at this LSN for the duration of the lease.

Caveats: Leases can only prevent the common case. Branch creation at a LSN after lease is expired can still race with GC.

Summary of changes

• Added an optional with_lease field in the request and an optional valid_until field in the response to optionally grant leases.
• Revised OpenAPI spec for get_lsn_by_timestamp.
• Modified test_lsn_mapping to parametrize over with_lease.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

2922 tests run: 2806 passed, 0 failed, 116 skipped (full report)

---

Code coverage* (full report)

• functions: 32.6% (6883 of 21133 functions)
• lines: 50.2% (53658 of 106976 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
022eecc at 2024-06-24T20:06:24.042Z :recycle:}

[VladLazar]

Looks good to me. I also peeked at the lease implementation and was surprised the leases are not persisted to disk anywhere. Are we ok with losing them on restart?

[yliang412]

TODO: Investigate how get_lsn_by_timestamp API is used. How often is a lease not needed? This will help us decide whether we want the default behavior to grant a lease (or not).