feat(pageserver): add aux-file-v2 flag on tenant level

[skyzh]

Problem

Changing metadata format is not easy. This pull request adds a tenant-level flag on whether to enable aux file v2. As long as we don't roll this out to the user and guarantee our staging projects can persist tenant config correctly, we can test the aux file v2 change with setting this flag. Previous discussion at #7424.

Summary of changes

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

2766 tests run: 2645 passed, 0 failed, 121 skipped (full report)

---

Flaky tests (1)

Postgres 16

• test_gc_of_remote_layers: debug

Code coverage* (full report)

• functions: 28.1% (6481 of 23049 functions)
• lines: 47.0% (46064 of 98030 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
d2ab310 at 2024-04-24T20:28:00.836Z :recycle:}