feat(per-tenant throttling): exclude throttled time from page_service metrics + regression test

[problame]

part of #5899

Problem

Before this PR, the time spent waiting on the throttle was charged towards the higher-level page_service metrics, i.e., pageserver_smgr_query_seconds.
The metrics are the foundation of internal SLIs / SLOs.
A throttled tenant would cause the SLI to degrade / SLO alerts to fire.

Changes

• don't charge time spent in throttle towards the page_service metrics
  • record time spent in throttle in RequestContext and subtract it from the elapsed time
  • this works because the page_service path doesn't create child context, so, all the throttle time is recorded in the parent
  • it's quite brittle and will break if we ever decide to spawn child tasks that need child RequestContexts, which would have separate instances of the micros_spent_throttled counter.
  • however, let's punt that to a more general refactoring of RequestContext
• add a test case that ensures that
  • throttling happens for getpage requests; this aspect of the test passed before this PR
  • throttling delays aren't charged towards the page_service metrics; this aspect of the test only passes with this PR
• drive-by: make the throttle log message info!, it's an expected condition

Performance

I took the same measurements as in #6706 , no meaningful change in CPU overhead.

Future Work

This PR enables us to experiment with the throttle for select tenants without affecting the SLI metrics / triggering SLO alerts.

Before declaring this feature done, we need more work to happen, specifically:

• decide on whether we want to retain the flexibility of throttling any Timeline::get call, filtered by TaskKind
• versus: separate throttles for each page_service endpoint, potentially with separate config options
• the trouble here is that this decision implies changes to the TenantConfig, so, if we start using the current config style now, then decide to switch to a different config, it'll be a breaking change

Nice-to-haves but probably not worth the time right now:

• Equivalent tests to ensure the throttle applies to all other page_service handlers.

[github-actions]

2490 tests run: 2368 passed, 0 failed, 122 skipped (full report)

---

Code coverage* (full report)

• functions: 28.8% (6974 of 24232 functions)
• lines: 47.3% (42710 of 90340 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
2dcf5d8 at 2024-03-05T13:05:15.472Z :recycle:}