Epic: Pageserver should support 200K tenants

[shanyp]

Motivation

To reduce costs and support higher density PS

DoD

Pageserver P90 get_page_request is the same with 200K tenants

Implementation ideas

Detaching in-active tenants
Reducing metrics

Tasks

• [ ]

Other related tasks and Epics

[LizardWizzard]

Lets mention our assumption on how many of those 200k are active. I'd say that it is ~1-3%. Taking the average of 2% gives us 4k active tenants. Is it accurate?

Current problems are:

• Startup time. Steps here are (1) moving to async file io: Epic: convert remaining IO stack for Timeline::get to async fn #4743, (2) switching to use catalog instead of metadata files: Epic: Pageserver internal catalog #4636. Schedule recently active tenants to become active first
• Get memory usage under control. This may include introducing some "low energy" mode for tenants where layer map is unloaded, background jobs are not running, all data is evicted to s3.
• What if such big pageserver fails? How quickly we can restore it?

[jcsp]

Our latest thinking on this topic is that it does not make sense to attach idle tenants and then layer a low-energy mechanism on top of that. Instead, low energy tenant should not be attached at all.

The Secondary mode in #5299 is also suitable for idle tenants: whereas a normal active tenant would have one attached location and one secondary location, a long-idle tenant can just have one secondary layer with warm=false. Its layers will gradually fall off disk due to disk pressure. Eventually, it will make sense to not even have secondary locations (these are cheaper than attachments but still have some cost), especially for small tenants whose layers can rapidly be downloaded from S3 on-demand.

The responsibility for orchestrating this gets pushed up to the control plane, or to some new intermediate service that manages pageservers.

To make this complete, it is also necessary to handle:

• Consumption metrics: either the billing infrastructure needs a "last recieved value still holds" behavior, or something will have to send redundant repeats of consumption metrics for these idle tenants.

Nice to have:

• If a tenant goes idle while it has consumption/gc work pending, then wake it up at some point to do that. In practice, it might be sufficient to wake idle tenants after 1 week before putting them to sleep indefinitely, since we use a default 1 week PITR target.