Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage controller: use 'lazy' mode for location_config #6987

Merged
merged 3 commits into from
Mar 6, 2024
Merged

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented Mar 1, 2024

Problem

If large numbers of shards are attached to a pageserver concurrently, for example after another node fails, it can cause excessive I/O queue depths due to all the newly attached shards trying to calculate logical sizes concurrently.

#6907 added the lazy flag to handle this.

Summary of changes

  • Use lazy=true from all /location_config calls in the storage controller Reconciler.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@jcsp jcsp added a/tech_debt Area: related to tech debt c/storage/controller Component: Storage Controller labels Mar 1, 2024
Copy link

github-actions bot commented Mar 1, 2024

2490 tests run: 2367 passed, 0 failed, 123 skipped (full report)


Flaky tests (1)

Postgres 14

  • test_delete_tenant_exercise_crash_safety_failpoints[real_s3-tenant-delete-before-create-remote-mark-False-Check.RETRY_WITH_RESTART]: debug

Code coverage* (full report)

  • functions: 28.8% (6981 of 24255 functions)
  • lines: 47.3% (42876 of 90553 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
d5a694f at 2024-03-06T10:47:23.455Z :recycle:

@jcsp jcsp requested a review from koivunej March 1, 2024 14:19
@jcsp jcsp marked this pull request as ready for review March 1, 2024 14:19
@jcsp jcsp requested a review from a team as a code owner March 1, 2024 14:19
Copy link
Member

@koivunej koivunej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the live migration lazy=true issue this is looking fine.

@jcsp jcsp enabled auto-merge (squash) March 6, 2024 10:04
@jcsp jcsp merged commit a3ef50c into main Mar 6, 2024
53 checks passed
@jcsp jcsp deleted the jcsp/storcon-lazy branch March 6, 2024 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/controller Component: Storage Controller
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants