-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add neon.running_xacts_overflow_policy to make it possible for RO replica to startup without primary even in case running xacts overflow #8323
Conversation
…lica to startup without primary even in case running xacts overflow
Hmm, what's the plan on how this would be used? Setting GUCs for individual endpoints is not very convenient. Alternatively, you can briefly start and stop the primary, so that it writes the shutdown checkpoint. That seems more convenient. |
3060 tests run: 2945 passed, 0 failed, 115 skipped (full report)Flaky tests (4)Postgres 16
Postgres 14Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
b3f0388 at 2024-07-12T20:26:40.236Z :recycle: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
You are still not a member of @neondatabase/compute so your review is not considered as review of code owner:) |
Added myself to the group, so I think it's good to go, unless you want me to ping someone else for review :) |
…lica to startup without primary even in case running xacts overflow (#8323) ## Problem Right now if there are too many running xacts to be restored from CLOG at replica startup, then replica is not trying to restore them and wait for non-overflown running-xacs WAL record from primary. But if primary is not active, then replica will not start at all. Too many running xacts can be caused by transactions with large number of subtractions. But right now it can be also cause by two reasons: - Lack of shutdown checkpoint which updates `oldestRunningXid` (because of immediate shutdown) - nextXid alignment on 1024 boundary (which cause loosing ~1k XIDs on each restart) Both problems are somehow addressed now. But we have existed customers with "sparse" CLOG and lack of checkpoints. To be able to start RO replicas for such customers I suggest to add GUC which allows replica to start even in case of subxacts overflow. ## Summary of changes Add `neon.running_xacts_overflow_policy` with the following values: - ignore: restore from CLOG last N XIDs and accept connections - skip: do not restore any XIDs from CXLOGbut still accept connections - wait: wait non-overflown running xacts record from primary node ## Checklist before requesting a review - [ ] I have performed a self-review of my code. - [ ] If it is a core feature, I have added thorough tests. - [ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard? - [ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section. ## Checklist before merging - [ ] Do not forget to reformat commit message to not include the above checklist --------- Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Problem
Right now if there are too many running xacts to be restored from CLOG at replica startup,
then replica is not trying to restore them and wait for non-overflown running-xacs WAL record from primary.
But if primary is not active, then replica will not start at all.
Too many running xacts can be caused by transactions with large number of subtractions.
But right now it can be also cause by two reasons:
oldestRunningXid
(because of immediate shutdown)Both problems are somehow addressed now.
But we have existed customers with "sparse" CLOG and lack of checkpoints.
To be able to start RO replicas for such customers I suggest to add GUC which allows replica to start even in case of subxacts overflow.
Summary of changes
Add
neon.running_xacts_overflow_policy
with the following values:Checklist before requesting a review
Checklist before merging