Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add tests for #1763 #1889

Closed
2 tasks
aome510 opened this issue Jun 6, 2022 · 1 comment · Fixed by #1919
Closed
2 tasks

Add tests for #1763 #1889

aome510 opened this issue Jun 6, 2022 · 1 comment · Fixed by #1919
Assignees
@aome510
Labels
a/test Area: related to testing

Comments

@aome510
Copy link
Contributor

aome510 commented Jun 6, 2022 •
edited
Loading

Context

Intensive write workload blocks PG instance because the speed of WAL generated is much higher than the amount of WALs that the page server can process.

We do have max_replication_write_lag to avoid having an infinite amount of WAL produced. However, when max_replication_write_lag is set to be high, performing a read will need to wait for all the WALs to be processed, even if none of the writes affected the read, because the system doesn't know that.

This is the problem we encounter in #1763

Tasks

Related PRs/issues
@aome510
Copy link
Contributor Author

aome510 commented Jun 6, 2022

Slack thread: https://neondb.slack.com/archives/C033RQ5SPDH/p1654535501868279

@aome510 aome510 self-assigned this Jun 6, 2022
@aome510 aome510 added the a/test Area: related to testing label Jun 6, 2022
@aome510 aome510 mentioned this issue Jun 8, 2022
Closed
@aome510 aome510 mentioned this issue Jun 17, 2022
Merged
aome510 added a commit that referenced this issue Jun 20, 2022
@aome510
Add wal backpressure tests (#1919)
37465da 
Resolves #1889.

This PR adds new tests to measure the WAL backpressure's performance under different workloads.

## Changes
- add new performance tests in `test_wal_backpressure.py`
- allow safekeeper's fsync to be configurable when running tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aome510 aome510

Labels
a/test Area: related to testing
Projects
None yet
Milestone
No milestone
1 participant
@aome510