Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statement timeout + replica imbalance #122

Merged
merged 10 commits into from
Aug 13, 2022
Merged

Statement timeout + replica imbalance #122

merged 10 commits into from
Aug 13, 2022

Conversation

levkk
Copy link
Contributor

@levkk levkk commented Aug 13, 2022

Add a statement_timeout that protects against servers that were not health checked and also sets a reasonable maximum query duration at the pooler.

Fix replica load balancing imbalance in the case when a client connects to only issue one query.

@levkk levkk changed the title Statement timeout Statement timeout + replica imbalance Aug 13, 2022
@@ -499,7 +499,7 @@ where
// The query router determines where the query is going to go,
// e.g. primary, replica, which shard.
let mut query_router = QueryRouter::new();
let mut round_robin = 0;
let mut round_robin = rand::random();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch @drdrsh !

@levkk levkk merged commit 3285006 into main Aug 13, 2022
@levkk levkk deleted the levkk-safety-timeouts branch August 13, 2022 20:46
jmeagher pushed a commit to jmeagher/pgcat that referenced this pull request Feb 17, 2023
* Prevent clients from sticking to old pools after config update (postgresml#113)

* Re-acquire pool at the beginning of Protocol loop

* Fix query router + add tests for recycling behavior

* create a prometheus exporter on a standard http port (postgresml#107)

* create a hyper server and add option to enable it in config

* move prometheus stuff to its own file; update format

* create metric type and help lookup table

* finish the metric help type map

* switch to a boolean and a standard port

* dont emit unimplemented metrics

* fail if curl returns a non 200

* resolve conflicts

* move log out of config.show and into main

* terminating new line

* upgrade curl

* include unimplemented stats

* Validates pgcat is closed after shutdown python tests (postgresml#116)

* Validates pgcat is closed after shutdown python tests

* Fix pgrep logic

* Moves sigterm step to after cleanup to decouple

* Replace subprocess with os.system for running pgcat

* fix docker compose port allocation for local dev (postgresml#117)

change docker compose port to right prometheus port

* Update CONTRIBUTING.md

* Health check delay (postgresml#118)

* initial commit of server check delay implementation

* fmt

* spelling

* Update name to last_healthcheck and some comments

* Moved server tested stat to after require_healthcheck check

* Make health check delay configurable

* Rename to last_activity

* Fix typo

* Add debug log for healthcheck

* Add address to debug log

* Speed up CI a bit (postgresml#119)

* Sleep for 1s

* use premade image

* quicker

* revert shutdown timeout

* Fix debug log (postgresml#120)

* Make prometheus port configurable (postgresml#121)

* Make prometheus port configurable

* Update circleci config

* Statement timeout + replica imbalance fix (postgresml#122)

* Statement timeout

* send error message too

* Correct error messages

* Fix replica inbalance

* disable stmt timeout by default

* Redundant mark_bad

* revert healthcheck delay

* tests

* set it to 0

* reload config again

* pgcat toml

Co-authored-by: Nicholas Dujay <3258756+dat2@users.noreply.github.com>
Co-authored-by: zainkabani <77307340+zainkabani@users.noreply.github.com>
Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com>
Co-authored-by: Pradeep Chhetri <30620077+chhetripradeep@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant