Increase ServerConnector accept queue size #15596
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
georgew5656
changed the title
gianm
reviewed
Jan 5, 2024
| return Integer.parseInt(in.readLine().trim()); | ||
| } | ||
| catch (Exception e) { | ||
| log.warn("Unable to read /proc/sys/net/core/somaxconn, falling back to default value for TCP accept queue size"); |
There was a problem hiding this comment.
It'd be good to only log this warning on Linux. The file will never be there on macOS so the warning is always going to happen, which isn't ideal. Maybe use System.getProperty("os.name")?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Increase TCP accept queue size.
Description
The ServerConnector class defaults _acceptQueueSize to 0 which appears to result in a value of min(net.core.somaxconn, 50) on Jetty being passed to the listen() system call. This number can be too low especially when there are many brokers/coordinators trying to setup connections to a historical or real-time index process.
When this queue fills up there will be dropped packets and there can be downstream connection timeouts/connection resets depending on the underlying system settings (e.g. if syncookies are enabled).
Fixed the bug ...
Renamed the class ...
Added a forbidden-apis entry ...
I originally decided to make _acceptQueueSize a configurable property, but to avoid adding another configuration I decided to follow the logic I saw in dropwizard (a java web framework) when configuring jetty.(https://github.com/dropwizard/dropwizard/blob/7f6827d119639a57d2a696d371c3bbdb3007acb5/dropwizard-jetty/src/main/java/io/dropwizard/jetty/NetUtil.java#L37).
Essentially this checks the value of '/proc/sys/net/core/somaxconn' and sets _acceptQueueSize to that value, or the default linux value for somaxconn of 128 if the file is not readable.
Release note
Increase _acceptQueueSize based on value of net.core.somaxconn.
Key changed/added classes in this PR
JettyServerModuleThis PR has: