-
Notifications
You must be signed in to change notification settings - Fork 427
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Server scalability improvements #2752
Server scalability improvements #2752
Conversation
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
Just adding @avishniakov as a reviewer on the PR as it feels like OSS team should have a sense of these changes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me save for a few comments.
# We only use one connection pool to be cached because we | ||
# only communicate with one remote server (the control | ||
# plane) | ||
pool_connections=1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For my understanding, is it costly to have more cached hosts? I understand we need only one but want to know what impact this would have.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no impact in this case. We can set this to 10 or 100, there will only be one connection pool because we only need to connect to one host. This is better covered here, if you want the gruesome details: https://stackoverflow.com/questions/34837026/whats-the-meaning-of-pool-connections-in-requests-adapters-httpadapter
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really well explained answer; had this in bookmarks for a while ;) thanks!
ZENML_STORE_POOL_SIZE: {{ .Values.zenml.database.poolSize | b64enc | quote }} | ||
{{- end }} | ||
{{- if .Values.zenml.database.maxOverflow }} | ||
ZENML_STORE_POOL_SIZE: {{ .Values.zenml.database.maxOverflow | b64enc | quote }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicate key?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch ! thank you !
from fastapi import FastAPI, HTTPException, Request | ||
from fastapi.exceptions import RequestValidationError | ||
from fastapi.responses import ORJSONResponse | ||
from fastapi.staticfiles import StaticFiles | ||
from fastapi.templating import Jinja2Templates | ||
from py import log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This and the sleep
import seem unused?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct, I removed them. The linter was also flagging them.
* configurable thread pool size * sync the cloud API requests connection pool to the server's thread pool size * make the health check endpoint async * make all dashboard endpoints async as well
d1e5313
to
5a15ac7
Compare
Quickstart template updates in |
E2E template updates in |
NLP template updates in |
Describe changes
This PR brings a number of improvements meant to make the ZenML server more scalable and resilient under load:
Pre-requisites
Please ensure you have done the following:
develop
and the open PR is targetingdevelop
. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes