Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rpc): workaround for tonic hanging on large error message #18639

Merged
merged 8 commits into from
Sep 23, 2024

Conversation

BugenZhao
Copy link
Member

@BugenZhao BugenZhao commented Sep 23, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

See background and explanation in #18039.

TL;DR: due to impl bug in hyper v1, HTTP2 requests with a header size larger than configured max_header_list_size will hang, which can occur in our system because we (and also gRPC protocol) embed error messages in gRPC header. It hasn't been fixed yet, so all we can do is to configure the header list size to a larger value to mitigate such issue.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@github-actions github-actions bot added the type/fix Bug fix label Sep 23, 2024
Copy link
Member Author

BugenZhao commented Sep 23, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @BugenZhao and the rest of your teammates on Graphite Graphite

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao BugenZhao marked this pull request as ready for review September 23, 2024 09:07
@BugenZhao BugenZhao requested a review from a team as a code owner September 23, 2024 09:07
@BugenZhao BugenZhao requested review from xxchan, stdrc and wenym1 and removed request for a team September 23, 2024 09:07
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Copy link
Contributor

@wenym1 wenym1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Base automatically changed from bz/grpc-client-refactor to main September 23, 2024 12:13
@BugenZhao BugenZhao added this pull request to the merge queue Sep 23, 2024
@graphite-app graphite-app bot requested a review from a team September 23, 2024 12:13
Merged via the queue into main with commit d5dbc6a Sep 23, 2024
28 of 29 checks passed
@BugenZhao BugenZhao deleted the bz/grpc-error-hang branch September 23, 2024 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants