Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor agent GetObject error not to wrap with details for performance issue #2154

Merged
merged 1 commit into from
Aug 22, 2023

Conversation

ykadowak
Copy link
Contributor

Description:

This PR removes error details from agent GetObject and Exists rpc error. This is required because the following performance issue was discovered while developing the index correction feature.

The issues is that when the client issues a large number of GetObject to the agent, the process of the http2 client huffman decoding the error detail from the header becomes a bottleneck in CPU processing, resulting in a decrease of QPS. Same goes to Exists.

Even if this error detail is removed, its impact is considered to be minor. This is because it only affects the use case of the agent standalone, which is limited. Also, even in that case, the client side can manage the error without details because it must be a unary request, not stream, to the agent.

Related Issue:

Versions:

  • Go Version: 1.20.6
  • Docker Version: 20.10.8
  • Kubernetes Version: v1.27.3
  • NGT Version: 2.0.16

Checklist:

Special notes for your reviewer:

@vdaas-ci
Copy link
Collaborator

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 💌 /changelog - replace the PR body by changelog details
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • /rebase - rebase main
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

@codecov
Copy link

codecov bot commented Aug 21, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.05% ⚠️

Comparison is base (760d586) 31.18% compared to head (4d32e31) 31.13%.
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2154      +/-   ##
==========================================
- Coverage   31.18%   31.13%   -0.05%     
==========================================
  Files         339      339              
  Lines       32920    32901      -19     
==========================================
- Hits        10266    10244      -22     
+ Misses      22202    22200       -2     
- Partials      452      457       +5     
Files Changed Coverage Δ
pkg/agent/core/ngt/handler/grpc/object.go 55.26% <100.00%> (-4.07%) ⬇️

... and 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ykadowak ykadowak requested review from a team, kpango and hlts2 and removed request for a team August 21, 2023 08:56
@ykadowak ykadowak changed the title Refactor agent error not to wrap with details for performance issue Refactor agent GetObject error not to wrap with details for performance issue Aug 22, 2023
Copy link
Collaborator

@kpango kpango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ykadowak ykadowak merged commit bb04cb2 into main Aug 22, 2023
96 of 99 checks passed
@ykadowak ykadowak deleted the feature/agent/refactor-getobject-error branch August 22, 2023 03:55
@hlts2 hlts2 mentioned this pull request Sep 13, 2023
ykadowak added a commit that referenced this pull request Nov 30, 2023
ykadowak added a commit that referenced this pull request Nov 30, 2023
ykadowak added a commit that referenced this pull request Nov 30, 2023
ykadowak added a commit that referenced this pull request Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants