Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport 2.x] [Search Pipelines] Add request-scoped state shared between processors #11473

Merged
merged 1 commit into from
Dec 5, 2023
Merged

[Backport 2.x] [Search Pipelines] Add request-scoped state shared between processors #11473

merged 1 commit into from
Dec 5, 2023

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport c204585 from #9405.

@github-actions
[Search Pipelines] Add request-scoped state shared between processors (…
95ac00d 
…#9405)

To handle cases where multiple search pipeline processors need to share
information, we will allocate a context holder for the lifetime of
the request and pass it to each processor to get/set values.

To explain this behavior and benefit from it, this change also introduces
three new processors:
1. The "oversample" request processor that increases "size", storing the
original size in the context.
2. The "truncate" response processor that discards results after some
number, by default using the original size before oversampling.
3. The "collapse" response processor offers similar behavior to a collapse
query, discarding results that have a field value in common with a
higher-scoring result.

Signed-off-by: Michael Froh <froh@amazon.com>
(cherry picked from commit c204585)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

Compatibility status:

Checks if related components are compatible with change 95ac00d

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/performance-analyzer.git]

@codecov Codecov
Copy link

codecov bot commented Dec 5, 2023 •
edited
Loading

Codecov Report

Attention: 52 lines in your changes are missing coverage. Please review.

Comparison is base (82bf0be) 70.99% compared to head (95ac00d) 71.16%.

Files Patch % Lines
...rch/pipeline/common/CollapseResponseProcessor.java 71.42% 7 Missing and 5 partials ⚠️
...ch/pipeline/common/helpers/SearchResponseUtil.java 69.69% 8 Missing and 2 partials ⚠️
...earch/search/pipeline/common/SearchRequestMap.java 80.00% 6 Missing and 3 partials ⚠️
...search/pipeline/common/ScriptRequestProcessor.java 40.00% 6 Missing ⚠️
...ch/pipeline/common/OversampleRequestProcessor.java 73.68% 3 Missing and 2 partials ⚠️
...pipeline/common/TruncateHitsResponseProcessor.java 84.61% 2 Missing and 2 partials ⚠️
...rg/opensearch/search/pipeline/common/BasicMap.java 90.90% 2 Missing ⚠️
...eline/common/SearchPipelineCommonModulePlugin.java 0.00% 1 Missing ⚠️
...h/search/pipeline/common/helpers/ContextUtils.java 75.00% 0 Missing and 1 partial ⚠️
...earch/pipeline/StatefulSearchRequestProcessor.java 0.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files 
@@             Coverage Diff              @@
##                2.x   #11473      +/-   ##
============================================
+ Coverage     70.99%   71.16%   +0.17%     
- Complexity    59190    59267      +77     
============================================
  Files          4870     4880      +10     
  Lines        278966   279114     +148     
  Branches      40880    40895      +15     
============================================
+ Hits         198038   198644     +606     
+ Misses        64161    63664     -497     
- Partials      16767    16806      +39

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 5, 2023

❕ Gradle check result for 95ac00d: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@reta reta merged commit 4ffd66a into 2.x Dec 5, 2023
78 checks passed
@github-actions github-actions bot deleted the backport/backport-9405-to-2.x branch December 5, 2023 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@reta reta reta approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@peternied peternied Awaiting requested review from peternied

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@tlfeng tlfeng Awaiting requested review from tlfeng

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@reta