Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] [Derived Fields] Add aggregation support for derived fields #15009

Merged
merged 2 commits into from
Jul 30, 2024

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport e26608b from #14618.

* Add aggregation support for derived fields

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* add unit test for a terms agg with derived fields

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* Fix license header and add changelog entry

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* move matrix_stats tests to aggs-matrix-stats module

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* Move matrix tests back and add dependency to painless module

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* add tests for all aggregations types and support ip_range

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* Add tests for agg script returned from DerivedFieldType

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* remove children aggs test as its not yet supported

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* Add more tests

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

* fix changelog

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>

---------

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>
(cherry picked from commit e26608b)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Copy link
Contributor

❌ Gradle check result for 1e69ade: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@mch2
Copy link
Member

mch2 commented Jul 29, 2024

This backport is failing because the version of tdigest is different between main (3.3) and 2.x (3.2) and using a different percentiles calculation.

These tests fail to backport to 2.x becuase 2.x uses a different branch of
tdigest that computes percentiles differently.  Rather than chase these over time,
change the assertions to check for the length of results returned instead of their values.

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>
(cherry picked from commit 0cde7ba)
Signed-off-by: Marc Handalian <marc.handalian@gmail.com>
@mch2
Copy link
Member

mch2 commented Jul 30, 2024

cherry picked #15015 on top of this pr to fix the percentile ranks difference on 2.x

Copy link
Contributor

❌ Gradle check result for 23f7128: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 23f7128: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@mch2
Copy link
Member

mch2 commented Jul 30, 2024

FAILURE

#14292
#14325

Copy link
Contributor

✅ Gradle check result for 23f7128: SUCCESS

Copy link

codecov bot commented Jul 30, 2024

Codecov Report

Attention: Patch coverage is 87.17949% with 5 lines in your changes missing coverage. Please review.

Project coverage is 71.55%. Comparing base (e5e1bad) to head (23f7128).
Report is 4 commits behind head on 2.x.

Files Patch % Lines
.../org/opensearch/index/mapper/DerivedFieldType.java 90.47% 1 Missing and 1 partial ⚠️
...aggregations/support/values/ScriptBytesValues.java 33.33% 1 Missing and 1 partial ⚠️
...pensearch/index/mapper/ObjectDerivedFieldType.java 91.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                2.x   #15009      +/-   ##
============================================
+ Coverage     71.48%   71.55%   +0.06%     
- Complexity    62677    62751      +74     
============================================
  Files          5123     5123              
  Lines        294480   294510      +30     
  Branches      42912    42915       +3     
============================================
+ Hits         210518   210733     +215     
+ Misses        66286    66095     -191     
- Partials      17676    17682       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mch2 mch2 merged commit 4b44592 into 2.x Jul 30, 2024
32 checks passed
@mch2 mch2 deleted the backport/backport-14618-to-2.x branch July 30, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants