Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Disallow removing some metadata fields by remove ingest processor #11311

Merged

Conversation

gaobinlong
Copy link
Collaborator

@gaobinlong gaobinlong commented Nov 23, 2023

Description

Backport #10895.

Related Issues

#10732

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ensearch-project#10895)

* Ignore metadata fields when removing fields by remove ingest processor

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Throw exception when removing some metadata fields

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Format the code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove calling toLowerCase()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove calling toLowerCase()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Add skip config in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Improve test coverage

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
(cherry picked from commit a0b016b)
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Copy link
Contributor

github-actions bot commented Nov 23, 2023

Compatibility status:

Checks if related components are compatible with change fe42a5a

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/sql.git]

Copy link
Contributor

✅ Gradle check result for fe42a5a: SUCCESS

Copy link

codecov bot commented Nov 23, 2023

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (c5730ba) 71.15% compared to head (fb1a7c6) 70.96%.

Files Patch % Lines
.../org/opensearch/ingest/common/RemoveProcessor.java 88.88% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                2.x   #11311      +/-   ##
============================================
- Coverage     71.15%   70.96%   -0.19%     
+ Complexity    59161    59059     -102     
============================================
  Files          4869     4869              
  Lines        278660   278669       +9     
  Branches      40831    40836       +5     
============================================
- Hits         198271   197755     -516     
- Misses        63605    64171     +566     
+ Partials      16784    16743      -41     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

❌ Gradle check result for fb1a7c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@gaobinlong
Copy link
Collaborator Author

Flaky test: #11032.

Copy link
Contributor

❕ Gradle check result for fb1a7c6: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=pit/10_basic/Delete all}
      1 org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@reta reta added bug Something isn't working v2.12.0 Issues and PRs related to version 2.12.0 labels Nov 23, 2023
@reta reta merged commit d3fca0d into opensearch-project:2.x Nov 23, 2023
28 checks passed
@gaobinlong gaobinlong deleted the backport/backport-10895-to-2.x branch June 19, 2024 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v2.12.0 Issues and PRs related to version 2.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants