Run IT tests with security plugin #335

Yury-Fridlyand · 2023-08-08T23:27:51Z

Description

This PR adds a new IT gradle task: integTestWithSecurity. It starts a cluster with security plugin installed (it takes latest snapshot), configures cluster, http client for tests and runs one test which required to be run with security plugin.
A new GHA is added which runs this test task.

This PR temporary includes files from opensearch-project#1943.

Issues Resolved

opensearch-project#1713

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

integ-test/rc/esnode-key.pem

peternied

I am a big fan of integration test around security scenarios, added a couple of comments/considerations.

peternied · 2023-08-09T14:49:12Z

integ-test/build.gradle

@@ -57,6 +60,20 @@ ext {
    projectSubstitutions = [:]
    licenseFile = rootProject.file('LICENSE.TXT')
    noticeFile = rootProject.file('NOTICE')
+
+    getSecurityPluginDownloadLink = { ->
+        var repo = "https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/" +


The Security Plugin maintainers (myself included) have had trouble with build breaks and fixing them in a timely manner - it would be unfortunate for your integration tests to be flaky due to downstream issues. This isn't an issue with release builds with the trade-off of longer time between updates.

Tracking issue OpenSearch's Unreliable Dependency - A Call for Enhanced API Compatibility opensearch-project/OpenSearch#8982

peternied · 2023-08-09T14:57:09Z

integ-test/build.gradle

+        setting 'plugins.security.ssl.transport.enforce_hostname_verification', 'false'
+        // https is disabled, because `OpenSearchCluster` is hardcoded to validate cluster health by http
+        // refer how IT framework implemented in security plugin and reuse/copy to activate https
+        setting 'plugins.security.ssl.http.enabled', 'false'


There are some features of the security plugin - such as being able to make changes to the security configuration dynamically that will not be possible with this setting disabled. Depends if your tests will ever need to exercise that kind of functionality.

plugins.security.ssl.http.enabled setting cannot be disabled on Rest layer as securityadmin.sh execution fails. opensearch-project/security#1952

This issue was raised because Release testing runs IT tests with the security plugin. If we can catch errors early, we can avoid blocking release.

integ-test/rc/esnode-key.pem

.github/workflows/integ-tests-with-security.yml

integ-test/build.gradle

acarbonetto · 2023-08-09T21:15:55Z

integ-test/build.gradle

+        setting 'plugins.security.ssl.transport.enforce_hostname_verification', 'false'
+        // https is disabled, because `OpenSearchCluster` is hardcoded to validate cluster health by http
+        // refer how IT framework implemented in security plugin and reuse/copy to activate https
+        setting 'plugins.security.ssl.http.enabled', 'false'


This issue was raised because Release testing runs IT tests with the security plugin. If we can catch errors early, we can avoid blocking release.

integ-test/src/test/java/org/opensearch/sql/legacy/OpenSearchSQLRestTestCase.java

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

integ-test/build.gradle

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

integ-test/build.gradle

integ-test/src/test/java/org/opensearch/sql/legacy/OpenSearchSQLRestTestCase.java

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

codecov · 2023-08-17T22:36:47Z

Codecov Report

Merging #335 (a618be8) into integ-IT-with-security (e377cf2) will not change coverage.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##             integ-IT-with-security     #335   +/-   ##
=========================================================
  Coverage                     97.51%   97.51%           
  Complexity                     4657     4657           
=========================================================
  Files                           408      408           
  Lines                         11933    11933           
  Branches                        829      829           
=========================================================
  Hits                          11637    11637           
  Misses                          289      289           
  Partials                          7        7

Flag	Coverage Δ
sql-engine	`97.51% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

* Run IT tests with security plugin (#335) * Add extra IT flow. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Remove unneeded files. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Fix GHA matrix syntax. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Fix GHA matrix syntax. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Code clean up. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Optimize downloading. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Apply suggestions from code review Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Update integ-test/build.gradle Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Rework implementation. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR review. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR feedback + some fixes. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> --------- Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Minor fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR feedback. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> --------- Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* Run IT tests with security plugin (#335) * Add extra IT flow. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Remove unneeded files. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Fix GHA matrix syntax. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Fix GHA matrix syntax. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Code clean up. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Optimize downloading. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Apply suggestions from code review Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Update integ-test/build.gradle Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Rework implementation. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR review. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR feedback + some fixes. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> --------- Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> * Minor fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Address PR feedback. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> * Typo fix. Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> --------- Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com> (cherry picked from commit 7e3a718) Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand requested review from MitchellGale and matthewryanwells August 8, 2023 23:27

Yury-Fridlyand requested review from MaxKsyunz, forestmvey, acarbonetto and GumpacG as code owners August 8, 2023 23:27

Yury-Fridlyand changed the title ~~Dev it with security~~ Run IT tests with security plugin Aug 8, 2023

MitchellGale reviewed Aug 8, 2023

View reviewed changes

integ-test/rc/esnode-key.pem Outdated Show resolved Hide resolved

peternied reviewed Aug 9, 2023

View reviewed changes

acarbonetto reviewed Aug 9, 2023

View reviewed changes

Yury-Fridlyand force-pushed the dev-IT-with-security branch from 58c5c1f to 90f2873 Compare August 9, 2023 23:08

GumpacG approved these changes Aug 14, 2023

View reviewed changes

Yury-Fridlyand and others added 11 commits August 14, 2023 18:05

Add extra IT flow.

e2f0da9

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Remove unneeded files.

e840ac2

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Typo fix.

79d7373

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Fix GHA matrix syntax.

b28f328

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Fix GHA matrix syntax.

6049800

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Code clean up.

25b22e3

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Optimize downloading.

28fabfd

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Apply suggestions from code review

9eb74bd

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>

Update integ-test/build.gradle

6d37117

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>

Typo fix.

e68aedf

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Rework implementation.

e25e7af

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand force-pushed the dev-IT-with-security branch from 9277659 to e25e7af Compare August 15, 2023 01:17

acarbonetto approved these changes Aug 16, 2023

View reviewed changes

integ-test/build.gradle Outdated Show resolved Hide resolved

integ-test/build.gradle Show resolved Hide resolved

Address PR review.

42fd960

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

acarbonetto approved these changes Aug 17, 2023

View reviewed changes

Bit-Quill deleted a comment from codecov bot Aug 17, 2023

Address PR feedback + some fixes.

a618be8

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>

Yury-Fridlyand mentioned this pull request Aug 17, 2023

Allow test clusters to run with TLS opensearch-project/OpenSearch#8900

Merged

6 tasks

Yury-Fridlyand merged commit f99d895 into integ-IT-with-security Aug 17, 2023
32 checks passed

Yury-Fridlyand deleted the dev-IT-with-security branch August 17, 2023 23:28

Yury-Fridlyand mentioned this pull request Aug 17, 2023

Run IT tests with security plugin (#335) opensearch-project/sql#1986

Merged

6 tasks

MitchellGale mentioned this pull request Aug 23, 2023

[Backport 2.x] Run IT tests with security plugin (#335) #1986 opensearch-project/sql#2022

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run IT tests with security plugin #335

Run IT tests with security plugin #335

Yury-Fridlyand commented Aug 8, 2023 •

edited

Loading

peternied left a comment

peternied Aug 9, 2023

peternied Aug 9, 2023

acarbonetto Aug 9, 2023

acarbonetto Aug 9, 2023

codecov bot commented Aug 17, 2023 •

edited

Loading

Run IT tests with security plugin #335

Run IT tests with security plugin #335

Conversation

Yury-Fridlyand commented Aug 8, 2023 • edited Loading

Description

Issues Resolved

Check List

peternied left a comment

Choose a reason for hiding this comment

peternied Aug 9, 2023

Choose a reason for hiding this comment

peternied Aug 9, 2023

Choose a reason for hiding this comment

acarbonetto Aug 9, 2023

Choose a reason for hiding this comment

acarbonetto Aug 9, 2023

Choose a reason for hiding this comment

codecov bot commented Aug 17, 2023 • edited Loading

Codecov Report

Yury-Fridlyand commented Aug 8, 2023 •

edited

Loading

codecov bot commented Aug 17, 2023 •

edited

Loading