Integrating Protobuf with OpenSearch Transport

[VachaShah]

Description

The change in this PR is a WIP POC for integrating protobuf end-to-end for one OpenSearch API: cat nodes.

Once protobuf is end-to-end integrated for this API, we will test the API and run benchmarks to see the level of improvements. Once the benchmarks are finalized, we will start integrating protobuf in the codebase.

When the change is ready to be added in the codebase, the files in this PR will be divided into multiple parts to make the changes readable and easy to review. For now, this PR shows the scope of the project.

The idea is to provide a separate Transport option for protobuf along with the current Transport which uses Java serialization and deserialization.

Things to note in this PR:

• A separate Transport layer with protobuf integration using CodedInputStream and CodedOutputStream from protobuf instead of StreamInput and StreamOutput.
• New classes related to request and response for Cluster, Nodes API etc. have been created with protobuf integration for the sake of this end-to-end POC. When we decide to commit this code in the repo after benchmarks, we should be able to modify the existing classes to integrate with protobuf. It is not done right now since changing these classes results in a lot of changes to dependent classes which are not required for POC.

Issues Resolved

#6844

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/13958/
• CommitID: a3593b8
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14325/
• CommitID: 4f311b4
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14432/
• CommitID: f60609c
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testReadBlobWithRetries

• URL: https://build.ci.opensearch.org/job/gradle-check/14510/
• CommitID: d2c709d
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov-commenter]

Codecov Report

Merging #7131 (d2c709d) into main (0763d02) will decrease coverage by 0.17%.
The diff coverage is 0.00%.

❗ Current head d2c709d differs from pull request most recent head 3566241. Consider uploading reports for the commit 3566241 to get more accurate results

@@             Coverage Diff              @@
##               main    #7131      +/-   ##
============================================
- Coverage     70.72%   70.55%   -0.17%     
+ Complexity    59568    59508      -60     
============================================
  Files          4862     4873      +11     
  Lines        285530   285947     +417     
  Branches      41153    41201      +48     
============================================
- Hits         201936   201750     -186     
- Misses        66980    67650     +670     
+ Partials      16614    16547      -67     

Impacted Files	Coverage Δ
...ensearch/common/io/stream/ProtobufStreamInput.java	0.00% <0.00%> (ø)
...nsearch/common/io/stream/ProtobufStreamOutput.java	0.00% <0.00%> (ø)
.../org/opensearch/tasks/ProtobufCancellableTask.java	0.00% <0.00%> (ø)
...c/main/java/org/opensearch/tasks/ProtobufTask.java	0.00% <0.00%> (ø)
...org/opensearch/tasks/ProtobufTaskAwareRequest.java	0.00% <0.00%> (ø)
...main/java/org/opensearch/tasks/ProtobufTaskId.java	0.00% <0.00%> (ø)
...in/java/org/opensearch/tasks/ProtobufTaskInfo.java	0.00% <0.00%> (ø)
...rg/opensearch/tasks/ProtobufTaskResourceStats.java	0.00% <0.00%> (ø)
.../java/org/opensearch/tasks/ProtobufTaskResult.java	0.00% <0.00%> (ø)
...opensearch/transport/ProtobufTransportMessage.java	0.00% <0.00%> (ø)
... and 1 more

... and 477 files with indirect coverage changes

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14996/
• CommitID: 83b5f49
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16337/
• CommitID: 3af58a4
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18123/
• CommitID: 893741d
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18129/
• CommitID: 21c1ab5
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/18798/
• CommitID: 3566241
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity. Remove stalled label or comment or this will be closed in 7 days.

[VachaShah]

Will reopen a new one with the latest changes and some code clean up.