QueryFetchSearchResult as a proto message and node-to-node communication with protobuf

[VachaShah]

Description

This PR is part 1 of the changes and POC done in #10684. It contains the following changes:

1. QueryFetchSearchResult as a proto message.
2. A new interface for serialization and deserialization using Protobuf - ProtobufWriteable.
3. Node-to-node communication with protobuf for a basic response QueryFetchSearchResult during a search API.
4. Feature flag for the above changes.
5. Supporting framework for adding protobuf.

Incoming changes in this PR:

• Add tests for protobuf messages and node-to-node communication for QueryFetchSearchResult.

Next part of changes:

• Make the feature flag dynamic so that it can be accessed via cluster settings.
• Add support for Aggregations and some remaining fields.
• Abstractions which support making the transport layer pluggable.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Compatibility status:

Checks if related components are compatible with change 324c58f

Incompatible components

Incompatible components: [https://github.com/opensearch-project/asynchronous-search.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git]

[github-actions]

❌ Gradle check result for 5c92d5c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[dblock]

First, it's awesome that this works end to end.

Some high level thoughts on the implementation. The way I imagined this would look is that the transport parts are abstracted enough that we can add another transport protocol after protobuf. This means putting current transport and protobuf transport at the same level, not switched by an if statement when the message is protobuf, and not adding protobuf to the current hierarchy without extracting the current transport protocol. WDYT?

[reta]

@VachaShah ever more endorsements than @dblock (:-D), but on general note: the Protobuf support (as of this pull request) is very much entangled with the transport APIs and implementation (mostly every layer has to be aware of it). I think this is a good opportunity to make breaking changes so adding new serialization protocols could be more or less simple (let say we would need to add FlatBuffers, you name it, what would be the cost of that)

[github-actions]

❌ Gradle check result for ed00f89: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[VachaShah]

The transport abstractions from PR #12967 are now merged in this PR and protobuf is incorporated in that framework.

[github-actions]

✅ Gradle check result for 324c58f: SUCCESS

[dblock]

@VachaShah this is still a meaty change, do you want us to go over it, or is there a smaller piece extractable?

[VachaShah]

@VachaShah this is still a meaty change, do you want us to go over it, or is there a smaller piece extractable?

@dblock I am working on a PR for extracting protos for some search classes in another PR to make it easier to review. Then this PR can hopefully be for tying the pieces together.

[VachaShah]

Extracted protos and its serialization in PR #13178.

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity.

[VachaShah]

Closing this PR since won't be working on this actively. If this change needs to be picked up, please feel free to use my branch https://github.com/VachaShah/OpenSearch/tree/search-protobuf-part1.