[Weighted Routing] Add support for discovered master and remove local weights in the response #5680

anshu1106 · 2023-01-03T11:36:18Z

Signed-off-by: Anshu Agarwal anshukag@amazon.com

Description

The local weights don't add much value since weights are relative and having just the local value gives no indication of how traffic would get distributed.
Discovered master is helpful to understand cases where after weights are set and also that the node is unable to connect to the leader either due to
Master not discovered -- This might happen to newly launched after decommission or even otherwise
Decommissioned -- The node was decommissioned

Issues Resolved

[List any issues this PR will resolve]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2023-01-03T11:57:03Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8728/
CommitID: 70971dc
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-01-03T12:28:26Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8730/
CommitID: 73fd9b8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2023-01-04T05:46:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8769/
CommitID: ef429b2
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2023-01-04T07:43:07Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/8783/
CommitID: 5ad7e3c

codecov-commenter · 2023-01-04T07:45:03Z

Codecov Report

Merging #5680 (4e15525) into main (2746517) will decrease coverage by 0.10%.
The diff coverage is 52.38%.

@@             Coverage Diff              @@
##               main    #5680      +/-   ##
============================================
- Coverage     71.11%   71.01%   -0.11%     
+ Complexity    58641    58600      -41     
============================================
  Files          4760     4760              
  Lines        279515   279516       +1     
  Branches      40348    40347       -1     
============================================
- Hits         198782   198491     -291     
- Misses        64463    64876     +413     
+ Partials      16270    16149     -121

Impacted Files	Coverage Δ
...eighted/get/TransportGetWeightedRoutingAction.java	`73.68% <50.00%> (-2.32%)`	⬇️
...eighted/get/ClusterGetWeightedRoutingResponse.java	`62.26% <52.63%> (-2.96%)`	⬇️
...adonly/AddIndexBlockClusterStateUpdateRequest.java	`0.00% <0.00%> (-75.00%)`	⬇️
...readonly/TransportVerifyShardIndexBlockAction.java	`9.75% <0.00%> (-73.18%)`	⬇️
...n/admin/indices/readonly/AddIndexBlockRequest.java	`17.85% <0.00%> (-53.58%)`	⬇️
.../java/org/opensearch/node/NodeClosedException.java	`50.00% <0.00%> (-50.00%)`	⬇️
...ava/org/opensearch/action/NoSuchNodeException.java	`0.00% <0.00%> (-50.00%)`	⬇️
...a/org/opensearch/tasks/TaskCancelledException.java	`50.00% <0.00%> (-50.00%)`	⬇️
...opensearch/persistent/PersistentTasksExecutor.java	`22.22% <0.00%> (-44.45%)`	⬇️
...adcast/BroadcastShardOperationFailedException.java	`55.55% <0.00%> (-44.45%)`	⬇️
... and 474 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

gbbafna · 2023-01-04T13:18:34Z

I think we should let this functionality be as is . A local collector could use this to find out weights in a light weight manner , which otherwise would be heavy to go to the cluster manager.

Bukhtawar · 2023-01-04T13:28:09Z

I think we should let this functionality be as is . A local collector could use this to find out weights in a light weight manner , which otherwise would be heavy to go to the cluster manager.

@anshu1106 can correct me but this doesn't remove _local query but removes local_weight since it doesn't add much value as weights are relative. The caller is responsible to select specific weights based on the awareness attribute

gbbafna · 2023-01-04T13:35:48Z

I think we should let this functionality be as is . A local collector could use this to find out weights in a light weight manner , which otherwise would be heavy to go to the cluster manager.

@anshu1106 can correct me but this doesn't remove _local query but removes local_weight since it doesn't add much value as weights are relative. The caller is responsible to select specific weights based on the awareness attribute

Got it. My bad, i misread the title .

gbbafna · 2023-01-04T13:44:43Z

server/src/internalClusterTest/java/org/opensearch/cluster/routing/WeightedRoutingIT.java

@@ -219,7 +219,6 @@ public void testGetWeightedRouting_WeightsAreSet() throws IOException {
            .setRequestLocal(true)
            .get();
        assertEquals(weightedRouting, weightedRoutingResponse.weights());
-        assertEquals("1.0", weightedRoutingResponse.getLocalNodeWeight());


add discovered_cluster_manager assertion as well .

gbbafna · 2023-01-04T13:45:21Z

...arch/action/admin/cluster/shards/routing/weighted/get/ClusterGetWeightedRoutingResponse.java


-    public String getLocalNodeWeight() {
-        return localNodeWeight;
+    public Boolean getDiscoveredMaster() {


lets use cluster_manager everywhere . master is going to get deprecated in 3.0.

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

github-actions · 2023-01-04T15:16:44Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/8801/
CommitID: 1d904be

github-actions · 2023-01-04T15:36:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/8803/
CommitID: 4e15525

github-actions · 2023-01-05T12:43:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8873/
CommitID: 0dcfbbd
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-01-05T13:33:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/8882/
CommitID: dff4cce

Bukhtawar · 2023-01-07T17:05:22Z

...arch/action/admin/cluster/shards/routing/weighted/get/ClusterGetWeightedRoutingResponse.java

    ClusterGetWeightedRoutingResponse() {
        this.weightedRouting = null;
+        this.discoveredClusterManager = null;
    }


There is no explicit need to set them null

opensearch-trigger-bot · 2023-01-09T08:09:35Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-5680-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3eaf129b969d7c5dd9b502c747034bd60d8a4591
# Push it to GitHub
git push --set-upstream origin backport/backport-5680-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5680-to-2.x.

… weights in the response (opensearch-project#5680) * Add support for discovered master and remove local weights in the weighted routing API response Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

#5784) * [Weighted Routing] Add support for discovered master and remove local weights in the response (#5680) * Add support for discovered master and remove local weights in the weighted routing API response Signed-off-by: Anshu Agarwal <anshukag@amazon.com> * [Weighted Shard Routing] API versioning (#5255) * Support API versioning for weighted shard routing Signed-off-by: Anshu Agarwal <anshukag@amazon.com> * [Weighted Shard Routing] Fail open requests on search shard failures (#5072) * Fail open requests on search shard failures ( Signed-off-by: Anshu Agarwal <anshukag@amazon.com> * Address fail open comments (#5778) [Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting Signed-off-by: Anshu Agarwal <anshukag@amazon.com> * remove unintended changes in changelog Signed-off-by: Anshu Agarwal <anshukag@amazon.com> * remove unintended changes from changelog Signed-off-by: Anshu Agarwal <anshukag@amazon.com> Signed-off-by: Anshu Agarwal <anshukag@amazon.com> Co-authored-by: Anshu Agarwal <anshukag@amazon.com>

…ed shard routing (#5781) * [Backport 2.x] [Weighted Shard Routing] Add support for discovered master and remove local weights in the response #5680 [Weighted Shard Routing] API versioning #5255 [Weighted Shard Routing] Fail open requests on search shard failures #5072 [Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting #5778 Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Anshu Agarwal added 2 commits January 3, 2023 17:04

Initial Commit

70971dc

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Fix test

73fd9b8

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Fix test failure

ef429b2

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Fix test

5ad7e3c

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 changed the title ~~[Weighted Routing] Deprecate GET local node weight~~ [Weighted Routing] Add support for discovered master and remove local weights Jan 4, 2023

Add changelog

311363a

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

anshu1106 marked this pull request as ready for review January 4, 2023 08:40

anshu1106 requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, adnapibar, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja and dreamer-89 as code owners January 4, 2023 08:40

gbbafna requested changes Jan 4, 2023

View reviewed changes

Anshu Agarwal added 2 commits January 4, 2023 20:17

Address review comments

1d904be

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Fix comments

4e15525

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

gbbafna approved these changes Jan 5, 2023

View reviewed changes

Merge branch 'main' into feature/wrr-deprecate-get-local

dff4cce

anshu1106 force-pushed the feature/wrr-deprecate-get-local branch from 0dcfbbd to dff4cce Compare January 5, 2023 13:00

Bukhtawar changed the title ~~[Weighted Routing] Add support for discovered master and remove local weights~~ [Weighted Routing] Add support for discovered master and remove local weights in the response Jan 6, 2023

Bukhtawar reviewed Jan 7, 2023

View reviewed changes

Bukhtawar approved these changes Jan 7, 2023

View reviewed changes

Bukhtawar merged commit 3eaf129 into opensearch-project:main Jan 7, 2023

Bukhtawar added the backport 2.x Backport to 2.x branch label Jan 9, 2023

anshu1106 mentioned this pull request Jan 9, 2023

[META] Weighted Routing Issue List #4526

Open

20 tasks

anshu1106 mentioned this pull request Jan 10, 2023

[Backport 2.x] Add support for API versioning and fail open in weighted shard routing #5779

Closed

6 tasks

anshu1106 mentioned this pull request Jan 10, 2023

[Backport 2.x] Add support for API versioning and fail open in weighted shard routing #5781

Merged

6 tasks

anshu1106 mentioned this pull request Jan 10, 2023

[Backport 2.5] Add support for API versioning and fail open in weighted shard routing #5784

Merged

6 tasks

Bukhtawar mentioned this pull request Jan 19, 2023

Add support for discovered master and remove local weights #5407

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Weighted Routing] Add support for discovered master and remove local weights in the response #5680

[Weighted Routing] Add support for discovered master and remove local weights in the response #5680

anshu1106 commented Jan 3, 2023 •

edited

Loading

github-actions bot commented Jan 3, 2023

github-actions bot commented Jan 3, 2023

github-actions bot commented Jan 4, 2023

github-actions bot commented Jan 4, 2023

codecov-commenter commented Jan 4, 2023 •

edited

Loading

gbbafna commented Jan 4, 2023

Bukhtawar commented Jan 4, 2023

gbbafna commented Jan 4, 2023

gbbafna Jan 4, 2023

gbbafna Jan 4, 2023

github-actions bot commented Jan 4, 2023

github-actions bot commented Jan 4, 2023

github-actions bot commented Jan 5, 2023

github-actions bot commented Jan 5, 2023

Bukhtawar Jan 7, 2023

opensearch-trigger-bot bot commented Jan 9, 2023

[Weighted Routing] Add support for discovered master and remove local weights in the response #5680

[Weighted Routing] Add support for discovered master and remove local weights in the response #5680

Conversation

anshu1106 commented Jan 3, 2023 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Jan 3, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 3, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 4, 2023

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Jan 4, 2023 • edited Loading

Codecov Report

gbbafna commented Jan 4, 2023

Bukhtawar commented Jan 4, 2023

gbbafna commented Jan 4, 2023

gbbafna Jan 4, 2023

Choose a reason for hiding this comment

gbbafna Jan 4, 2023

Choose a reason for hiding this comment

github-actions bot commented Jan 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

Bukhtawar Jan 7, 2023

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Jan 9, 2023

anshu1106 commented Jan 3, 2023 •

edited

Loading

codecov-commenter commented Jan 4, 2023 •

edited

Loading