SnapshotShardsService Simplifications #38025
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rvice-simplifications
…rvice-simplifications
original-brownbear
added
>non-issue
:Distributed/Snapshot/Restore
|
Pinging @elastic/es-distributed |
…rvice-simplifications
|
@original-brownbear can you update this with #37686 and add me as reviewer once done? |
|
@ywelsch already did that, just forgot to update the PR description :D Thanks for the ping! => should be good for review now. |
ywelsch
approved these changes
Feb 1, 2019
| return shardSnapshots.get(snapshot); | ||
| synchronized (shardSnapshots) { | ||
| final Map<ShardId, IndexShardSnapshotStatus> current = shardSnapshots.get(snapshot); | ||
| return current == null ? null : new HashMap<>(current); |
There was a problem hiding this comment.
should this be an immutable map (i.e. wrapped in Collections.unmodifiableMap)?
There was a problem hiding this comment.
I don't really have a strong opinion here, but that seems a little redundant.
But my subjective sense of aesthetics would say:
The caller is free to do whatever with that Map since it's a copy already => just another needless level of indirection and more symbols when reading the code.
| @@ -220,160 +209,133 @@ public void beforeIndexShardClosed(ShardId shardId, @Nullable IndexShard indexSh | |||
| * @param event cluster state changed event | |||
| */ | |||
| private void processIndexShardSnapshots(ClusterChangedEvent event) { | |||
There was a problem hiding this comment.
perhaps only pass current cluster state to this method
There was a problem hiding this comment.
Yea, we can even just pass SnapshotsInProgress since we have that in the caller already :) Will simplify.
| Map<ShardId, IndexShardSnapshotStatus> startedShards = new HashMap<>(); | ||
| final Snapshot snapshot = entry.snapshot(); | ||
| Map<ShardId, IndexShardSnapshotStatus> snapshotShards = shardSnapshots.getOrDefault(snapshot, emptyMap()); | ||
| final String localNodeId = clusterService.localNode().getId(); |
There was a problem hiding this comment.
capture this outside of the loop
| for (SnapshotsInProgress.Entry entry : snapshotsInProgress.entries()) { | ||
| final State entryState = entry.state(); | ||
| if (entryState == State.STARTED) { | ||
| Map<ShardId, IndexShardSnapshotStatus> startedShards = new HashMap<>(); |
There was a problem hiding this comment.
perhaps create this lazily?
| public void doRun() { | ||
| final IndexShard indexShard = | ||
| indicesService.indexServiceSafe(shardId.getIndex()).getShardOrNull(shardId.id()); | ||
| assert indexId != null; |
There was a problem hiding this comment.
this assert could be further up where we assign indexId
…rvice-simplifications
original-brownbear
deleted the
snapshot-shards-service-simplifications
branch
|
@ywelsch thanks! |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 1, 2019
…ersion * elastic/master: SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126) Adding ml_settings entry to HLRC and Docs for deprecation_info (elastic#38118)
jasontedor
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Feb 1, 2019
* elastic/master: AwaitsFix testAbortedSnapshotDuringInitDoesNotStart (elastic#38227) Preserve ILM operation mode when creating new lifecycles (elastic#38134) Enable trace log in FollowerFailOverIT (elastic#38148) SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126)
jasontedor
added a commit
to gwbrown/elasticsearch
that referenced
this pull request
Feb 1, 2019
* elastic/master: (54 commits) Introduce retention leases versioning (elastic#37951) Correctly disable tests for FIPS JVMs (elastic#38214) AwaitsFix testAbortedSnapshotDuringInitDoesNotStart (elastic#38227) Preserve ILM operation mode when creating new lifecycles (elastic#38134) Enable trace log in FollowerFailOverIT (elastic#38148) SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126) Adding ml_settings entry to HLRC and Docs for deprecation_info (elastic#38118) Replace awaitBusy with assertBusy in atLeastDocsIndexed (elastic#38190) Adjust SearchRequest version checks (elastic#38181) AwaitsFix testClientSucceedsWithVerificationDisabled (elastic#38213) Zen2ify RareClusterStateIT (elastic#38184) ML: Fix error race condition on stop _all datafeeds and close _all jobs (elastic#38113) AwaitsFix PUT mapping with _doc on an index that has types (elastic#38204) Allow built-in monitoring_user role to call GET _xpack API (elastic#38060) Update geo_shape docs to include unsupported features (elastic#38138) [ML] Remove "8" prefixes from file structure finder timestamp formats (elastic#38016) Disable bwc tests while backporting elastic#38104 (elastic#38182) Enable TLSv1.3 by default for JDKs with support (elastic#38103) ...
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Feb 28, 2019
* Backport of various snapshot stability fixes from `master` to `6.7` * Includes elastic#38368, elastic#38025 and elastic#37612
original-brownbear
added a commit
that referenced
this pull request
Mar 1, 2019
original-brownbear
added a commit
that referenced
this pull request
Mar 4, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Inspired by @ywelsch 's work in https://github.com/ywelsch/elasticsearch/blob/snapshot-refactored/server/src/main/java/org/elasticsearch/snapshots/SnapshotShardsService.java
shardSnapshotsfield, we mutate it, explicitly removing entries from it in only a single spotstartNewShards(saves us two maps (keyed by snapshot) and iterations over them)