Add Package Level JavaDoc on Snapshots #38108
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| /* | ||
| * Licensed to Elasticsearch under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| /** | ||
| * <p>This package exposes the Elasticsearch Snapshot functionality.</p> | ||
| * | ||
| * <h1>Preliminaries</h1> | ||
| * | ||
| * <p>There are two communication channels between all nodes and master in the snapshot functionality:</p> | ||
| * <ul> | ||
| * <li>The master updates the cluster state by adding, removing or altering the contents of its custom entry | ||
| * {@link org.elasticsearch.cluster.SnapshotsInProgress}. All nodes consume the state of the {@code SnapshotsInProgress} and will start or | ||
| * abort relevant shard snapshot tasks accordingly.</li> | ||
| * <li>Nodes that are executing shard snapshot tasks report either success or failure of their snapshot task by submitting a | ||
| * {@link org.elasticsearch.snapshots.SnapshotShardsService.UpdateIndexShardSnapshotStatusRequest} to the master node that will | ||
| * update the snapshot's entry in the cluster-state accordingly.</li> | ||
| * </ul> | ||
| * | ||
| * <h1>Snapshot Creation</h1> | ||
| * <p>Snapshots are created by the following sequence of events:</p> | ||
| * <ol> | ||
| * <li>An invocation of {@link org.elasticsearch.snapshots.SnapshotsService#createSnapshot} enqueues a cluster state update to create | ||
| * a {@link org.elasticsearch.cluster.SnapshotsInProgress.Entry} in the cluster-state's {@code SnapshotsInProgress}. | ||
| * This initial snapshot entry has its state set to {@code INIT} and an empty map set for the state of the individual shard's snapshots. | ||
| * </li> | ||
| * | ||
| * <li>After the snapshot's entry with state {@code INIT} is in the cluster-state, | ||
| * {@link org.elasticsearch.snapshots.SnapshotsService} determines the primary shards' assignments for all indices that are being | ||
| * snapshotted and updates the existing {@code SnapshotsInProgress.Entry} with state {@code STARTED} and adds the map of | ||
| * {@link org.elasticsearch.index.shard.ShardId} to {@link org.elasticsearch.cluster.SnapshotsInProgress.ShardSnapshotStatus} that tracks | ||
| * the assignment of which node is to snapshot which shard. All shard snapshots are executed on the shard's primary node. | ||
| * Thus all shards for which the primary node was found to have a healthy copy of the shard are marked as being in state {@code INIT} in | ||
| * this map. | ||
| * If the primary for a shard is unassigned, it is marked as {@code MISSING} in this | ||
| * map. In case the primary is initializing at this point, it is marked as in state | ||
| * {@code WAITING}.</li> | ||
| * | ||
| * <li>The new {@code SnapshotsInProgress.Entry} is then observed by | ||
| * {@link org.elasticsearch.snapshots.SnapshotShardsService#clusterChanged} on all nodes and since the entry is in state | ||
| * {@code STARTED} the {@code SnapshotShardsService} will check if any local primary shards are to be snapshotted | ||
| * (signaled by the shard's snapshot state being {@code INIT}). | ||
| * For those local primary shards found in state {@code INIT}) the snapshot process | ||
| * of writing the shard's data files to the snapshot's {@link org.elasticsearch.repositories.Repository} is executed. | ||
| * Once the snapshot execution finishes for a shard an {@code UpdateIndexShardSnapshotStatusRequest} is sent to the master node signaling | ||
| * either status {@code SUCCESS} or {@code FAILED}. | ||
| * The master node will then update a shard's state in the snapshots {@code SnapshotsInProgress.Entry} whenever | ||
| * it receives such a {@code UpdateIndexShardSnapshotStatusRequest}.</li> | ||
| * | ||
| * <li>If as a result of the received status update requests, all shards in the cluster-state are in a completed state, i.e are marked as | ||
| * either {@code SUCCESS}, {@code FAILED} or {@code MISSING}, the {@code SnapshotShardsService} | ||
| * will update the state of the {@code Entry} itself and mark it as {@code SUCCESS}. | ||
| * At the same time {@link org.elasticsearch.snapshots.SnapshotsService#endSnapshot} is executed, writing the metadata necessary to | ||
| * finalize the snapshot in the repository to the repository.</li> | ||
| * | ||
| * <li>After writing the final metadata to the repository, a cluster-state update to remove the snapshot from the repository is submitted | ||
original-brownbear marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * and the removal of the snapshot's {@code SnapshotsInProgress.Entry} from the cluster-state completes the snapshot process.</li> | ||
| * </ol> | ||
| * | ||
| * <h1>Deleting a Snapshot</h1> | ||
| * | ||
| * <p>Deleting a snapshot can take the form of either simply deleting it from the repository or (if it has not completed yet) aborting it | ||
| * and subsequently deleting it from the repository.</p> | ||
| * | ||
| * <h2>Aborting a Snapshot</h2> | ||
| * | ||
| * <ol> | ||
| * <li>Aborting a snapshot starts by updating the state of the snapshot's {@code SnapshotsInProgress.Entry} to {@code ABORTED}.</li> | ||
| * | ||
| * <li>The snapshot's state change to {@code ABORTED} in cluster state is then picked up by the {@code SnapshotShardsService} on all nodes. | ||
| * Those nodes that have shard snapshot actions for the snapshot assigned to them, will abort them and notify master about the shards | ||
| * snapshot status accordingly. If the shard snapshot action completed or was in state {@code FINALIZE} when the abort was registered by | ||
| * the {@code SnapshotShardsService}, then the shard's state will be reported to master as {@code SUCCESS}. | ||
| * Otherwise, it will be reported as {@code FAILED}.</li> | ||
| * | ||
| * <li>Once all the shards are reported to master as either {@code SUCCESS} or | ||
| * {@code FAILED} the {@code SnapshotsService} on the master will finish the snapshot process as all shard's | ||
| * states are now completed and hence the snapshot can be completed as explained in point 4 of the snapshot creation section above.</li> | ||
| * </ol> | ||
| * | ||
| * <h2>Deleting a Snapshot from a Repository</h2> | ||
| * | ||
| * <ol> | ||
| * <li>Assuming there are no entries in the cluster-state's {@code SnapshotsInProgress}, deleting a snapshot | ||
|
There was a problem hiding this comment. It'd also be interesting to hear how we delete a snapshot after aborting it, i.e. what if this There was a problem hiding this comment. This is covered in the paragraph above, that's why I put There was a problem hiding this comment. The detail I felt we were missing is how we achieve that "after" relationship. Aborting a snapshot isn't atomic, it requires acks from all the shards first, so how do we know to delete the snapshot after aborting it? It looks like we add a listener on the current master so all is good as long as we don't elect a new master first. There was a problem hiding this comment. Yea true, the delete after abort will not happen if we have a master failover. Then the abort will go through clean but no delete will happen subsequently. That's something we could maybe fix I guess but I'm not sure it's worth the risk (at least at this point in time). |
||
| * starts by the {@code SnapshotsService} creating an entry for deleting the snapshot in | ||
| * the cluster-state's {@link org.elasticsearch.cluster.SnapshotDeletionsInProgress}.</li> | ||
| * | ||
| * <li>Once the cluster-state contains the deletion entry in {@code SnapshotDeletionsInProgress} the {@code SnapshotsService} will invoke | ||
| * {@link org.elasticsearch.repositories.Repository#deleteSnapshot} for the given snapshot, which will remove files associated with the | ||
| * snapshot from the repository as well as update its meta-data to reflect the deletion of the snapshot.</li> | ||
| * | ||
| * <li>After the deletion of the snapshot's data from the repository finishes, the {@code SnapshotsService} will submit a cluster-state | ||
| * update to remove the deletion's entry in {@code SnapshotDeletionsInProgress} which concludes the process of | ||
| * deleting a snapshot.</li> | ||
| * </ol> | ||
| */ | ||
| package org.elasticsearch.snapshots; | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This raise the question of what happens if a primary shard moves in between the creation of the
SnapshotsInProgressentry and the time when it is populated by theSnapshotsService. It'd be good to hear more on that.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a sentence on that in a699b72, hope it helps. Relocation between
INITandSTARTEDisn't really an issue since the assignment of nodes and moving toSTARTEDhappens in the sameClusterStateUpdateTask(Maybe make that more explicit in the docs? I opted not to since it seemed too detailed here but maybe not?).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, the addition is helpful. Is it right that we do not start to gracefully relocate a primary while it is being snapshotted (i.e. we have
SnapshotInProgressAllocationDecider), so that if the primary moves later then it's a genuine failure?What do we do with a primary that is already being relocated when the snapshot starts? Do we use the relocation target (which is initialising, so goes into the
WAITINGstate)?I'm still getting to grips with all of this, so perhaps I'm going into too much detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that's exactly right :) I felt like it was more confusing than helpful to add the "not graceful" qualifier to the text, but maybe I'm wrong?
Yea exactly, we have this in
SnapshotsService:What do we do with a primary that is already being relocated when the snapshot starts? Do we use the relocation target (which is initialising, so goes into the
WAITINGstate)?