Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no-store_v2: Store ConfState as part of WAL log snapshot #12735

Merged
merged 3 commits into from
Mar 25, 2021

Conversation

ptabor
Copy link
Contributor

@ptabor ptabor commented Feb 28, 2021

WAL log snapshots used to contain only Term & Index.
That made them diverging from raft.Snapshot metadata that contained ConfState as well
(list of Voters, Listeners, and candidating voters & listeners).
So far lookup of the full ConfState required dive into *.snap file that is reminiscence of store_v2.

I'm working on preparing for decomissioning of store_v2 in 3.6, so in particular etcd-3.5
should be able to run in situation that *.snap files does not exist (to be rollback ready).

@codecov-io
Copy link

codecov-io commented Feb 28, 2021

Codecov Report

Merging #12735 (2393c0d) into master (d06d93d) will decrease coverage by 2.64%.
The diff coverage is 71.42%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #12735      +/-   ##
==========================================
- Coverage   72.76%   70.11%   -2.65%     
==========================================
  Files         422      413       -9     
  Lines       33018    32063     -955     
==========================================
- Hits        24024    22481    -1543     
- Misses       7077     7681     +604     
+ Partials     1917     1901      -16     
Flag Coverage Δ
all 70.11% <71.42%> (-2.65%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/wal/wal.go 55.06% <0.00%> (-0.23%) ⬇️
server/wal/walpb/record.go 71.42% <33.33%> (-28.58%) ⬇️
etcdctl/ctlv2/command/backup_command.go 65.95% <100.00%> (ø)
etcdctl/snapshot/v3_snapshot.go 62.55% <100.00%> (-0.18%) ⬇️
server/etcdserver/storage.go 51.35% <100.00%> (+1.35%) ⬆️
pkg/tlsutil/cipher_suites.go 0.00% <0.00%> (-100.00%) ⬇️
...ver/proxy/grpcproxy/adapter/lock_client_adapter.go 0.00% <0.00%> (-100.00%) ⬇️
pkg/srv/srv.go 0.00% <0.00%> (-87.94%) ⬇️
pkg/osutil/interrupt_unix.go 0.00% <0.00%> (-82.61%) ⬇️
pkg/types/set.go 15.87% <0.00%> (-76.20%) ⬇️
... and 107 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d06d93d...2393c0d. Read the comment docs.

@ptabor ptabor force-pushed the 20210227-reduce-store-v2-clean branch 2 times, most recently from 92817d7 to 2393c0d Compare March 1, 2021 17:11
@ptabor ptabor added this to the etcd-v3.5 milestone Mar 1, 2021
func TestNew(t *testing.T) {
p, err := ioutil.TempDir(os.TempDir(), "waltest")
p, err := ioutil.TempDir(t.TempDir(), "waltest")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

@@ -942,6 +941,10 @@ func (w *WAL) Save(st raftpb.HardState, ents []raftpb.Entry) error {
}

func (w *WAL) SaveSnapshot(e walpb.Snapshot) error {
if err := walpb.ValidateSnapshotForWrite(&e); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say a, b, c

And c as a follower runs this code against a as a leader with old etcd version. Would the follower reject the snapshot from old-versioned leader?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there would be no problem as raft is speaking using raftpb.Snapshot

That always contains

optional SnapshotMetadata metadata = 2 [(gogoproto.nullable) = false];

with:
optional ConfState conf_state = 1 [(gogoproto.nullable) = false];
.

This validation just makes sure we never forgot to copy the content from raftpb.Snapshot to walpb.Snapshot.
That cannonically happens in server/etcdserver/storage.go (line 59)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ptabor +1. Yeah, this is for saving the snap locally.

@ptabor ptabor requested a review from gyuho March 3, 2021 20:34
@ptabor ptabor force-pushed the 20210227-reduce-store-v2-clean branch from 2393c0d to 917895e Compare March 4, 2021 21:33
@ptabor
Copy link
Contributor Author

ptabor commented Mar 8, 2021

@gyuho Shell I merge, or your concern remains ?

@gyuho
Copy link
Contributor

gyuho commented Mar 9, 2021

/cc @xiang90

@ptabor ptabor mentioned this pull request Mar 24, 2021
5 tasks
This will (among others) allow etcd-3.6 to not depend on store_v2 .snap files at all,
as WAL + db file will be self-sufficient.
@ptabor ptabor force-pushed the 20210227-reduce-store-v2-clean branch from 917895e to 4d4c84e Compare March 24, 2021 23:37
@ptabor ptabor merged commit 8ee1dd9 into etcd-io:master Mar 25, 2021
@ptabor ptabor deleted the 20210227-reduce-store-v2-clean branch March 25, 2021 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants