Constant request cluster ID mismatch (got X want X) on cluster boot

[cdloh]

Starting to initialise a new cluster.

I'm running etcd through a config file

name: post01
data-dir: /var/lib/etcd

listen-peer-urls: http://10.9.52.45:2380
initial-advertise-peer-urls: http://10.9.52.45:2380
listen-client-urls: http://10.9.52.45:2379,http://localhost:2379
advertise-client-urls: http://10.9.52.45:2379,http://localhost:2379
initial-cluster-token: 'etcd-patroni'

And running it via

sudo -u etcd /usr/local/bin/etcd --config-file /etc/etcd.conf.yaml

Will eventually be ran via systemd but running via cli to bootstrap

And it constantly complains about the cluster ID not matching

[root@post01 etcd]# sudo -u etcd /usr/local/bin/etcd --config-file /etc/etcd.conf.yaml
2020-10-01 07:53:58.045487 I | etcdmain: Loading server configuration from "/etc/etcd.conf.yaml"
2020-10-01 07:53:58.046495 I | etcdmain: etcd Version: 3.3.18
2020-10-01 07:53:58.046511 I | etcdmain: Git SHA: 3c8740a79
2020-10-01 07:53:58.046520 I | etcdmain: Go Version: go1.12.9
2020-10-01 07:53:58.046527 I | etcdmain: Go OS/Arch: linux/amd64
2020-10-01 07:53:58.046535 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 8
2020-10-01 07:53:58.046819 W | etcdmain: found invalid file/dir post01.etcd under data dir /var/lib/etcd (Ignore this if you are upgrading etcd)
2020-10-01 07:53:58.048170 I | embed: listening for peers on http://10.9.52.45:2380
2020-10-01 07:53:58.048242 I | embed: listening for client requests on 10.9.52.45:2379
2020-10-01 07:53:58.048721 I | embed: listening for client requests on localhost:2379
2020-10-01 07:53:58.054692 I | etcdserver: name = post01
2020-10-01 07:53:58.054720 I | etcdserver: data dir = /var/lib/etcd
2020-10-01 07:53:58.054727 I | etcdserver: member dir = /var/lib/etcd/member
2020-10-01 07:53:58.054734 I | etcdserver: heartbeat = 100ms
2020-10-01 07:53:58.054739 I | etcdserver: election = 1000ms
2020-10-01 07:53:58.054745 I | etcdserver: snapshot count = 100000
2020-10-01 07:53:58.054759 I | etcdserver: advertise client URLs = http://10.9.52.45:2379,http://localhost:2379
2020-10-01 07:53:58.054767 I | etcdserver: initial advertise peer URLs = http://10.9.52.45:2380
2020-10-01 07:53:58.054778 I | etcdserver: initial cluster = post01=http://10.9.52.45:2380
2020-10-01 07:53:58.058921 I | etcdserver: starting member 280074e36eed9680 in cluster 891a026fb2607182
name: post01
2020-10-01 07:53:58.058961 I | raft: 280074e36eed9680 became follower at term 0
2020-10-01 07:53:58.058982 I | raft: newRaft 280074e36eed9680 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2020-10-01 07:53:58.058990 I | raft: 280074e36eed9680 became follower at term 1
2020-10-01 07:53:58.064562 W | auth: simple token is not cryptographically signed
2020-10-01 07:53:58.066951 I | etcdserver: starting server... [version: 3.3.18, cluster version: to_be_decided]
2020-10-01 07:53:58.067074 I | etcdserver: 280074e36eed9680 as single-node; fast-forwarding 9 ticks (election ticks 10)
2020-10-01 07:53:58.067381 I | etcdserver/membership: added member 280074e36eed9680 [http://10.9.52.45:2380] to cluster 891a026fb2607182
2020-10-01 07:53:58.069189 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.138010 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.148297 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.150005 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.248711 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.250098 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.348582 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.349998 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.448665 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.450057 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.548595 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.550063 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.559403 I | raft: 280074e36eed9680 is starting a new election at term 1
2020-10-01 07:53:58.559436 I | raft: 280074e36eed9680 became candidate at term 2
2020-10-01 07:53:58.559457 I | raft: 280074e36eed9680 received MsgVoteResp from 280074e36eed9680 at term 2
2020-10-01 07:53:58.559475 I | raft: 280074e36eed9680 became leader at term 2
2020-10-01 07:53:58.559487 I | raft: raft.node: 280074e36eed9680 elected leader 280074e36eed9680 at term 2
2020-10-01 07:53:58.559694 I | etcdserver: setting up the initial cluster version to 3.3
2020-10-01 07:53:58.560881 N | etcdserver/membership: set the initial cluster version to 3.3
2020-10-01 07:53:58.560936 I | etcdserver/api: enabled capabilities for version 3.3
2020-10-01 07:53:58.560967 I | embed: ready to serve client requests
2020-10-01 07:53:58.561207 I | etcdserver: published {Name:post01 ClientURLs:[http://10.9.52.45:2379 http://localhost:2379]} to cluster 891a026fb2607182
2020-10-01 07:53:58.561293 I | embed: ready to serve client requests
2020-10-01 07:53:58.562596 N | embed: serving insecure client requests on 10.9.52.45:2379, this is strongly discouraged!
2020-10-01 07:53:58.562638 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
2020-10-01 07:53:58.562711 E | etcdmain: forgot to set Type=notify in systemd service file?
2020-10-01 07:53:58.648485 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.650081 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.748573 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.750199 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.848588 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.850052 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.948594 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:58.950075 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.048514 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.050128 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.148683 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.149983 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.248691 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.250281 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.348598 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.350064 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.448613 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.450082 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.548608 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
2020-10-01 07:53:59.550222 E | rafthttp: request cluster ID mismatch (got 3bf5856be24ec314 want 891a026fb2607182)
^C2020-10-01 07:53:59.644142 N | pkg/osutil: received interrupt signal, shutting down...
2020-10-01 07:53:59.644624 I | etcdserver: skipped leadership transfer for single member cluster
WARNING: 2020/10/01 07:53:59 grpc: addrConn.createTransport failed to connect to {localhost:2379 0  <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp: operation was canceled". Reconnecting...

[paulcaskey]

Remove everything under the data dir and start over: sudo rm -r /var/lib/etcd/*
Or try running etcd with --initial-cluster-state existing.
You need 3 hosts to start a cluster. Don't just start with 1 and expect it to work. If you want just one node, don't specify any peer or cluster options. If you want 3 nodes, line up all 3 config files at once with all 3 nodes listed in each other's cluster params. See https://etcd.io/docs/v3.3.13/op-guide/clustering/

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[ahrtr]

FYI. https://github.com/ahrtr/etcd-issues/blob/master/docs/cluster_id_mismatch.md