Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

persistNetworkIdentity cli option #5175

Merged
merged 4 commits into from
Mar 2, 2023
Merged

persistNetworkIdentity cli option #5175

merged 4 commits into from
Mar 2, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Feb 20, 2023

Motivation

  • To have a fixed peer id to integrate with other clients' nodes

Description

  • Add peerIdFile option as global option
  • Load peer id from peerIdFile, if it does exist it'll throw error Error: ENOENT: no such file or directory
  • Persist enr when node is shutdown

Closes #4992

@github-actions
Copy link
Contributor

github-actions bot commented Feb 20, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 6a35cc4 Previous: 34e72ae Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 492.47 us/op 535.85 us/op 0.92
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.529 us/op 45.623 us/op 1.02
BLS verify - blst-native 1.2296 ms/op 1.2157 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.5044 ms/op 2.4653 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.3377 ms/op 5.2932 ms/op 1.01
BLS verifyMultipleSignatures 32 - blst-native 19.280 ms/op 19.117 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 26.000 us/op 25.478 us/op 1.02
BLS aggregatePubkeys 128 - blst-native 100.55 us/op 99.942 us/op 1.01
getAttestationsForBlock 59.519 ms/op 97.201 ms/op 0.61
isKnown best case - 1 super set check 259.00 ns/op 320.00 ns/op 0.81
isKnown normal case - 2 super set checks 271.00 ns/op 312.00 ns/op 0.87
isKnown worse case - 16 super set checks 271.00 ns/op 300.00 ns/op 0.90
CheckpointStateCache - add get delete 6.0430 us/op 7.0420 us/op 0.86
validate gossip signedAggregateAndProof - struct 2.8455 ms/op 3.0269 ms/op 0.94
validate gossip attestation - struct 1.3397 ms/op 1.4731 ms/op 0.91
pickEth1Vote - no votes 1.3428 ms/op 1.5765 ms/op 0.85
pickEth1Vote - max votes 10.864 ms/op 11.992 ms/op 0.91
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.3259 ms/op 11.117 ms/op 0.84
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.444 ms/op 16.539 ms/op 0.87
pickEth1Vote - Eth1Data fastSerialize value x2048 672.80 us/op 787.95 us/op 0.85
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.3530 ms/op 9.4969 ms/op 0.56
bytes32 toHexString 539.00 ns/op 830.00 ns/op 0.65
bytes32 Buffer.toString(hex) 383.00 ns/op 463.00 ns/op 0.83
bytes32 Buffer.toString(hex) from Uint8Array 615.00 ns/op 576.00 ns/op 1.07
bytes32 Buffer.toString(hex) + 0x 415.00 ns/op 382.00 ns/op 1.09
Object access 1 prop 0.20600 ns/op 0.17400 ns/op 1.18
Map access 1 prop 0.17500 ns/op 0.16200 ns/op 1.08
Object get x1000 7.4250 ns/op 6.6220 ns/op 1.12
Map get x1000 0.63000 ns/op 0.61400 ns/op 1.03
Object set x1000 65.808 ns/op 84.800 ns/op 0.78
Map set x1000 53.650 ns/op 57.007 ns/op 0.94
Return object 10000 times 0.26690 ns/op 0.25320 ns/op 1.05
Throw Error 10000 times 4.3532 us/op 4.3315 us/op 1.00
fastMsgIdFn sha256 / 200 bytes 3.6940 us/op 3.6020 us/op 1.03
fastMsgIdFn h32 xxhash / 200 bytes 317.00 ns/op 303.00 ns/op 1.05
fastMsgIdFn h64 xxhash / 200 bytes 449.00 ns/op 521.00 ns/op 0.86
fastMsgIdFn sha256 / 1000 bytes 12.159 us/op 12.276 us/op 0.99
fastMsgIdFn h32 xxhash / 1000 bytes 442.00 ns/op 444.00 ns/op 1.00
fastMsgIdFn h64 xxhash / 1000 bytes 512.00 ns/op 495.00 ns/op 1.03
fastMsgIdFn sha256 / 10000 bytes 107.51 us/op 106.02 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 1.9980 us/op 2.0170 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.4190 us/op 1.4680 us/op 0.97
enrSubnets - fastDeserialize 64 bits 1.3790 us/op 1.4280 us/op 0.97
enrSubnets - ssz BitVector 64 bits 540.00 ns/op 551.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 181.00 ns/op 190.00 ns/op 0.95
enrSubnets - ssz BitVector 4 bits 526.00 ns/op 584.00 ns/op 0.90
prioritizePeers score -10:0 att 32-0.1 sync 2-0 99.024 us/op 111.75 us/op 0.89
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 141.69 us/op 138.26 us/op 1.02
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 194.56 us/op 177.07 us/op 1.10
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 352.15 us/op 327.81 us/op 1.07
prioritizePeers score 0:0 att 64-1 sync 4-1 415.75 us/op 387.69 us/op 1.07
array of 16000 items push then shift 1.7823 us/op 1.6852 us/op 1.06
LinkedList of 16000 items push then shift 9.8330 ns/op 9.1670 ns/op 1.07
array of 16000 items push then pop 116.96 ns/op 110.16 ns/op 1.06
LinkedList of 16000 items push then pop 10.758 ns/op 8.7410 ns/op 1.23
array of 24000 items push then shift 2.4451 us/op 2.3855 us/op 1.02
LinkedList of 24000 items push then shift 9.4940 ns/op 9.0640 ns/op 1.05
array of 24000 items push then pop 94.643 ns/op 79.635 ns/op 1.19
LinkedList of 24000 items push then pop 8.9340 ns/op 8.9430 ns/op 1.00
intersect bitArray bitLen 8 13.575 ns/op 13.564 ns/op 1.00
intersect array and set length 8 93.042 ns/op 80.883 ns/op 1.15
intersect bitArray bitLen 128 47.192 ns/op 44.842 ns/op 1.05
intersect array and set length 128 1.3435 us/op 1.0710 us/op 1.25
Buffer.concat 32 items 2.9580 us/op 3.0050 us/op 0.98
Uint8Array.set 32 items 3.3580 us/op 2.1410 us/op 1.57
pass gossip attestations to forkchoice per slot 2.6421 ms/op 3.3889 ms/op 0.78
computeDeltas 3.4064 ms/op 2.9462 ms/op 1.16
computeProposerBoostScoreFromBalances 1.8703 ms/op 1.9662 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 2.3853 ms/op 3.1241 ms/op 0.76
altair processAttestation - 250000 vs - 7PWei worstcase 3.3841 ms/op 3.5957 ms/op 0.94
altair processAttestation - setStatus - 1/6 committees join 143.48 us/op 189.53 us/op 0.76
altair processAttestation - setStatus - 1/3 committees join 292.64 us/op 308.12 us/op 0.95
altair processAttestation - setStatus - 1/2 committees join 379.77 us/op 382.69 us/op 0.99
altair processAttestation - setStatus - 2/3 committees join 480.73 us/op 469.35 us/op 1.02
altair processAttestation - setStatus - 4/5 committees join 664.72 us/op 678.95 us/op 0.98
altair processAttestation - setStatus - 100% committees join 780.12 us/op 779.98 us/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 18.087 ms/op 18.102 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.865 ms/op 26.732 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 50.480 ms/op 51.036 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 67.874 ms/op 67.045 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2234 ms/op 2.0973 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei worstcase 28.973 ms/op 26.886 ms/op 1.08
altair processEth1Data - 250000 vs - 7PWei normalcase 463.54 us/op 468.71 us/op 0.99
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 8.1970 us/op 8.6650 us/op 0.95
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 25.761 us/op 22.971 us/op 1.12
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.240 us/op 9.5000 us/op 1.08
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 9.1770 us/op 6.5610 us/op 1.40
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 100.91 us/op 94.435 us/op 1.07
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 653.65 us/op 605.25 us/op 1.08
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 932.39 us/op 907.16 us/op 1.03
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 909.68 us/op 907.44 us/op 1.00
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.4388 ms/op 2.3450 ms/op 1.04
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5531 ms/op 1.4811 ms/op 1.05
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 4.0239 ms/op 3.8294 ms/op 1.05
Tree 40 250000 create 316.51 ms/op 323.23 ms/op 0.98
Tree 40 250000 get(125000) 194.04 ns/op 187.05 ns/op 1.04
Tree 40 250000 set(125000) 987.47 ns/op 876.08 ns/op 1.13
Tree 40 250000 toArray() 21.225 ms/op 20.380 ms/op 1.04
Tree 40 250000 iterate all - toArray() + loop 22.013 ms/op 20.104 ms/op 1.09
Tree 40 250000 iterate all - get(i) 75.091 ms/op 74.397 ms/op 1.01
MutableVector 250000 create 11.034 ms/op 11.413 ms/op 0.97
MutableVector 250000 get(125000) 6.8420 ns/op 6.4510 ns/op 1.06
MutableVector 250000 set(125000) 245.66 ns/op 277.32 ns/op 0.89
MutableVector 250000 toArray() 3.1269 ms/op 3.2291 ms/op 0.97
MutableVector 250000 iterate all - toArray() + loop 3.2281 ms/op 3.1329 ms/op 1.03
MutableVector 250000 iterate all - get(i) 1.5305 ms/op 1.5560 ms/op 0.98
Array 250000 create 2.8298 ms/op 2.6679 ms/op 1.06
Array 250000 clone - spread 1.1222 ms/op 1.1734 ms/op 0.96
Array 250000 get(125000) 0.53000 ns/op 0.62900 ns/op 0.84
Array 250000 set(125000) 0.61200 ns/op 0.71100 ns/op 0.86
Array 250000 iterate all - loop 103.15 us/op 86.925 us/op 1.19
effectiveBalanceIncrements clone Uint8Array 300000 29.164 us/op 30.238 us/op 0.96
effectiveBalanceIncrements clone MutableVector 300000 329.00 ns/op 361.00 ns/op 0.91
effectiveBalanceIncrements rw all Uint8Array 300000 172.37 us/op 171.26 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 79.028 ms/op 84.372 ms/op 0.94
phase0 afterProcessEpoch - 250000 vs - 7PWei 114.94 ms/op 117.24 ms/op 0.98
phase0 beforeProcessEpoch - 250000 vs - 7PWei 41.431 ms/op 41.433 ms/op 1.00
altair processEpoch - mainnet_e81889 310.01 ms/op 333.97 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 68.671 ms/op 52.114 ms/op 1.32
mainnet_e81889 - altair processJustificationAndFinalization 16.882 us/op 16.483 us/op 1.02
mainnet_e81889 - altair processInactivityUpdates 6.1353 ms/op 5.7021 ms/op 1.08
mainnet_e81889 - altair processRewardsAndPenalties 49.112 ms/op 64.709 ms/op 0.76
mainnet_e81889 - altair processRegistryUpdates 2.4030 us/op 2.7620 us/op 0.87
mainnet_e81889 - altair processSlashings 528.00 ns/op 666.00 ns/op 0.79
mainnet_e81889 - altair processEth1DataReset 560.00 ns/op 540.00 ns/op 1.04
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2456 ms/op 1.2746 ms/op 0.98
mainnet_e81889 - altair processSlashingsReset 4.9110 us/op 4.6610 us/op 1.05
mainnet_e81889 - altair processRandaoMixesReset 4.2240 us/op 4.9010 us/op 0.86
mainnet_e81889 - altair processHistoricalRootsUpdate 605.00 ns/op 955.00 ns/op 0.63
mainnet_e81889 - altair processParticipationFlagUpdates 2.4260 us/op 3.2560 us/op 0.75
mainnet_e81889 - altair processSyncCommitteeUpdates 591.00 ns/op 785.00 ns/op 0.75
mainnet_e81889 - altair afterProcessEpoch 120.58 ms/op 132.77 ms/op 0.91
phase0 processEpoch - mainnet_e58758 320.48 ms/op 385.13 ms/op 0.83
mainnet_e58758 - phase0 beforeProcessEpoch 120.15 ms/op 146.94 ms/op 0.82
mainnet_e58758 - phase0 processJustificationAndFinalization 17.432 us/op 16.982 us/op 1.03
mainnet_e58758 - phase0 processRewardsAndPenalties 56.687 ms/op 61.167 ms/op 0.93
mainnet_e58758 - phase0 processRegistryUpdates 7.9160 us/op 7.8860 us/op 1.00
mainnet_e58758 - phase0 processSlashings 502.00 ns/op 480.00 ns/op 1.05
mainnet_e58758 - phase0 processEth1DataReset 487.00 ns/op 468.00 ns/op 1.04
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 962.29 us/op 991.82 us/op 0.97
mainnet_e58758 - phase0 processSlashingsReset 3.6290 us/op 3.6380 us/op 1.00
mainnet_e58758 - phase0 processRandaoMixesReset 4.5880 us/op 4.5300 us/op 1.01
mainnet_e58758 - phase0 processHistoricalRootsUpdate 554.00 ns/op 534.00 ns/op 1.04
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.4860 us/op 4.2600 us/op 1.05
mainnet_e58758 - phase0 afterProcessEpoch 94.110 ms/op 93.334 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2494 ms/op 1.2237 ms/op 1.02
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4678 ms/op 1.4513 ms/op 1.01
altair processInactivityUpdates - 250000 normalcase 18.554 ms/op 24.693 ms/op 0.75
altair processInactivityUpdates - 250000 worstcase 25.716 ms/op 25.898 ms/op 0.99
phase0 processRegistryUpdates - 250000 normalcase 6.7030 us/op 6.5710 us/op 1.02
phase0 processRegistryUpdates - 250000 badcase_full_deposits 231.89 us/op 230.08 us/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 126.03 ms/op 124.10 ms/op 1.02
altair processRewardsAndPenalties - 250000 normalcase 68.370 ms/op 69.066 ms/op 0.99
altair processRewardsAndPenalties - 250000 worstcase 68.564 ms/op 70.527 ms/op 0.97
phase0 getAttestationDeltas - 250000 normalcase 6.7724 ms/op 6.7383 ms/op 1.01
phase0 getAttestationDeltas - 250000 worstcase 6.5664 ms/op 6.6174 ms/op 0.99
phase0 processSlashings - 250000 worstcase 3.2267 ms/op 3.6486 ms/op 0.88
altair processSyncCommitteeUpdates - 250000 174.00 ms/op 168.76 ms/op 1.03
BeaconState.hashTreeRoot - No change 318.00 ns/op 312.00 ns/op 1.02
BeaconState.hashTreeRoot - 1 full validator 53.065 us/op 55.012 us/op 0.96
BeaconState.hashTreeRoot - 32 full validator 516.70 us/op 541.58 us/op 0.95
BeaconState.hashTreeRoot - 512 full validator 5.2315 ms/op 5.1751 ms/op 1.01
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 61.742 us/op 63.712 us/op 0.97
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 919.87 us/op 977.32 us/op 0.94
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.457 ms/op 11.609 ms/op 0.99
BeaconState.hashTreeRoot - 1 balances 48.002 us/op 53.372 us/op 0.90
BeaconState.hashTreeRoot - 32 balances 440.07 us/op 454.32 us/op 0.97
BeaconState.hashTreeRoot - 512 balances 4.2547 ms/op 4.5757 ms/op 0.93
BeaconState.hashTreeRoot - 250000 balances 75.756 ms/op 78.941 ms/op 0.96
aggregationBits - 2048 els - zipIndexesInBitList 15.052 us/op 16.279 us/op 0.92
regular array get 100000 times 32.064 us/op 42.697 us/op 0.75
wrappedArray get 100000 times 32.068 us/op 33.077 us/op 0.97
arrayWithProxy get 100000 times 15.490 ms/op 15.457 ms/op 1.00
ssz.Root.equals 522.00 ns/op 561.00 ns/op 0.93
byteArrayEquals 518.00 ns/op 543.00 ns/op 0.95
shuffle list - 16384 els 6.6137 ms/op 7.1553 ms/op 0.92
shuffle list - 250000 els 96.362 ms/op 103.24 ms/op 0.93
processSlot - 1 slots 8.5190 us/op 9.2430 us/op 0.92
processSlot - 32 slots 1.3319 ms/op 1.3571 ms/op 0.98
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 189.57 us/op 202.56 us/op 0.94
getCommitteeAssignments - req 1 vs - 250000 vc 2.9022 ms/op 2.9657 ms/op 0.98
getCommitteeAssignments - req 100 vs - 250000 vc 4.0317 ms/op 4.2274 ms/op 0.95
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3623 ms/op 4.5079 ms/op 0.97
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6400 ns/op 4.7100 ns/op 0.99
state getBlockRootAtSlot - 250000 vs - 7PWei 904.23 ns/op 751.69 ns/op 1.20
computeProposers - vc 250000 9.9456 ms/op 10.838 ms/op 0.92
computeEpochShuffling - vc 250000 97.997 ms/op 103.28 ms/op 0.95
getNextSyncCommittee - vc 250000 165.02 ms/op 174.48 ms/op 0.95

by benchmarkbot/action

@twoeths twoeths marked this pull request as ready for review February 20, 2023 09:49
@twoeths twoeths requested a review from a team as a code owner February 20, 2023 09:49
@@ -140,7 +145,7 @@ export async function beaconHandlerInit(args: BeaconArgs & GlobalArgs) {
}

// Create new PeerId everytime by default, unless peerIdFile is provided
const peerId = await createSecp256k1PeerId();
const peerId = args.peerIdFile ? await readPeerId(args.peerIdFile) : await createSecp256k1PeerId();
const enr = SignableENR.createV4(createKeypairFromPeerId(peerId));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need something like:

// If peerIdFile
//   attempt to read enr from disk, beaconDir/enr
//   accept enr if enr.peerId == peerId
// else
//    create new enr

Otherwise, when using peerIdFile, then restarting, the discv5 network will ignore our enr updates since the enr seqno has been reset.

@wemeetagain wemeetagain changed the title peerIdFile cli option persistNetworkIdentity cli option Feb 23, 2023
@wemeetagain wemeetagain closed this Mar 1, 2023
@wemeetagain wemeetagain reopened this Mar 1, 2023
@wemeetagain wemeetagain merged commit e2312e0 into unstable Mar 2, 2023
@wemeetagain wemeetagain deleted the tuyen/peerIdFile branch March 2, 2023 13:48
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add CLI option to persist peerId and ENR
2 participants