Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add strategy support for state archives #7170

Open
wants to merge 6 commits into
base: unstable
Choose a base branch
from

Conversation

nazarhussain
Copy link
Contributor

Motivation

Make the state-archive process more generic so we can extend in future.

Description

  • Make the state-archive process to generic
  • Add feature flag to the cli to select a strategy

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner October 16, 2024 14:15
@nazarhussain nazarhussain self-assigned this Oct 16, 2024
@@ -0,0 +1,163 @@
import {Logger} from "@lodestar/utils";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is actually old index.ts file, with only strategy specific changes.

Copy link
Contributor

github-actions bot commented Oct 16, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c158658 Previous: de0d6ab Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9961 ms/op 1.7183 ms/op 1.16
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 41.180 us/op 37.146 us/op 1.11
BLS verify - blst 895.39 us/op 905.57 us/op 0.99
BLS verifyMultipleSignatures 3 - blst 1.7205 ms/op 1.3407 ms/op 1.28
BLS verifyMultipleSignatures 8 - blst 1.6922 ms/op 2.5955 ms/op 0.65
BLS verifyMultipleSignatures 32 - blst 4.4379 ms/op 4.5610 ms/op 0.97
BLS verifyMultipleSignatures 64 - blst 8.7083 ms/op 8.4932 ms/op 1.03
BLS verifyMultipleSignatures 128 - blst 16.752 ms/op 16.014 ms/op 1.05
BLS deserializing 10000 signatures 639.56 ms/op 626.15 ms/op 1.02
BLS deserializing 100000 signatures 6.3851 s/op 6.3426 s/op 1.01
BLS verifyMultipleSignatures - same message - 3 - blst 970.54 us/op 947.47 us/op 1.02
BLS verifyMultipleSignatures - same message - 8 - blst 1.0418 ms/op 1.0534 ms/op 0.99
BLS verifyMultipleSignatures - same message - 32 - blst 1.6604 ms/op 1.6538 ms/op 1.00
BLS verifyMultipleSignatures - same message - 64 - blst 2.4675 ms/op 2.4378 ms/op 1.01
BLS verifyMultipleSignatures - same message - 128 - blst 4.0200 ms/op 4.2739 ms/op 0.94
BLS aggregatePubkeys 32 - blst 18.738 us/op 17.835 us/op 1.05
BLS aggregatePubkeys 128 - blst 63.589 us/op 63.594 us/op 1.00
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.739 ms/op 70.717 ms/op 0.90
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 51.401 ms/op 41.418 ms/op 1.24
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 48.120 ms/op 41.765 ms/op 1.15
getSlashingsAndExits - default max 85.865 us/op 63.632 us/op 1.35
getSlashingsAndExits - 2k 417.64 us/op 247.29 us/op 1.69
proposeBlockBody type=full, size=empty 6.6482 ms/op 4.9248 ms/op 1.35
isKnown best case - 1 super set check 892.00 ns/op 465.00 ns/op 1.92
isKnown normal case - 2 super set checks 910.00 ns/op 457.00 ns/op 1.99
isKnown worse case - 16 super set checks 850.00 ns/op 461.00 ns/op 1.84
InMemoryCheckpointStateCache - add get delete 3.7510 us/op 2.8650 us/op 1.31
updateUnfinalizedPubkeys - updating 10 pubkeys 1.7592 ms/op 804.08 us/op 2.19
updateUnfinalizedPubkeys - updating 100 pubkeys 4.4302 ms/op 3.1182 ms/op 1.42
updateUnfinalizedPubkeys - updating 1000 pubkeys 50.339 ms/op 41.093 ms/op 1.22
validate api signedAggregateAndProof - struct 1.5928 ms/op 1.4519 ms/op 1.10
validate gossip signedAggregateAndProof - struct 1.7304 ms/op 2.0245 ms/op 0.85
batch validate gossip attestation - vc 640000 - chunk 32 130.78 us/op 126.33 us/op 1.04
batch validate gossip attestation - vc 640000 - chunk 64 111.01 us/op 109.96 us/op 1.01
batch validate gossip attestation - vc 640000 - chunk 128 103.60 us/op 105.89 us/op 0.98
batch validate gossip attestation - vc 640000 - chunk 256 113.30 us/op 99.791 us/op 1.14
pickEth1Vote - no votes 1.0133 ms/op 840.40 us/op 1.21
pickEth1Vote - max votes 6.7953 ms/op 8.3039 ms/op 0.82
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.852 ms/op 20.749 ms/op 0.72
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.291 ms/op 31.035 ms/op 0.78
pickEth1Vote - Eth1Data fastSerialize value x2048 372.26 us/op 390.97 us/op 0.95
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.1078 ms/op 5.2330 ms/op 0.78
bytes32 toHexString 606.00 ns/op 807.00 ns/op 0.75
bytes32 Buffer.toString(hex) 482.00 ns/op 489.00 ns/op 0.99
bytes32 Buffer.toString(hex) from Uint8Array 617.00 ns/op 595.00 ns/op 1.04
bytes32 Buffer.toString(hex) + 0x 459.00 ns/op 488.00 ns/op 0.94
Object access 1 prop 0.35800 ns/op 0.34000 ns/op 1.05
Map access 1 prop 0.33600 ns/op 0.32400 ns/op 1.04
Object get x1000 5.1890 ns/op 5.0250 ns/op 1.03
Map get x1000 6.2220 ns/op 5.7550 ns/op 1.08
Object set x1000 23.577 ns/op 27.231 ns/op 0.87
Map set x1000 19.134 ns/op 19.840 ns/op 0.96
Return object 10000 times 0.30980 ns/op 0.29480 ns/op 1.05
Throw Error 10000 times 2.8722 us/op 2.6575 us/op 1.08
toHex 114.53 ns/op 119.78 ns/op 0.96
Buffer.from 103.41 ns/op 98.237 ns/op 1.05
shared Buffer 75.901 ns/op 66.791 ns/op 1.14
fastMsgIdFn sha256 / 200 bytes 2.1100 us/op 2.0190 us/op 1.05
fastMsgIdFn h32 xxhash / 200 bytes 484.00 ns/op 426.00 ns/op 1.14
fastMsgIdFn h64 xxhash / 200 bytes 506.00 ns/op 454.00 ns/op 1.11
fastMsgIdFn sha256 / 1000 bytes 6.2290 us/op 5.8230 us/op 1.07
fastMsgIdFn h32 xxhash / 1000 bytes 610.00 ns/op 553.00 ns/op 1.10
fastMsgIdFn h64 xxhash / 1000 bytes 539.00 ns/op 507.00 ns/op 1.06
fastMsgIdFn sha256 / 10000 bytes 49.425 us/op 46.684 us/op 1.06
fastMsgIdFn h32 xxhash / 10000 bytes 2.0380 us/op 1.9090 us/op 1.07
fastMsgIdFn h64 xxhash / 10000 bytes 1.4070 us/op 1.3160 us/op 1.07
send data - 1000 256B messages 9.4404 ms/op 11.863 ms/op 0.80
send data - 1000 512B messages 13.566 ms/op 14.082 ms/op 0.96
send data - 1000 1024B messages 21.276 ms/op 21.913 ms/op 0.97
send data - 1000 1200B messages 22.867 ms/op 22.746 ms/op 1.01
send data - 1000 2048B messages 35.062 ms/op 30.022 ms/op 1.17
send data - 1000 4096B messages 35.731 ms/op 27.678 ms/op 1.29
send data - 1000 16384B messages 77.835 ms/op 63.945 ms/op 1.22
send data - 1000 65536B messages 312.99 ms/op 236.04 ms/op 1.33
enrSubnets - fastDeserialize 64 bits 1.1980 us/op 1.0830 us/op 1.11
enrSubnets - ssz BitVector 64 bits 598.00 ns/op 519.00 ns/op 1.15
enrSubnets - fastDeserialize 4 bits 390.00 ns/op 322.00 ns/op 1.21
enrSubnets - ssz BitVector 4 bits 634.00 ns/op 535.00 ns/op 1.19
prioritizePeers score -10:0 att 32-0.1 sync 2-0 125.96 us/op 120.75 us/op 1.04
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 153.33 us/op 141.69 us/op 1.08
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 283.47 us/op 219.78 us/op 1.29
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 458.47 us/op 425.15 us/op 1.08
prioritizePeers score 0:0 att 64-1 sync 4-1 618.31 us/op 527.13 us/op 1.17
array of 16000 items push then shift 1.4087 us/op 1.2816 us/op 1.10
LinkedList of 16000 items push then shift 6.7590 ns/op 7.4570 ns/op 0.91
array of 16000 items push then pop 107.18 ns/op 125.64 ns/op 0.85
LinkedList of 16000 items push then pop 6.5500 ns/op 6.2970 ns/op 1.04
array of 24000 items push then shift 1.9975 us/op 1.8973 us/op 1.05
LinkedList of 24000 items push then shift 6.5960 ns/op 6.9310 ns/op 0.95
array of 24000 items push then pop 132.90 ns/op 158.41 ns/op 0.84
LinkedList of 24000 items push then pop 6.5550 ns/op 6.2220 ns/op 1.05
intersect bitArray bitLen 8 6.0610 ns/op 5.2700 ns/op 1.15
intersect array and set length 8 52.104 ns/op 40.509 ns/op 1.29
intersect bitArray bitLen 128 27.667 ns/op 26.214 ns/op 1.06
intersect array and set length 128 717.79 ns/op 647.64 ns/op 1.11
bitArray.getTrueBitIndexes() bitLen 128 2.3910 us/op 2.2460 us/op 1.06
bitArray.getTrueBitIndexes() bitLen 248 3.4970 us/op 3.4510 us/op 1.01
bitArray.getTrueBitIndexes() bitLen 512 5.7940 us/op 8.6390 us/op 0.67
Buffer.concat 32 items 1.1370 us/op 1.0860 us/op 1.05
Uint8Array.set 32 items 1.7650 us/op 1.7170 us/op 1.03
Buffer.copy 2.0140 us/op 1.7040 us/op 1.18
Uint8Array.set - with subarray 2.2830 us/op 2.1400 us/op 1.07
Uint8Array.set - without subarray 1.6050 us/op 1.4640 us/op 1.10
getUint32 - dataview 459.00 ns/op 406.00 ns/op 1.13
getUint32 - manual 414.00 ns/op 343.00 ns/op 1.21
Set add up to 64 items then delete first 1.8531 us/op 1.7844 us/op 1.04
OrderedSet add up to 64 items then delete first 2.8890 us/op 2.7672 us/op 1.04
Set add up to 64 items then delete last 2.6302 us/op 2.0333 us/op 1.29
OrderedSet add up to 64 items then delete last 4.8691 us/op 3.0735 us/op 1.58
Set add up to 64 items then delete middle 2.9680 us/op 2.0367 us/op 1.46
OrderedSet add up to 64 items then delete middle 7.0046 us/op 4.5966 us/op 1.52
Set add up to 128 items then delete first 6.4360 us/op 4.0016 us/op 1.61
OrderedSet add up to 128 items then delete first 9.4564 us/op 6.3192 us/op 1.50
Set add up to 128 items then delete last 5.6549 us/op 3.8364 us/op 1.47
OrderedSet add up to 128 items then delete last 9.0498 us/op 5.8751 us/op 1.54
Set add up to 128 items then delete middle 5.5721 us/op 3.8081 us/op 1.46
OrderedSet add up to 128 items then delete middle 15.015 us/op 11.979 us/op 1.25
Set add up to 256 items then delete first 11.306 us/op 7.8387 us/op 1.44
OrderedSet add up to 256 items then delete first 20.216 us/op 12.585 us/op 1.61
Set add up to 256 items then delete last 8.4467 us/op 7.5312 us/op 1.12
OrderedSet add up to 256 items then delete last 13.055 us/op 11.797 us/op 1.11
Set add up to 256 items then delete middle 8.2385 us/op 7.4386 us/op 1.11
OrderedSet add up to 256 items then delete middle 36.431 us/op 35.240 us/op 1.03
transfer serialized Status (84 B) 1.5500 us/op 1.4970 us/op 1.04
copy serialized Status (84 B) 1.4520 us/op 1.3040 us/op 1.11
transfer serialized SignedVoluntaryExit (112 B) 1.7860 us/op 1.7480 us/op 1.02
copy serialized SignedVoluntaryExit (112 B) 1.5470 us/op 1.4170 us/op 1.09
transfer serialized ProposerSlashing (416 B) 2.3510 us/op 2.0370 us/op 1.15
copy serialized ProposerSlashing (416 B) 2.2790 us/op 2.0880 us/op 1.09
transfer serialized Attestation (485 B) 2.1160 us/op 2.1500 us/op 0.98
copy serialized Attestation (485 B) 2.0220 us/op 2.1380 us/op 0.95
transfer serialized AttesterSlashing (33232 B) 3.0830 us/op 2.7250 us/op 1.13
copy serialized AttesterSlashing (33232 B) 8.9860 us/op 6.0530 us/op 1.48
transfer serialized Small SignedBeaconBlock (128000 B) 2.8710 us/op 3.9780 us/op 0.72
copy serialized Small SignedBeaconBlock (128000 B) 20.109 us/op 16.287 us/op 1.23
transfer serialized Avg SignedBeaconBlock (200000 B) 4.2430 us/op 3.9630 us/op 1.07
copy serialized Avg SignedBeaconBlock (200000 B) 39.956 us/op 23.972 us/op 1.67
transfer serialized BlobsSidecar (524380 B) 4.4200 us/op 3.3600 us/op 1.32
copy serialized BlobsSidecar (524380 B) 295.16 us/op 113.65 us/op 2.60
transfer serialized Big SignedBeaconBlock (1000000 B) 4.1840 us/op 2.9700 us/op 1.41
copy serialized Big SignedBeaconBlock (1000000 B) 197.25 us/op 188.02 us/op 1.05
pass gossip attestations to forkchoice per slot 2.6772 ms/op 2.5462 ms/op 1.05
forkChoice updateHead vc 100000 bc 64 eq 0 411.34 us/op 366.17 us/op 1.12
forkChoice updateHead vc 600000 bc 64 eq 0 3.0159 ms/op 2.7097 ms/op 1.11
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3054 ms/op 3.6948 ms/op 1.17
forkChoice updateHead vc 600000 bc 320 eq 0 2.6352 ms/op 2.1906 ms/op 1.20
forkChoice updateHead vc 600000 bc 1200 eq 0 2.6157 ms/op 2.2462 ms/op 1.16
forkChoice updateHead vc 600000 bc 7200 eq 0 2.8574 ms/op 3.1150 ms/op 0.92
forkChoice updateHead vc 600000 bc 64 eq 1000 9.7876 ms/op 9.5812 ms/op 1.02
forkChoice updateHead vc 600000 bc 64 eq 10000 9.9943 ms/op 9.5213 ms/op 1.05
forkChoice updateHead vc 600000 bc 64 eq 300000 13.790 ms/op 11.784 ms/op 1.17
computeDeltas 500000 validators 300 proto nodes 3.3868 ms/op 3.2398 ms/op 1.05
computeDeltas 500000 validators 1200 proto nodes 3.4070 ms/op 3.2681 ms/op 1.04
computeDeltas 500000 validators 7200 proto nodes 3.4863 ms/op 3.2607 ms/op 1.07
computeDeltas 750000 validators 300 proto nodes 5.2238 ms/op 4.8665 ms/op 1.07
computeDeltas 750000 validators 1200 proto nodes 5.1211 ms/op 4.8411 ms/op 1.06
computeDeltas 750000 validators 7200 proto nodes 4.8935 ms/op 4.7775 ms/op 1.02
computeDeltas 1400000 validators 300 proto nodes 8.8207 ms/op 8.8315 ms/op 1.00
computeDeltas 1400000 validators 1200 proto nodes 8.9545 ms/op 9.1316 ms/op 0.98
computeDeltas 1400000 validators 7200 proto nodes 9.0053 ms/op 8.7758 ms/op 1.03
computeDeltas 2100000 validators 300 proto nodes 13.678 ms/op 13.563 ms/op 1.01
computeDeltas 2100000 validators 1200 proto nodes 13.653 ms/op 13.531 ms/op 1.01
computeDeltas 2100000 validators 7200 proto nodes 14.059 ms/op 13.478 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei normalcase 1.6058 ms/op 1.3997 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 2.1958 ms/op 2.1942 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 72.227 us/op 66.958 us/op 1.08
altair processAttestation - setStatus - 1/3 committees join 124.24 us/op 141.72 us/op 0.88
altair processAttestation - setStatus - 1/2 committees join 195.13 us/op 194.96 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 260.07 us/op 252.07 us/op 1.03
altair processAttestation - setStatus - 4/5 committees join 390.32 us/op 383.85 us/op 1.02
altair processAttestation - setStatus - 100% committees join 493.04 us/op 460.59 us/op 1.07
altair processBlock - 250000 vs - 7PWei normalcase 3.7461 ms/op 5.4814 ms/op 0.68
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.114 ms/op 23.561 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 34.239 ms/op 38.362 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase hashState 70.452 ms/op 85.734 ms/op 0.82
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9401 ms/op 2.0426 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei worstcase 24.306 ms/op 21.687 ms/op 1.12
altair processEth1Data - 250000 vs - 7PWei normalcase 249.84 us/op 235.92 us/op 1.06
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.6250 us/op 3.5870 us/op 1.29
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 27.711 us/op 17.366 us/op 1.60
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.8430 us/op 10.844 us/op 0.91
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 4.6950 us/op 5.0780 us/op 0.92
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 107.08 us/op 118.75 us/op 0.90
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 855.72 us/op 1.3286 ms/op 0.64
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.9010 ms/op 1.2265 ms/op 1.55
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1451 ms/op 1.0515 ms/op 1.09
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.8600 ms/op 2.9483 ms/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.1987 ms/op 1.2049 ms/op 0.99
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 2.9731 ms/op 2.7890 ms/op 1.07
Tree 40 250000 create 171.83 ms/op 179.09 ms/op 0.96
Tree 40 250000 get(125000) 109.97 ns/op 101.75 ns/op 1.08
Tree 40 250000 set(125000) 528.79 ns/op 517.05 ns/op 1.02
Tree 40 250000 toArray() 9.7625 ms/op 12.218 ms/op 0.80
Tree 40 250000 iterate all - toArray() + loop 9.7057 ms/op 12.347 ms/op 0.79
Tree 40 250000 iterate all - get(i) 40.501 ms/op 39.782 ms/op 1.02
Array 250000 create 2.4315 ms/op 2.4037 ms/op 1.01
Array 250000 clone - spread 1.3395 ms/op 1.3163 ms/op 1.02
Array 250000 get(125000) 0.58600 ns/op 0.56300 ns/op 1.04
Array 250000 set(125000) 0.60000 ns/op 0.56400 ns/op 1.06
Array 250000 iterate all - loop 77.629 us/op 72.363 us/op 1.07
phase0 afterProcessEpoch - 250000 vs - 7PWei 41.860 ms/op 39.430 ms/op 1.06
Array.fill - length 1000000 2.6506 ms/op 2.4982 ms/op 1.06
Array push - length 1000000 14.011 ms/op 14.061 ms/op 1.00
Array.get 0.26761 ns/op 0.25331 ns/op 1.06
Uint8Array.get 0.35442 ns/op 0.32592 ns/op 1.09
phase0 beforeProcessEpoch - 250000 vs - 7PWei 17.666 ms/op 15.385 ms/op 1.15
altair processEpoch - mainnet_e81889 276.21 ms/op 278.69 ms/op 0.99
mainnet_e81889 - altair beforeProcessEpoch 17.659 ms/op 16.290 ms/op 1.08
mainnet_e81889 - altair processJustificationAndFinalization 13.455 us/op 10.909 us/op 1.23
mainnet_e81889 - altair processInactivityUpdates 4.6710 ms/op 4.1307 ms/op 1.13
mainnet_e81889 - altair processRewardsAndPenalties 43.379 ms/op 55.977 ms/op 0.77
mainnet_e81889 - altair processRegistryUpdates 1.8060 us/op 1.3450 us/op 1.34
mainnet_e81889 - altair processSlashings 828.00 ns/op 823.00 ns/op 1.01
mainnet_e81889 - altair processEth1DataReset 794.00 ns/op 748.00 ns/op 1.06
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.0469 ms/op 1.6386 ms/op 0.64
mainnet_e81889 - altair processSlashingsReset 3.4420 us/op 2.1560 us/op 1.60
mainnet_e81889 - altair processRandaoMixesReset 4.7480 us/op 2.7200 us/op 1.75
mainnet_e81889 - altair processHistoricalRootsUpdate 726.00 ns/op 692.00 ns/op 1.05
mainnet_e81889 - altair processParticipationFlagUpdates 2.0650 us/op 1.8100 us/op 1.14
mainnet_e81889 - altair processSyncCommitteeUpdates 1.6700 us/op 703.00 ns/op 2.38
mainnet_e81889 - altair afterProcessEpoch 42.525 ms/op 41.143 ms/op 1.03
capella processEpoch - mainnet_e217614 1.0192 s/op 965.12 ms/op 1.06
mainnet_e217614 - capella beforeProcessEpoch 72.867 ms/op 68.213 ms/op 1.07
mainnet_e217614 - capella processJustificationAndFinalization 13.890 us/op 11.636 us/op 1.19
mainnet_e217614 - capella processInactivityUpdates 14.699 ms/op 12.576 ms/op 1.17
mainnet_e217614 - capella processRewardsAndPenalties 225.07 ms/op 220.74 ms/op 1.02
mainnet_e217614 - capella processRegistryUpdates 13.683 us/op 13.795 us/op 0.99
mainnet_e217614 - capella processSlashings 887.00 ns/op 740.00 ns/op 1.20
mainnet_e217614 - capella processEth1DataReset 922.00 ns/op 714.00 ns/op 1.29
mainnet_e217614 - capella processEffectiveBalanceUpdates 13.784 ms/op 13.243 ms/op 1.04
mainnet_e217614 - capella processSlashingsReset 4.0640 us/op 2.4310 us/op 1.67
mainnet_e217614 - capella processRandaoMixesReset 3.7440 us/op 10.228 us/op 0.37
mainnet_e217614 - capella processHistoricalRootsUpdate 1.9600 us/op 1.0120 us/op 1.94
mainnet_e217614 - capella processParticipationFlagUpdates 3.3870 us/op 2.2060 us/op 1.54
mainnet_e217614 - capella afterProcessEpoch 109.41 ms/op 106.38 ms/op 1.03
phase0 processEpoch - mainnet_e58758 313.26 ms/op 345.60 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 102.36 ms/op 87.280 ms/op 1.17
mainnet_e58758 - phase0 processJustificationAndFinalization 21.718 us/op 13.319 us/op 1.63
mainnet_e58758 - phase0 processRewardsAndPenalties 42.828 ms/op 26.270 ms/op 1.63
mainnet_e58758 - phase0 processRegistryUpdates 14.665 us/op 5.5300 us/op 2.65
mainnet_e58758 - phase0 processSlashings 1.4690 us/op 770.00 ns/op 1.91
mainnet_e58758 - phase0 processEth1DataReset 1.1290 us/op 719.00 ns/op 1.57
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3173 ms/op 1.3597 ms/op 0.97
mainnet_e58758 - phase0 processSlashingsReset 6.6280 us/op 2.3320 us/op 2.84
mainnet_e58758 - phase0 processRandaoMixesReset 8.0610 us/op 3.2170 us/op 2.51
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.0840 us/op 776.00 ns/op 1.40
mainnet_e58758 - phase0 processParticipationRecordUpdates 9.0470 us/op 3.1250 us/op 2.90
mainnet_e58758 - phase0 afterProcessEpoch 39.198 ms/op 34.321 ms/op 1.14
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.7507 ms/op 1.6808 ms/op 1.64
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2589 ms/op 1.6333 ms/op 1.38
altair processInactivityUpdates - 250000 normalcase 22.354 ms/op 17.134 ms/op 1.30
altair processInactivityUpdates - 250000 worstcase 22.505 ms/op 16.320 ms/op 1.38
phase0 processRegistryUpdates - 250000 normalcase 14.671 us/op 5.6000 us/op 2.62
phase0 processRegistryUpdates - 250000 badcase_full_deposits 371.14 us/op 312.49 us/op 1.19
phase0 processRegistryUpdates - 250000 worstcase 0.5 188.75 ms/op 110.94 ms/op 1.70
altair processRewardsAndPenalties - 250000 normalcase 66.613 ms/op 41.598 ms/op 1.60
altair processRewardsAndPenalties - 250000 worstcase 52.424 ms/op 33.371 ms/op 1.57
phase0 getAttestationDeltas - 250000 normalcase 11.403 ms/op 6.4047 ms/op 1.78
phase0 getAttestationDeltas - 250000 worstcase 9.8138 ms/op 5.8999 ms/op 1.66
phase0 processSlashings - 250000 worstcase 100.59 us/op 81.890 us/op 1.23
altair processSyncCommitteeUpdates - 250000 148.66 ms/op 94.417 ms/op 1.57
BeaconState.hashTreeRoot - No change 553.00 ns/op 433.00 ns/op 1.28
BeaconState.hashTreeRoot - 1 full validator 151.81 us/op 111.85 us/op 1.36
BeaconState.hashTreeRoot - 32 full validator 1.1246 ms/op 1.1559 ms/op 0.97
BeaconState.hashTreeRoot - 512 full validator 12.044 ms/op 9.0742 ms/op 1.33
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 132.51 us/op 119.37 us/op 1.11
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.2552 ms/op 1.8957 ms/op 1.19
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 31.763 ms/op 22.872 ms/op 1.39
BeaconState.hashTreeRoot - 1 balances 133.34 us/op 107.81 us/op 1.24
BeaconState.hashTreeRoot - 32 balances 1.2172 ms/op 1.0188 ms/op 1.19
BeaconState.hashTreeRoot - 512 balances 9.6042 ms/op 6.6326 ms/op 1.45
BeaconState.hashTreeRoot - 250000 balances 202.11 ms/op 149.20 ms/op 1.35
aggregationBits - 2048 els - zipIndexesInBitList 34.093 us/op 44.387 us/op 0.77
byteArrayEquals 32 50.193 ns/op 48.995 ns/op 1.02
Buffer.compare 32 15.953 ns/op 15.863 ns/op 1.01
byteArrayEquals 1024 1.3269 us/op 1.2943 us/op 1.03
Buffer.compare 1024 23.550 ns/op 24.342 ns/op 0.97
byteArrayEquals 16384 21.165 us/op 20.492 us/op 1.03
Buffer.compare 16384 230.63 ns/op 211.94 ns/op 1.09
byteArrayEquals 123687377 152.58 ms/op 151.11 ms/op 1.01
Buffer.compare 123687377 4.0438 ms/op 5.4824 ms/op 0.74
byteArrayEquals 32 - diff last byte 48.015 ns/op 44.901 ns/op 1.07
Buffer.compare 32 - diff last byte 15.936 ns/op 15.056 ns/op 1.06
byteArrayEquals 1024 - diff last byte 1.2810 us/op 1.2103 us/op 1.06
Buffer.compare 1024 - diff last byte 24.881 ns/op 24.005 ns/op 1.04
byteArrayEquals 16384 - diff last byte 20.400 us/op 19.086 us/op 1.07
Buffer.compare 16384 - diff last byte 215.08 ns/op 193.78 ns/op 1.11
byteArrayEquals 123687377 - diff last byte 155.25 ms/op 149.48 ms/op 1.04
Buffer.compare 123687377 - diff last byte 5.3517 ms/op 3.7173 ms/op 1.44
byteArrayEquals 32 - random bytes 4.9320 ns/op 4.8390 ns/op 1.02
Buffer.compare 32 - random bytes 15.448 ns/op 16.581 ns/op 0.93
byteArrayEquals 1024 - random bytes 4.9800 ns/op 4.8860 ns/op 1.02
Buffer.compare 1024 - random bytes 16.169 ns/op 15.835 ns/op 1.02
byteArrayEquals 16384 - random bytes 4.9120 ns/op 4.8280 ns/op 1.02
Buffer.compare 16384 - random bytes 15.591 ns/op 16.472 ns/op 0.95
byteArrayEquals 123687377 - random bytes 8.0500 ns/op 7.7300 ns/op 1.04
Buffer.compare 123687377 - random bytes 19.400 ns/op 19.230 ns/op 1.01
regular array get 100000 times 32.605 us/op 30.671 us/op 1.06
wrappedArray get 100000 times 31.613 us/op 30.631 us/op 1.03
arrayWithProxy get 100000 times 10.362 ms/op 9.0841 ms/op 1.14
ssz.Root.equals 45.536 ns/op 40.830 ns/op 1.12
byteArrayEquals 45.223 ns/op 39.044 ns/op 1.16
Buffer.compare 9.7770 ns/op 8.2280 ns/op 1.19
processSlot - 1 slots 15.835 us/op 15.216 us/op 1.04
processSlot - 32 slots 2.3173 ms/op 2.7862 ms/op 0.83
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 35.885 ms/op 41.041 ms/op 0.87
getCommitteeAssignments - req 1 vs - 250000 vc 1.8736 ms/op 1.7195 ms/op 1.09
getCommitteeAssignments - req 100 vs - 250000 vc 3.6624 ms/op 3.4603 ms/op 1.06
getCommitteeAssignments - req 1000 vs - 250000 vc 3.8837 ms/op 3.6948 ms/op 1.05
findModifiedValidators - 10000 modified validators 251.78 ms/op 223.07 ms/op 1.13
findModifiedValidators - 1000 modified validators 178.56 ms/op 139.22 ms/op 1.28
findModifiedValidators - 100 modified validators 176.84 ms/op 139.41 ms/op 1.27
findModifiedValidators - 10 modified validators 148.21 ms/op 142.44 ms/op 1.04
findModifiedValidators - 1 modified validators 139.72 ms/op 120.70 ms/op 1.16
findModifiedValidators - no difference 149.86 ms/op 142.71 ms/op 1.05
compare ViewDUs 2.9450 s/op 2.9467 s/op 1.00
compare each validator Uint8Array 934.32 ms/op 1.6494 s/op 0.57
compare ViewDU to Uint8Array 816.59 ms/op 730.18 ms/op 1.12
migrate state 1000000 validators, 24 modified, 0 new 620.44 ms/op 713.73 ms/op 0.87
migrate state 1000000 validators, 1700 modified, 1000 new 918.50 ms/op 963.65 ms/op 0.95
migrate state 1000000 validators, 3400 modified, 2000 new 1.0616 s/op 1.1693 s/op 0.91
migrate state 1500000 validators, 24 modified, 0 new 696.69 ms/op 709.21 ms/op 0.98
migrate state 1500000 validators, 1700 modified, 1000 new 916.90 ms/op 893.32 ms/op 1.03
migrate state 1500000 validators, 3400 modified, 2000 new 1.1001 s/op 1.0812 s/op 1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.0200 ns/op 6.1700 ns/op 1.14
state getBlockRootAtSlot - 250000 vs - 7PWei 683.44 ns/op 611.33 ns/op 1.12
computeProposers - vc 250000 7.0731 ms/op 5.2258 ms/op 1.35
computeEpochShuffling - vc 250000 36.150 ms/op 33.809 ms/op 1.07
getNextSyncCommittee - vc 250000 125.74 ms/op 99.630 ms/op 1.26
computeSigningRoot for AttestationData 28.991 us/op 19.417 us/op 1.49
hash AttestationData serialized data then Buffer.toString(base64) 1.2472 us/op 1.1259 us/op 1.11
toHexString serialized data 768.97 ns/op 730.51 ns/op 1.05
Buffer.toString(base64) 139.48 ns/op 123.94 ns/op 1.13
nodejs block root to RootHex using toHex 121.12 ns/op 111.37 ns/op 1.09
nodejs block root to RootHex using toRootHex 77.266 ns/op 67.610 ns/op 1.14
browser block root to RootHex using the deprecated toHexString 207.20 ns/op 189.42 ns/op 1.09
browser block root to RootHex using toHex 165.48 ns/op 154.91 ns/op 1.07
browser block root to RootHex using toRootHex 150.17 ns/op 140.99 ns/op 1.07

by benchmarkbot/action

import {RootHex} from "@lodestar/types";

export enum ArchiveMode {
Full = "full",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think of a better name for current archiver strategy we use, please suggest.

Copy link

codecov bot commented Oct 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49.22%. Comparing base (245d63e) to head (25293b4).
Report is 2 commits behind head on unstable.

Additional details and impacted files
@@            Coverage Diff            @@
##           unstable    #7170   +/-   ##
=========================================
  Coverage     49.21%   49.22%           
=========================================
  Files           598      598           
  Lines         39726    39735    +9     
  Branches       2093     2087    -6     
=========================================
+ Hits          19551    19558    +7     
- Misses        20135    20137    +2     
  Partials         40       40           

packages/cli/src/options/beaconNodeOptions/chain.ts Outdated Show resolved Hide resolved
packages/cli/src/options/beaconNodeOptions/chain.ts Outdated Show resolved Hide resolved
import {RootHex} from "@lodestar/types";

export enum ArchiveMode {
Full = "full",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More of a general question, once we have differential archive mode implemented, will there be any advantage to use our current (full) archiving strategy?

If not, we should probably not make it overly generic / switchable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea is to get the differential strategy released as experimental first. As I am a bit skeptical about it's memory implication. So during that time user can choose which strategy to use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants