Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migrate to blob side cars validation from blobs side car #5687

Merged
merged 4 commits into from
Jun 27, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jun 23, 2023

further integration of

This PR majorly targets

  • blob side cars validation
  • remove obsolete beacon_block_and_blobs_sidecar gossip topic

Next and one of the final steps:

  • migrate blockinput from blobs side car to blobs side car

@g11tech g11tech requested a review from a team as a code owner June 23, 2023 13:31
@github-actions
Copy link
Contributor

github-actions bot commented Jun 23, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 840e21e Previous: a208afb Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 865.82 us/op 816.44 us/op 1.06
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.628 us/op 46.294 us/op 1.01
BLS verify - blst-native 1.2376 ms/op 1.2151 ms/op 1.02
BLS verifyMultipleSignatures 3 - blst-native 2.5177 ms/op 2.4652 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.4284 ms/op 5.2976 ms/op 1.02
BLS verifyMultipleSignatures 32 - blst-native 19.750 ms/op 19.152 ms/op 1.03
BLS aggregatePubkeys 32 - blst-native 26.551 us/op 25.886 us/op 1.03
BLS aggregatePubkeys 128 - blst-native 103.02 us/op 100.04 us/op 1.03
getAttestationsForBlock 58.876 ms/op 56.262 ms/op 1.05
isKnown best case - 1 super set check 263.00 ns/op 263.00 ns/op 1.00
isKnown normal case - 2 super set checks 256.00 ns/op 261.00 ns/op 0.98
isKnown worse case - 16 super set checks 251.00 ns/op 258.00 ns/op 0.97
CheckpointStateCache - add get delete 5.8890 us/op 5.3160 us/op 1.11
validate gossip signedAggregateAndProof - struct 2.8894 ms/op 2.8217 ms/op 1.02
validate gossip attestation - struct 1.3740 ms/op 1.3492 ms/op 1.02
pickEth1Vote - no votes 1.3790 ms/op 1.3552 ms/op 1.02
pickEth1Vote - max votes 9.8809 ms/op 8.6700 ms/op 1.14
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.0926 ms/op 8.5185 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 15.826 ms/op 15.017 ms/op 1.05
pickEth1Vote - Eth1Data fastSerialize value x2048 816.12 us/op 707.88 us/op 1.15
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.7914 ms/op 7.1009 ms/op 0.82
bytes32 toHexString 699.00 ns/op 554.00 ns/op 1.26
bytes32 Buffer.toString(hex) 416.00 ns/op 418.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 632.00 ns/op 635.00 ns/op 1.00
bytes32 Buffer.toString(hex) + 0x 433.00 ns/op 394.00 ns/op 1.10
Object access 1 prop 0.18400 ns/op 0.19000 ns/op 0.97
Map access 1 prop 0.16500 ns/op 0.16500 ns/op 1.00
Object get x1000 7.5550 ns/op 7.1970 ns/op 1.05
Map get x1000 0.66700 ns/op 0.65800 ns/op 1.01
Object set x1000 61.045 ns/op 63.017 ns/op 0.97
Map set x1000 51.515 ns/op 52.610 ns/op 0.98
Return object 10000 times 0.24730 ns/op 0.26290 ns/op 0.94
Throw Error 10000 times 4.2614 us/op 4.4858 us/op 0.95
fastMsgIdFn sha256 / 200 bytes 3.6010 us/op 3.6600 us/op 0.98
fastMsgIdFn h32 xxhash / 200 bytes 319.00 ns/op 333.00 ns/op 0.96
fastMsgIdFn h64 xxhash / 200 bytes 442.00 ns/op 495.00 ns/op 0.89
fastMsgIdFn sha256 / 1000 bytes 11.854 us/op 12.515 us/op 0.95
fastMsgIdFn h32 xxhash / 1000 bytes 453.00 ns/op 457.00 ns/op 0.99
fastMsgIdFn h64 xxhash / 1000 bytes 543.00 ns/op 547.00 ns/op 0.99
fastMsgIdFn sha256 / 10000 bytes 107.25 us/op 110.05 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 2.0630 us/op 2.2260 us/op 0.93
fastMsgIdFn h64 xxhash / 10000 bytes 1.5040 us/op 1.5270 us/op 0.98
enrSubnets - fastDeserialize 64 bits 1.6020 us/op 1.4530 us/op 1.10
enrSubnets - ssz BitVector 64 bits 591.00 ns/op 606.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 211.00 ns/op 205.00 ns/op 1.03
enrSubnets - ssz BitVector 4 bits 630.00 ns/op 640.00 ns/op 0.98
prioritizePeers score -10:0 att 32-0.1 sync 2-0 125.15 us/op 118.85 us/op 1.05
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 167.58 us/op 162.14 us/op 1.03
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 191.36 us/op 193.63 us/op 0.99
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 355.86 us/op 359.73 us/op 0.99
prioritizePeers score 0:0 att 64-1 sync 4-1 445.83 us/op 467.24 us/op 0.95
array of 16000 items push then shift 1.7324 us/op 1.8918 us/op 0.92
LinkedList of 16000 items push then shift 9.4660 ns/op 10.074 ns/op 0.94
array of 16000 items push then pop 123.35 ns/op 123.72 ns/op 1.00
LinkedList of 16000 items push then pop 9.5010 ns/op 9.2460 ns/op 1.03
array of 24000 items push then shift 2.4368 us/op 2.5396 us/op 0.96
LinkedList of 24000 items push then shift 9.4770 ns/op 9.2170 ns/op 1.03
array of 24000 items push then pop 84.417 ns/op 78.854 ns/op 1.07
LinkedList of 24000 items push then pop 9.0080 ns/op 9.2680 ns/op 0.97
intersect bitArray bitLen 8 14.360 ns/op 14.199 ns/op 1.01
intersect array and set length 8 101.40 ns/op 85.083 ns/op 1.19
intersect bitArray bitLen 128 47.574 ns/op 47.892 ns/op 0.99
intersect array and set length 128 1.2312 us/op 1.2012 us/op 1.02
Buffer.concat 32 items 3.1950 us/op 3.1160 us/op 1.03
Uint8Array.set 32 items 2.6270 us/op 2.6370 us/op 1.00
transfer serialized Status (84 B) 2.1900 us/op 2.2120 us/op 0.99
copy serialized Status (84 B) 1.8580 us/op 1.8260 us/op 1.02
transfer serialized SignedVoluntaryExit (112 B) 2.1240 us/op 2.3180 us/op 0.92
copy serialized SignedVoluntaryExit (112 B) 1.6710 us/op 2.0320 us/op 0.82
transfer serialized ProposerSlashing (416 B) 2.7120 us/op 2.8210 us/op 0.96
copy serialized ProposerSlashing (416 B) 2.5200 us/op 3.1570 us/op 0.80
transfer serialized Attestation (485 B) 2.6000 us/op 2.9220 us/op 0.89
copy serialized Attestation (485 B) 2.9520 us/op 2.5130 us/op 1.17
transfer serialized AttesterSlashing (33232 B) 3.0770 us/op 2.9550 us/op 1.04
copy serialized AttesterSlashing (33232 B) 6.0060 us/op 6.7140 us/op 0.89
transfer serialized Small SignedBeaconBlock (128000 B) 3.3900 us/op 3.5200 us/op 0.96
copy serialized Small SignedBeaconBlock (128000 B) 14.816 us/op 14.702 us/op 1.01
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6300 us/op 3.6620 us/op 0.99
copy serialized Avg SignedBeaconBlock (200000 B) 21.115 us/op 22.309 us/op 0.95
transfer serialized BlobsSidecar (524380 B) 3.5750 us/op 3.6730 us/op 0.97
copy serialized BlobsSidecar (524380 B) 70.544 us/op 170.49 us/op 0.41
transfer serialized Big SignedBeaconBlock (1000000 B) 3.6500 us/op 4.0630 us/op 0.90
copy serialized Big SignedBeaconBlock (1000000 B) 266.67 us/op 289.00 us/op 0.92
pass gossip attestations to forkchoice per slot 2.5804 ms/op 2.8638 ms/op 0.90
forkChoice updateHead vc 100000 bc 64 eq 0 1.9688 ms/op 2.1249 ms/op 0.93
forkChoice updateHead vc 600000 bc 64 eq 0 10.809 ms/op 12.022 ms/op 0.90
forkChoice updateHead vc 1000000 bc 64 eq 0 21.225 ms/op 23.707 ms/op 0.90
forkChoice updateHead vc 600000 bc 320 eq 0 15.781 ms/op 17.998 ms/op 0.88
forkChoice updateHead vc 600000 bc 1200 eq 0 76.669 ms/op 88.577 ms/op 0.87
forkChoice updateHead vc 600000 bc 64 eq 1000 19.676 ms/op 21.368 ms/op 0.92
forkChoice updateHead vc 600000 bc 64 eq 10000 22.339 ms/op 23.551 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 300000 29.849 ms/op 32.167 ms/op 0.93
computeDeltas 3.7936 ms/op 3.6116 ms/op 1.05
computeProposerBoostScoreFromBalances 1.8658 ms/op 1.9292 ms/op 0.97
altair processAttestation - 250000 vs - 7PWei normalcase 2.3091 ms/op 3.1373 ms/op 0.74
altair processAttestation - 250000 vs - 7PWei worstcase 3.6355 ms/op 4.7994 ms/op 0.76
altair processAttestation - setStatus - 1/6 committees join 148.02 us/op 157.35 us/op 0.94
altair processAttestation - setStatus - 1/3 committees join 286.69 us/op 302.95 us/op 0.95
altair processAttestation - setStatus - 1/2 committees join 385.05 us/op 430.62 us/op 0.89
altair processAttestation - setStatus - 2/3 committees join 488.41 us/op 502.34 us/op 0.97
altair processAttestation - setStatus - 4/5 committees join 673.01 us/op 809.01 us/op 0.83
altair processAttestation - setStatus - 100% committees join 782.22 us/op 917.22 us/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase 16.920 ms/op 19.850 ms/op 0.85
altair processBlock - 250000 vs - 7PWei normalcase hashState 30.021 ms/op 27.696 ms/op 1.08
altair processBlock - 250000 vs - 7PWei worstcase 50.290 ms/op 57.765 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 68.319 ms/op 79.888 ms/op 0.86
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1219 ms/op 2.2065 ms/op 0.96
phase0 processBlock - 250000 vs - 7PWei worstcase 29.486 ms/op 34.406 ms/op 0.86
altair processEth1Data - 250000 vs - 7PWei normalcase 476.73 us/op 497.79 us/op 0.96
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 10.499 us/op 7.9140 us/op 1.33
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 28.013 us/op 24.055 us/op 1.16
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 11.100 us/op 13.172 us/op 0.84
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.9620 us/op 7.1700 us/op 1.11
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 101.70 us/op 100.08 us/op 1.02
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 649.63 us/op 791.27 us/op 0.82
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 924.31 us/op 938.97 us/op 0.98
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 912.05 us/op 934.13 us/op 0.98
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.3700 ms/op 3.1501 ms/op 0.75
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.5034 ms/op 1.8368 ms/op 0.82
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.0538 ms/op 5.0921 ms/op 0.80
Tree 40 250000 create 336.74 ms/op 432.16 ms/op 0.78
Tree 40 250000 get(125000) 192.04 ns/op 217.79 ns/op 0.88
Tree 40 250000 set(125000) 1.0593 us/op 1.1860 us/op 0.89
Tree 40 250000 toArray() 23.458 ms/op 24.402 ms/op 0.96
Tree 40 250000 iterate all - toArray() + loop 23.221 ms/op 24.516 ms/op 0.95
Tree 40 250000 iterate all - get(i) 79.059 ms/op 82.194 ms/op 0.96
MutableVector 250000 create 11.130 ms/op 13.592 ms/op 0.82
MutableVector 250000 get(125000) 6.4390 ns/op 7.7770 ns/op 0.83
MutableVector 250000 set(125000) 300.26 ns/op 308.06 ns/op 0.97
MutableVector 250000 toArray() 3.4405 ms/op 3.7496 ms/op 0.92
MutableVector 250000 iterate all - toArray() + loop 3.1460 ms/op 4.1234 ms/op 0.76
MutableVector 250000 iterate all - get(i) 1.5356 ms/op 1.7789 ms/op 0.86
Array 250000 create 2.6684 ms/op 3.7051 ms/op 0.72
Array 250000 clone - spread 1.1339 ms/op 1.5088 ms/op 0.75
Array 250000 get(125000) 0.54600 ns/op 0.93600 ns/op 0.58
Array 250000 set(125000) 0.62500 ns/op 0.89300 ns/op 0.70
Array 250000 iterate all - loop 82.533 us/op 116.11 us/op 0.71
effectiveBalanceIncrements clone Uint8Array 300000 25.279 us/op 49.209 us/op 0.51
effectiveBalanceIncrements clone MutableVector 300000 330.00 ns/op 414.00 ns/op 0.80
effectiveBalanceIncrements rw all Uint8Array 300000 168.32 us/op 184.02 us/op 0.91
effectiveBalanceIncrements rw all MutableVector 300000 78.709 ms/op 160.78 ms/op 0.49
phase0 afterProcessEpoch - 250000 vs - 7PWei 115.38 ms/op 138.73 ms/op 0.83
phase0 beforeProcessEpoch - 250000 vs - 7PWei 42.012 ms/op 54.400 ms/op 0.77
altair processEpoch - mainnet_e81889 307.61 ms/op 418.42 ms/op 0.74
mainnet_e81889 - altair beforeProcessEpoch 52.294 ms/op 100.92 ms/op 0.52
mainnet_e81889 - altair processJustificationAndFinalization 18.985 us/op 39.354 us/op 0.48
mainnet_e81889 - altair processInactivityUpdates 5.6051 ms/op 8.1918 ms/op 0.68
mainnet_e81889 - altair processRewardsAndPenalties 68.360 ms/op 87.563 ms/op 0.78
mainnet_e81889 - altair processRegistryUpdates 2.6850 us/op 5.9890 us/op 0.45
mainnet_e81889 - altair processSlashings 722.00 ns/op 1.4540 us/op 0.50
mainnet_e81889 - altair processEth1DataReset 883.00 ns/op 1.4880 us/op 0.59
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2622 ms/op 4.6905 ms/op 0.27
mainnet_e81889 - altair processSlashingsReset 4.9020 us/op 14.240 us/op 0.34
mainnet_e81889 - altair processRandaoMixesReset 4.4020 us/op 6.6500 us/op 0.66
mainnet_e81889 - altair processHistoricalRootsUpdate 1.8410 us/op 2.4640 us/op 0.75
mainnet_e81889 - altair processParticipationFlagUpdates 6.1260 us/op 8.3850 us/op 0.73
mainnet_e81889 - altair processSyncCommitteeUpdates 1.2710 us/op 1.5200 us/op 0.84
mainnet_e81889 - altair afterProcessEpoch 130.73 ms/op 164.05 ms/op 0.80
phase0 processEpoch - mainnet_e58758 429.85 ms/op 503.21 ms/op 0.85
mainnet_e58758 - phase0 beforeProcessEpoch 175.02 ms/op 211.00 ms/op 0.83
mainnet_e58758 - phase0 processJustificationAndFinalization 24.169 us/op 25.647 us/op 0.94
mainnet_e58758 - phase0 processRewardsAndPenalties 70.068 ms/op 79.361 ms/op 0.88
mainnet_e58758 - phase0 processRegistryUpdates 11.258 us/op 17.987 us/op 0.63
mainnet_e58758 - phase0 processSlashings 1.5550 us/op 1.4440 us/op 1.08
mainnet_e58758 - phase0 processEth1DataReset 705.00 ns/op 847.00 ns/op 0.83
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0621 ms/op 1.4087 ms/op 0.75
mainnet_e58758 - phase0 processSlashingsReset 5.7150 us/op 7.7880 us/op 0.73
mainnet_e58758 - phase0 processRandaoMixesReset 5.8240 us/op 10.193 us/op 0.57
mainnet_e58758 - phase0 processHistoricalRootsUpdate 901.00 ns/op 1.8500 us/op 0.49
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.5750 us/op 7.8540 us/op 0.71
mainnet_e58758 - phase0 afterProcessEpoch 104.49 ms/op 122.30 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2962 ms/op 1.5551 ms/op 0.83
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6301 ms/op 1.6595 ms/op 0.98
altair processInactivityUpdates - 250000 normalcase 26.649 ms/op 25.836 ms/op 1.03
altair processInactivityUpdates - 250000 worstcase 29.619 ms/op 27.493 ms/op 1.08
phase0 processRegistryUpdates - 250000 normalcase 8.6160 us/op 8.3560 us/op 1.03
phase0 processRegistryUpdates - 250000 badcase_full_deposits 275.87 us/op 289.55 us/op 0.95
phase0 processRegistryUpdates - 250000 worstcase 0.5 149.51 ms/op 141.76 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 79.965 ms/op 71.832 ms/op 1.11
altair processRewardsAndPenalties - 250000 worstcase 71.634 ms/op 78.510 ms/op 0.91
phase0 getAttestationDeltas - 250000 normalcase 10.697 ms/op 8.3068 ms/op 1.29
phase0 getAttestationDeltas - 250000 worstcase 9.8627 ms/op 7.2078 ms/op 1.37
phase0 processSlashings - 250000 worstcase 4.4852 ms/op 4.1729 ms/op 1.07
altair processSyncCommitteeUpdates - 250000 211.43 ms/op 198.47 ms/op 1.07
BeaconState.hashTreeRoot - No change 319.00 ns/op 376.00 ns/op 0.85
BeaconState.hashTreeRoot - 1 full validator 58.126 us/op 55.154 us/op 1.05
BeaconState.hashTreeRoot - 32 full validator 667.51 us/op 611.82 us/op 1.09
BeaconState.hashTreeRoot - 512 full validator 7.5048 ms/op 5.7844 ms/op 1.30
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 86.787 us/op 79.440 us/op 1.09
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.0042 ms/op 969.03 us/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.250 ms/op 13.342 ms/op 1.14
BeaconState.hashTreeRoot - 1 balances 76.375 us/op 56.077 us/op 1.36
BeaconState.hashTreeRoot - 32 balances 531.80 us/op 545.17 us/op 0.98
BeaconState.hashTreeRoot - 512 balances 5.3799 ms/op 4.8891 ms/op 1.10
BeaconState.hashTreeRoot - 250000 balances 82.131 ms/op 84.624 ms/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 27.064 us/op 18.165 us/op 1.49
regular array get 100000 times 36.472 us/op 48.003 us/op 0.76
wrappedArray get 100000 times 38.553 us/op 36.208 us/op 1.06
arrayWithProxy get 100000 times 17.417 ms/op 18.806 ms/op 0.93
ssz.Root.equals 646.00 ns/op 648.00 ns/op 1.00
byteArrayEquals 636.00 ns/op 635.00 ns/op 1.00
shuffle list - 16384 els 7.7830 ms/op 7.1492 ms/op 1.09
shuffle list - 250000 els 117.25 ms/op 104.03 ms/op 1.13
processSlot - 1 slots 10.692 us/op 9.4190 us/op 1.14
processSlot - 32 slots 1.5098 ms/op 1.4028 ms/op 1.08
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 38.779 ms/op 37.577 ms/op 1.03
getCommitteeAssignments - req 1 vs - 250000 vc 3.0299 ms/op 3.0118 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 4.2218 ms/op 4.2533 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5543 ms/op 4.5951 ms/op 0.99
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.9000 ns/op 4.3000 ns/op 1.14
state getBlockRootAtSlot - 250000 vs - 7PWei 619.12 ns/op 924.74 ns/op 0.67
computeProposers - vc 250000 10.846 ms/op 10.781 ms/op 1.01
computeEpochShuffling - vc 250000 103.75 ms/op 104.74 ms/op 0.99
getNextSyncCommittee - vc 250000 191.84 ms/op 180.65 ms/op 1.06
computeSigningRoot for AttestationData 14.677 us/op 14.154 us/op 1.04
hash AttestationData serialized data then Buffer.toString(base64) 2.4865 us/op 2.4775 us/op 1.00
toHexString serialized data 1.1159 us/op 1.0962 us/op 1.02
Buffer.toString(base64) 334.09 ns/op 328.42 ns/op 1.02

by benchmarkbot/action

@g11tech g11tech force-pushed the g11tech/sidecarvalidations branch from 5539b4b to 055efac Compare June 27, 2023 07:16
@wemeetagain wemeetagain merged commit d9fa355 into unstable Jun 27, 2023
@wemeetagain wemeetagain deleted the g11tech/sidecarvalidations branch June 27, 2023 12:28
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.10.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants