Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and extract payload to header conversion #5101

Merged
merged 1 commit into from
Feb 7, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Feb 5, 2023

Earlier blinded header on/post capella was assumed to be different fromexecution header in terms of having only transactions replaced by transaction root.

However since, blinded header is just same as execution header now so blinded will already have withdrawalsRoot rather than withdrawals
This is relevant from capella onwards and doesn't affect bellatrix

and the conversion is abstracted and imported to be used in PR:

@g11tech g11tech requested a review from a team as a code owner February 5, 2023 14:22
@g11tech g11tech enabled auto-merge (squash) February 5, 2023 14:23
@github-actions
Copy link
Contributor

github-actions bot commented Feb 5, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: d8a0f06 Previous: 4e25368 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 857.23 us/op 506.00 us/op 1.69
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 51.207 us/op 46.916 us/op 1.09
BLS verify - blst-native 1.2645 ms/op 1.2326 ms/op 1.03
BLS verifyMultipleSignatures 3 - blst-native 2.5875 ms/op 2.4882 ms/op 1.04
BLS verifyMultipleSignatures 8 - blst-native 5.4232 ms/op 5.3593 ms/op 1.01
BLS verifyMultipleSignatures 32 - blst-native 19.977 ms/op 19.438 ms/op 1.03
BLS aggregatePubkeys 32 - blst-native 26.671 us/op 25.785 us/op 1.03
BLS aggregatePubkeys 128 - blst-native 103.91 us/op 101.52 us/op 1.02
getAttestationsForBlock 63.321 ms/op 56.150 ms/op 1.13
isKnown best case - 1 super set check 275.00 ns/op 285.00 ns/op 0.96
isKnown normal case - 2 super set checks 269.00 ns/op 277.00 ns/op 0.97
isKnown worse case - 16 super set checks 267.00 ns/op 277.00 ns/op 0.96
CheckpointStateCache - add get delete 5.7950 us/op 5.4190 us/op 1.07
validate gossip signedAggregateAndProof - struct 2.9586 ms/op 2.7827 ms/op 1.06
validate gossip attestation - struct 1.3599 ms/op 1.3218 ms/op 1.03
pickEth1Vote - no votes 1.3488 ms/op 1.2717 ms/op 1.06
pickEth1Vote - max votes 11.087 ms/op 11.458 ms/op 0.97
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.7019 ms/op 9.1114 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.049 ms/op 15.986 ms/op 1.07
pickEth1Vote - Eth1Data fastSerialize value x2048 780.94 us/op 683.39 us/op 1.14
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.4068 ms/op 6.9909 ms/op 1.06
bytes32 toHexString 625.00 ns/op 521.00 ns/op 1.20
bytes32 Buffer.toString(hex) 437.00 ns/op 365.00 ns/op 1.20
bytes32 Buffer.toString(hex) from Uint8Array 615.00 ns/op 552.00 ns/op 1.11
bytes32 Buffer.toString(hex) + 0x 434.00 ns/op 349.00 ns/op 1.24
Object access 1 prop 0.20700 ns/op 0.17600 ns/op 1.18
Map access 1 prop 0.16900 ns/op 0.16100 ns/op 1.05
Object get x1000 7.2840 ns/op 6.6100 ns/op 1.10
Map get x1000 0.66200 ns/op 0.55100 ns/op 1.20
Object set x1000 64.693 ns/op 54.743 ns/op 1.18
Map set x1000 51.859 ns/op 44.174 ns/op 1.17
Return object 10000 times 0.26460 ns/op 0.24280 ns/op 1.09
Throw Error 10000 times 4.6081 us/op 4.2589 us/op 1.08
fastMsgIdFn sha256 / 200 bytes 3.6900 us/op 3.4960 us/op 1.06
fastMsgIdFn h32 xxhash / 200 bytes 314.00 ns/op 277.00 ns/op 1.13
fastMsgIdFn h64 xxhash / 200 bytes 455.00 ns/op 386.00 ns/op 1.18
fastMsgIdFn sha256 / 1000 bytes 12.223 us/op 11.610 us/op 1.05
fastMsgIdFn h32 xxhash / 1000 bytes 465.00 ns/op 430.00 ns/op 1.08
fastMsgIdFn h64 xxhash / 1000 bytes 532.00 ns/op 474.00 ns/op 1.12
fastMsgIdFn sha256 / 10000 bytes 107.66 us/op 105.11 us/op 1.02
fastMsgIdFn h32 xxhash / 10000 bytes 2.0840 us/op 2.0530 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.5130 us/op 1.4270 us/op 1.06
enrSubnets - fastDeserialize 64 bits 1.5400 us/op 1.2890 us/op 1.19
enrSubnets - ssz BitVector 64 bits 562.00 ns/op 485.00 ns/op 1.16
enrSubnets - fastDeserialize 4 bits 195.00 ns/op 166.00 ns/op 1.17
enrSubnets - ssz BitVector 4 bits 619.00 ns/op 483.00 ns/op 1.28
prioritizePeers score -10:0 att 32-0.1 sync 2-0 108.86 us/op 96.902 us/op 1.12
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 142.29 us/op 124.81 us/op 1.14
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 219.61 us/op 172.97 us/op 1.27
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 383.04 us/op 302.99 us/op 1.26
prioritizePeers score 0:0 att 64-1 sync 4-1 457.56 us/op 369.98 us/op 1.24
array of 16000 items push then shift 1.7995 us/op 1.6564 us/op 1.09
LinkedList of 16000 items push then shift 9.4350 ns/op 8.8520 ns/op 1.07
array of 16000 items push then pop 122.64 ns/op 96.169 ns/op 1.28
LinkedList of 16000 items push then pop 8.9680 ns/op 9.0180 ns/op 0.99
array of 24000 items push then shift 2.4426 us/op 2.4588 us/op 0.99
LinkedList of 24000 items push then shift 9.6420 ns/op 9.6920 ns/op 0.99
array of 24000 items push then pop 85.905 ns/op 90.276 ns/op 0.95
LinkedList of 24000 items push then pop 9.0180 ns/op 8.9790 ns/op 1.00
intersect bitArray bitLen 8 14.162 ns/op 13.452 ns/op 1.05
intersect array and set length 8 102.06 ns/op 87.149 ns/op 1.17
intersect bitArray bitLen 128 45.497 ns/op 45.358 ns/op 1.00
intersect array and set length 128 1.2323 us/op 1.1409 us/op 1.08
Buffer.concat 32 items 2.8090 us/op 2.7930 us/op 1.01
Uint8Array.set 32 items 2.7060 us/op 2.7350 us/op 0.99
pass gossip attestations to forkchoice per slot 2.4208 ms/op 2.4477 ms/op 0.99
computeDeltas 3.1326 ms/op 2.8650 ms/op 1.09
computeProposerBoostScoreFromBalances 1.9435 ms/op 1.8019 ms/op 1.08
altair processAttestation - 250000 vs - 7PWei normalcase 3.1501 ms/op 2.4781 ms/op 1.27
altair processAttestation - 250000 vs - 7PWei worstcase 4.9555 ms/op 3.6864 ms/op 1.34
altair processAttestation - setStatus - 1/6 committees join 153.20 us/op 155.03 us/op 0.99
altair processAttestation - setStatus - 1/3 committees join 318.31 us/op 280.09 us/op 1.14
altair processAttestation - setStatus - 1/2 committees join 456.57 us/op 391.61 us/op 1.17
altair processAttestation - setStatus - 2/3 committees join 530.71 us/op 485.41 us/op 1.09
altair processAttestation - setStatus - 4/5 committees join 773.53 us/op 671.26 us/op 1.15
altair processAttestation - setStatus - 100% committees join 908.01 us/op 787.61 us/op 1.15
altair processBlock - 250000 vs - 7PWei normalcase 18.937 ms/op 18.815 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 33.683 ms/op 29.385 ms/op 1.15
altair processBlock - 250000 vs - 7PWei worstcase 56.852 ms/op 46.271 ms/op 1.23
altair processBlock - 250000 vs - 7PWei worstcase hashState 74.625 ms/op 70.207 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei normalcase 3.0487 ms/op 2.2001 ms/op 1.39
phase0 processBlock - 250000 vs - 7PWei worstcase 39.241 ms/op 29.043 ms/op 1.35
altair processEth1Data - 250000 vs - 7PWei normalcase 737.56 us/op 497.44 us/op 1.48
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 15.327 us/op 8.1280 us/op 1.89
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 34.747 us/op 24.934 us/op 1.39
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 14.937 us/op 10.057 us/op 1.49
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 12.717 us/op 7.6490 us/op 1.66
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 115.48 us/op 98.751 us/op 1.17
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 824.98 us/op 650.40 us/op 1.27
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 924.29 us/op 919.22 us/op 1.01
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 992.98 us/op 909.94 us/op 1.09
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 3.1110 ms/op 2.3720 ms/op 1.31
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.9657 ms/op 1.7213 ms/op 1.14
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 7.0181 ms/op 4.1482 ms/op 1.69
Tree 40 250000 create 523.00 ms/op 333.32 ms/op 1.57
Tree 40 250000 get(125000) 204.65 ns/op 187.15 ns/op 1.09
Tree 40 250000 set(125000) 1.2305 us/op 969.12 ns/op 1.27
Tree 40 250000 toArray() 24.393 ms/op 21.029 ms/op 1.16
Tree 40 250000 iterate all - toArray() + loop 26.564 ms/op 20.846 ms/op 1.27
Tree 40 250000 iterate all - get(i) 85.112 ms/op 73.281 ms/op 1.16
MutableVector 250000 create 12.976 ms/op 11.245 ms/op 1.15
MutableVector 250000 get(125000) 7.2600 ns/op 6.4520 ns/op 1.13
MutableVector 250000 set(125000) 645.43 ns/op 280.44 ns/op 2.30
MutableVector 250000 toArray() 4.3290 ms/op 3.4943 ms/op 1.24
MutableVector 250000 iterate all - toArray() + loop 4.1449 ms/op 4.2385 ms/op 0.98
MutableVector 250000 iterate all - get(i) 1.6074 ms/op 1.5671 ms/op 1.03
Array 250000 create 3.4568 ms/op 3.3848 ms/op 1.02
Array 250000 clone - spread 1.1985 ms/op 1.1307 ms/op 1.06
Array 250000 get(125000) 0.62300 ns/op 0.55300 ns/op 1.13
Array 250000 set(125000) 0.71800 ns/op 0.62800 ns/op 1.14
Array 250000 iterate all - loop 92.413 us/op 85.938 us/op 1.08
effectiveBalanceIncrements clone Uint8Array 300000 48.717 us/op 33.412 us/op 1.46
effectiveBalanceIncrements clone MutableVector 300000 343.00 ns/op 339.00 ns/op 1.01
effectiveBalanceIncrements rw all Uint8Array 300000 180.82 us/op 173.41 us/op 1.04
effectiveBalanceIncrements rw all MutableVector 300000 97.177 ms/op 90.345 ms/op 1.08
phase0 afterProcessEpoch - 250000 vs - 7PWei 122.02 ms/op 115.57 ms/op 1.06
phase0 beforeProcessEpoch - 250000 vs - 7PWei 55.753 ms/op 44.857 ms/op 1.24
altair processEpoch - mainnet_e81889 349.79 ms/op 337.04 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 85.704 ms/op 54.561 ms/op 1.57
mainnet_e81889 - altair processJustificationAndFinalization 33.561 us/op 19.817 us/op 1.69
mainnet_e81889 - altair processInactivityUpdates 8.7627 ms/op 6.9169 ms/op 1.27
mainnet_e81889 - altair processRewardsAndPenalties 76.362 ms/op 73.844 ms/op 1.03
mainnet_e81889 - altair processRegistryUpdates 6.6310 us/op 4.2070 us/op 1.58
mainnet_e81889 - altair processSlashings 1.2120 us/op 818.00 ns/op 1.48
mainnet_e81889 - altair processEth1DataReset 1.5420 us/op 773.00 ns/op 1.99
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.5804 ms/op 1.2733 ms/op 1.24
mainnet_e81889 - altair processSlashingsReset 10.237 us/op 4.9060 us/op 2.09
mainnet_e81889 - altair processRandaoMixesReset 8.7100 us/op 4.9570 us/op 1.76
mainnet_e81889 - altair processHistoricalRootsUpdate 1.8550 us/op 826.00 ns/op 2.25
mainnet_e81889 - altair processParticipationFlagUpdates 8.6330 us/op 5.0320 us/op 1.72
mainnet_e81889 - altair processSyncCommitteeUpdates 1.0430 us/op 1.1450 us/op 0.91
mainnet_e81889 - altair afterProcessEpoch 143.78 ms/op 132.58 ms/op 1.08
phase0 processEpoch - mainnet_e58758 436.80 ms/op 380.65 ms/op 1.15
mainnet_e58758 - phase0 beforeProcessEpoch 168.63 ms/op 162.46 ms/op 1.04
mainnet_e58758 - phase0 processJustificationAndFinalization 26.678 us/op 27.108 us/op 0.98
mainnet_e58758 - phase0 processRewardsAndPenalties 69.532 ms/op 69.179 ms/op 1.01
mainnet_e58758 - phase0 processRegistryUpdates 11.483 us/op 12.802 us/op 0.90
mainnet_e58758 - phase0 processSlashings 781.00 ns/op 1.0730 us/op 0.73
mainnet_e58758 - phase0 processEth1DataReset 684.00 ns/op 998.00 ns/op 0.69
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.4374 ms/op 1.0619 ms/op 1.35
mainnet_e58758 - phase0 processSlashingsReset 5.5260 us/op 4.2900 us/op 1.29
mainnet_e58758 - phase0 processRandaoMixesReset 5.9330 us/op 5.2250 us/op 1.14
mainnet_e58758 - phase0 processHistoricalRootsUpdate 563.00 ns/op 737.00 ns/op 0.76
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.5710 us/op 6.3020 us/op 1.20
mainnet_e58758 - phase0 afterProcessEpoch 107.10 ms/op 103.67 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3772 ms/op 1.3649 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6452 ms/op 1.5461 ms/op 1.06
altair processInactivityUpdates - 250000 normalcase 27.774 ms/op 24.799 ms/op 1.12
altair processInactivityUpdates - 250000 worstcase 28.962 ms/op 28.955 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 8.7120 us/op 8.0670 us/op 1.08
phase0 processRegistryUpdates - 250000 badcase_full_deposits 353.49 us/op 277.45 us/op 1.27
phase0 processRegistryUpdates - 250000 worstcase 0.5 152.11 ms/op 113.67 ms/op 1.34
altair processRewardsAndPenalties - 250000 normalcase 72.491 ms/op 67.536 ms/op 1.07
altair processRewardsAndPenalties - 250000 worstcase 79.763 ms/op 73.459 ms/op 1.09
phase0 getAttestationDeltas - 250000 normalcase 7.2774 ms/op 7.2087 ms/op 1.01
phase0 getAttestationDeltas - 250000 worstcase 8.2075 ms/op 7.1722 ms/op 1.14
phase0 processSlashings - 250000 worstcase 3.6314 ms/op 3.6490 ms/op 1.00
altair processSyncCommitteeUpdates - 250000 202.72 ms/op 322.95 ms/op 0.63
BeaconState.hashTreeRoot - No change 354.00 ns/op 477.00 ns/op 0.74
BeaconState.hashTreeRoot - 1 full validator 54.258 us/op 65.986 us/op 0.82
BeaconState.hashTreeRoot - 32 full validator 553.93 us/op 825.14 us/op 0.67
BeaconState.hashTreeRoot - 512 full validator 5.7552 ms/op 6.6218 ms/op 0.87
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 68.358 us/op 69.562 us/op 0.98
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1483 ms/op 960.16 us/op 1.20
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.576 ms/op 13.778 ms/op 0.91
BeaconState.hashTreeRoot - 1 balances 51.760 us/op 54.212 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 488.91 us/op 521.58 us/op 0.94
BeaconState.hashTreeRoot - 512 balances 4.5109 ms/op 5.1542 ms/op 0.88
BeaconState.hashTreeRoot - 250000 balances 82.457 ms/op 74.221 ms/op 1.11
aggregationBits - 2048 els - zipIndexesInBitList 22.777 us/op 24.643 us/op 0.92
regular array get 100000 times 34.574 us/op 44.860 us/op 0.77
wrappedArray get 100000 times 34.785 us/op 34.075 us/op 1.02
arrayWithProxy get 100000 times 18.106 ms/op 17.537 ms/op 1.03
ssz.Root.equals 707.00 ns/op 591.00 ns/op 1.20
byteArrayEquals 660.00 ns/op 602.00 ns/op 1.10
shuffle list - 16384 els 7.7317 ms/op 10.729 ms/op 0.72
shuffle list - 250000 els 108.99 ms/op 151.31 ms/op 0.72
processSlot - 1 slots 9.9240 us/op 14.880 us/op 0.67
processSlot - 32 slots 1.4881 ms/op 2.2254 ms/op 0.67
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 216.58 us/op 274.98 us/op 0.79
getCommitteeAssignments - req 1 vs - 250000 vc 3.1361 ms/op 4.2127 ms/op 0.74
getCommitteeAssignments - req 100 vs - 250000 vc 4.5518 ms/op 5.8854 ms/op 0.77
getCommitteeAssignments - req 1000 vs - 250000 vc 4.9048 ms/op 5.4724 ms/op 0.90
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.1600 ns/op 9.0700 ns/op 0.57
state getBlockRootAtSlot - 250000 vs - 7PWei 862.27 ns/op 1.3379 us/op 0.64
computeProposers - vc 250000 11.148 ms/op 12.398 ms/op 0.90
computeEpochShuffling - vc 250000 110.38 ms/op 135.57 ms/op 0.81
getNextSyncCommittee - vc 250000 187.03 ms/op 206.53 ms/op 0.91

by benchmarkbot/action

@g11tech g11tech force-pushed the g11tech/fix-payload-to-header branch from 4ae79dc to ee65856 Compare February 7, 2023 17:00
@g11tech g11tech merged commit 79301db into unstable Feb 7, 2023
@g11tech g11tech deleted the g11tech/fix-payload-to-header branch February 7, 2023 19:13
dapplion added a commit that referenced this pull request Feb 28, 2023
* v1.5.0

* Add validatior option to specify builder block selection strategy (#5064)

* Add validatior option to specify builder block selection strategy

* cleanup and improvements

* server api fx

* fix test

* fix the testcase

* Re-gossip bls changes if submitted earlier than capella (#5049)

* Cache and retransmit bls changes if submitted early

* regossip when synced on/post capella

* fix tests

* modify publish vs cache condition

* add logging

* add error logging

* batchify the processing

* fix some of the cases

* fix log issues

* also delete included keys

* fix lint

* simplify gossip loop

* fix build

* refac to a versioned bls change store

* update

* logging improvements

* improve logging and finish behavior

* cleanup

* Rename eip4844 fork as Deneb (#5082)

* Fix and extract payload to header conversion (#5101)

* Remove unused eslint disable directives (#5046)

* Add execution payload header to light client header capella onwards (#5027)

* Add execution payload header to light client header capella onwards

* add more forks

* add allfork types

* make other types multifork

* make events multifork compatible

* make lightclient routes multifork

* update light-client transport

* upgrade the blocktype on lightclient server import block

* fixes

* fix test

* fix event serialization/deserilization type

* update proof constants

* fix payload index

* update the light client update generation to multi fork

* fix types

* add spec override to pass specs

* make event types allforks

* make req/resp multifork

* make types multifork throughout

* fix the update serialize/deserialize

* fix error log

* better fork determination for update

* fix the fork of finalized

* fix tests

* fix header upgradation

* Fix zero finalized header in lightclient update gen (#5121)

* Schedule capella on sepolia (#5127)

* Get the capella builder functional (#5179)

* Allow blsChange reprocess/republish via API submission (#5185)

* Fix gossiping blsChange on submit if post capella

* comment improvement

* Use new db bucket for light client updates (#5205)

---------

Co-authored-by: Cayman <caymannava@gmail.com>
Co-authored-by: g11tech <develop@g11tech.io>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants