Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do single call to produce attestation #3917

Merged
merged 1 commit into from
Apr 13, 2022
Merged

Do single call to produce attestation #3917

merged 1 commit into from
Apr 13, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Apr 13, 2022 •
edited
Loading

Motivation

Beacon node's endpoint produceAttestationData return data is not dependant on committeeIndex. We can do a single call for all committees and save resources

Description

Produce a single attestation for all committees, and clone mutate before signing

Closes #3098

@dapplion dapplion requested a review from a team as a code owner April 13, 2022 10:43
twoeths
twoeths approved these changes Apr 13, 2022
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@codecov
Copy link

codecov bot commented Apr 13, 2022

Codecov Report

Merging #3917 (c51289e) into master (106a05a) will increase coverage by 0.01%.
The diff coverage is n/a.

❗ Current head c51289e differs from pull request most recent head 222a5cf. Consider uploading reports for the commit 222a5cf to get more accurate results

@@            Coverage Diff             @@
##           master    #3917      +/-   ##
==========================================
+ Coverage   36.52%   36.53%   +0.01%     
==========================================
  Files         324      324              
  Lines        8980     8982       +2     
  Branches     1410     1410              
==========================================
+ Hits         3280     3282       +2     
  Misses       5556     5556              
  Partials      144      144

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c51289e Previous: 106a05a Ratio
BeaconState.hashTreeRoot - No change 528.00 ns/op 440.00 ns/op 1.20
BeaconState.hashTreeRoot - 1 full validator 54.948 us/op 55.130 us/op 1.00
BeaconState.hashTreeRoot - 32 full validator 544.21 us/op 536.74 us/op 1.01
BeaconState.hashTreeRoot - 512 full validator 6.0717 ms/op 5.8997 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 68.683 us/op 67.312 us/op 1.02
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 953.13 us/op 940.01 us/op 1.01
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 13.080 ms/op 12.738 ms/op 1.03
BeaconState.hashTreeRoot - 1 balances 53.125 us/op 52.592 us/op 1.01
BeaconState.hashTreeRoot - 32 balances 479.93 us/op 461.04 us/op 1.04
BeaconState.hashTreeRoot - 512 balances 4.5712 ms/op 4.5207 ms/op 1.01
BeaconState.hashTreeRoot - 250000 balances 102.57 ms/op 95.171 ms/op 1.08
processSlot - 1 slots 9.5710 us/op 9.5890 us/op 1.00
processSlot - 32 slots 1.6008 ms/op 1.5246 ms/op 1.05
getCommitteeAssignments - req 1 vs - 250000 vc 5.2587 ms/op 5.2945 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 7.3402 ms/op 7.3714 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 7.8315 ms/op 7.8486 ms/op 1.00
computeProposers - vc 250000 16.834 ms/op 16.355 ms/op 1.03
computeEpochShuffling - vc 250000 159.49 ms/op 159.21 ms/op 1.00
getNextSyncCommittee - vc 250000 275.97 ms/op 265.84 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei normalcase 3.5791 ms/op 3.8302 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei worstcase 5.2805 ms/op 5.1141 ms/op 1.03
altair processAttestation - setStatus - 1/6 committees join 193.98 us/op 190.37 us/op 1.02
altair processAttestation - setStatus - 1/3 committees join 373.13 us/op 372.90 us/op 1.00
altair processAttestation - setStatus - 1/2 committees join 524.54 us/op 520.75 us/op 1.01
altair processAttestation - setStatus - 2/3 committees join 673.91 us/op 666.33 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 928.62 us/op 908.95 us/op 1.02
altair processAttestation - setStatus - 100% committees join 1.0984 ms/op 1.0747 ms/op 1.02
altair processBlock - 250000 vs - 7PWei normalcase 25.725 ms/op 24.927 ms/op 1.03
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.712 ms/op 34.794 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase 77.462 ms/op 76.633 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase hashState 96.651 ms/op 95.229 ms/op 1.01
altair processEth1Data - 250000 vs - 7PWei normalcase 804.25 us/op 769.54 us/op 1.05
altair processEpoch - mainnet_e81889 483.98 ms/op 557.49 ms/op 0.87
mainnet_e81889 - altair beforeProcessEpoch 147.63 ms/op 145.21 ms/op 1.02
mainnet_e81889 - altair processJustificationAndFinalization 25.592 us/op 41.264 us/op 0.62
mainnet_e81889 - altair processInactivityUpdates 10.859 ms/op 11.089 ms/op 0.98
mainnet_e81889 - altair processRewardsAndPenalties 133.79 ms/op 83.409 ms/op 1.60
mainnet_e81889 - altair processRegistryUpdates 3.6600 us/op 7.1120 us/op 0.51
mainnet_e81889 - altair processSlashings 1.0350 us/op 1.7690 us/op 0.59
mainnet_e81889 - altair processEth1DataReset 1.1550 us/op 1.7750 us/op 0.65
mainnet_e81889 - altair processEffectiveBalanceUpdates 6.7868 ms/op 6.4403 ms/op 1.05
mainnet_e81889 - altair processSlashingsReset 4.9720 us/op 9.0110 us/op 0.55
mainnet_e81889 - altair processRandaoMixesReset 6.4040 us/op 11.365 us/op 0.56
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2840 us/op 2.3890 us/op 0.54
mainnet_e81889 - altair processParticipationFlagUpdates 5.6560 us/op 5.3670 us/op 1.05
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1460 us/op 3.1060 us/op 0.37
mainnet_e81889 - altair afterProcessEpoch 207.81 ms/op 209.03 ms/op 0.99
altair processInactivityUpdates - 250000 normalcase 32.309 ms/op 31.760 ms/op 1.02
altair processInactivityUpdates - 250000 worstcase 27.104 ms/op 26.760 ms/op 1.01
altair processRewardsAndPenalties - 250000 normalcase 114.47 ms/op 113.70 ms/op 1.01
altair processRewardsAndPenalties - 250000 worstcase 104.16 ms/op 109.11 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 284.24 ms/op 297.22 ms/op 0.96
Tree 40 250000 create 679.18 ms/op 682.79 ms/op 0.99
Tree 40 250000 get(125000) 284.42 ns/op 281.39 ns/op 1.01
Tree 40 250000 set(125000) 2.2743 us/op 2.1985 us/op 1.03
Tree 40 250000 toArray() 32.452 ms/op 32.449 ms/op 1.00
Tree 40 250000 iterate all - toArray() + loop 32.996 ms/op 31.773 ms/op 1.04
Tree 40 250000 iterate all - get(i) 110.50 ms/op 109.90 ms/op 1.01
MutableVector 250000 create 15.043 ms/op 14.789 ms/op 1.02
MutableVector 250000 get(125000) 14.779 ns/op 14.751 ns/op 1.00
MutableVector 250000 set(125000) 607.86 ns/op 623.81 ns/op 0.97
MutableVector 250000 toArray() 6.4662 ms/op 6.0582 ms/op 1.07
MutableVector 250000 iterate all - toArray() + loop 6.2815 ms/op 6.2208 ms/op 1.01
MutableVector 250000 iterate all - get(i) 3.4369 ms/op 3.4691 ms/op 0.99
Array 250000 create 5.5359 ms/op 5.3134 ms/op 1.04
Array 250000 clone - spread 2.4811 ms/op 2.3928 ms/op 1.04
Array 250000 get(125000) 1.1910 ns/op 1.1320 ns/op 1.05
Array 250000 set(125000) 1.1950 ns/op 1.1240 ns/op 1.06
Array 250000 iterate all - loop 167.88 us/op 167.85 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 70.302 us/op 62.669 us/op 1.12
effectiveBalanceIncrements clone MutableVector 300000 575.00 ns/op 543.00 ns/op 1.06
effectiveBalanceIncrements rw all Uint8Array 300000 301.71 us/op 301.79 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 161.96 ms/op 153.18 ms/op 1.06
aggregationBits - 2048 els - zipIndexesInBitList 26.733 us/op 25.764 us/op 1.04
regular array get 100000 times 67.445 us/op 67.430 us/op 1.00
wrappedArray get 100000 times 67.436 us/op 67.397 us/op 1.00
arrayWithProxy get 100000 times 32.205 ms/op 32.821 ms/op 0.98
ssz.Root.equals 430.00 ns/op 442.00 ns/op 0.97
byteArrayEquals 412.00 ns/op 408.00 ns/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 3.4972 ms/op 3.2645 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei worstcase 45.323 ms/op 44.527 ms/op 1.02
phase0 afterProcessEpoch - 250000 vs - 7PWei 174.47 ms/op 175.74 ms/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 77.049 ms/op 78.346 ms/op 0.98
phase0 processEpoch - mainnet_e58758 542.68 ms/op 534.91 ms/op 1.01
mainnet_e58758 - phase0 beforeProcessEpoch 242.82 ms/op 211.46 ms/op 1.15
mainnet_e58758 - phase0 processJustificationAndFinalization 34.435 us/op 31.091 us/op 1.11
mainnet_e58758 - phase0 processRewardsAndPenalties 127.69 ms/op 70.734 ms/op 1.81
mainnet_e58758 - phase0 processRegistryUpdates 18.572 us/op 13.024 us/op 1.43
mainnet_e58758 - phase0 processSlashings 1.5930 us/op 1.4310 us/op 1.11
mainnet_e58758 - phase0 processEth1DataReset 1.8580 us/op 1.1550 us/op 1.61
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 5.7729 ms/op 5.5103 ms/op 1.05
mainnet_e58758 - phase0 processSlashingsReset 5.8540 us/op 9.3760 us/op 0.62
mainnet_e58758 - phase0 processRandaoMixesReset 6.4440 us/op 10.470 us/op 0.62
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2600 us/op 2.2260 us/op 0.57
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.2850 us/op 9.2800 us/op 0.57
mainnet_e58758 - phase0 afterProcessEpoch 156.39 ms/op 157.15 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 6.6366 ms/op 6.3676 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 7.0351 ms/op 6.8234 ms/op 1.03
phase0 processRegistryUpdates - 250000 normalcase 10.638 us/op 13.901 us/op 0.77
phase0 processRegistryUpdates - 250000 badcase_full_deposits 431.33 us/op 649.88 us/op 0.66
phase0 processRegistryUpdates - 250000 worstcase 0.5 216.00 ms/op 232.22 ms/op 0.93
phase0 getAttestationDeltas - 250000 normalcase 14.939 ms/op 14.170 ms/op 1.05
phase0 getAttestationDeltas - 250000 worstcase 14.729 ms/op 14.105 ms/op 1.04
phase0 processSlashings - 250000 worstcase 5.4844 ms/op 5.9337 ms/op 0.92
shuffle list - 16384 els 10.684 ms/op 11.233 ms/op 0.95
shuffle list - 250000 els 153.78 ms/op 154.67 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 381.12 us/op 373.32 us/op 1.02
pass gossip attestations to forkchoice per slot 6.0175 ms/op 5.8557 ms/op 1.03
computeDeltas 3.6586 ms/op 3.7220 ms/op 0.98
computeProposerBoostScoreFromBalances 503.13 us/op 502.90 us/op 1.00
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0329 ms/op 1.7884 ms/op 1.14
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 75.468 us/op 72.857 us/op 1.04
BLS verify - blst-native 1.8594 ms/op 1.8581 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8027 ms/op 3.8062 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.1883 ms/op 8.1980 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.667 ms/op 29.709 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.575 us/op 39.290 us/op 1.01
BLS aggregatePubkeys 128 - blst-native 154.15 us/op 153.81 us/op 1.00
getAttestationsForBlock 57.014 ms/op 54.063 ms/op 1.05
CheckpointStateCache - add get delete 10.305 us/op 10.091 us/op 1.02
validate gossip signedAggregateAndProof - struct 4.7103 ms/op 4.2544 ms/op 1.11
validate gossip attestation - struct 2.0234 ms/op 2.0247 ms/op 1.00
pickEth1Vote - no votes 2.1867 ms/op 2.1381 ms/op 1.02
pickEth1Vote - max votes 19.764 ms/op 20.019 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.426 ms/op 11.767 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 22.106 ms/op 20.732 ms/op 1.07
pickEth1Vote - Eth1Data fastSerialize value x2048 1.6478 ms/op 1.6060 ms/op 1.03
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.249 ms/op 11.671 ms/op 1.05
bytes32 toHexString 938.00 ns/op 929.00 ns/op 1.01
bytes32 Buffer.toString(hex) 673.00 ns/op 686.00 ns/op 0.98
bytes32 Buffer.toString(hex) from Uint8Array 882.00 ns/op 892.00 ns/op 0.99
bytes32 Buffer.toString(hex) + 0x 694.00 ns/op 696.00 ns/op 1.00
Object access 1 prop 0.32600 ns/op 0.32100 ns/op 1.02
Map access 1 prop 0.29100 ns/op 0.30600 ns/op 0.95
Object get x1000 18.000 ns/op 17.591 ns/op 1.02
Map get x1000 1.0570 ns/op 1.0360 ns/op 1.02
Object set x1000 103.79 ns/op 95.247 ns/op 1.09
Map set x1000 63.531 ns/op 60.859 ns/op 1.04
Return object 10000 times 0.37370 ns/op 0.36950 ns/op 1.01
Throw Error 10000 times 5.8661 us/op 5.8602 us/op 1.00
enrSubnets - fastDeserialize 64 bits 875.00 ns/op 853.00 ns/op 1.03
enrSubnets - ssz BitVector 64 bits 586.00 ns/op 601.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 376.00 ns/op 374.00 ns/op 1.01
enrSubnets - ssz BitVector 4 bits 590.00 ns/op 577.00 ns/op 1.02
RateTracker 1000000 limit, 1 obj count per request 179.21 ns/op 173.90 ns/op 1.03
RateTracker 1000000 limit, 2 obj count per request 134.72 ns/op 128.46 ns/op 1.05
RateTracker 1000000 limit, 4 obj count per request 111.62 ns/op 105.71 ns/op 1.06
RateTracker 1000000 limit, 8 obj count per request 99.700 ns/op 94.536 ns/op 1.05
RateTracker with prune 3.8960 us/op 3.6610 us/op 1.06
array of 16000 items push then shift 3.1480 us/op 3.1540 us/op 1.00
LinkedList of 16000 items push then shift 17.659 ns/op 16.483 ns/op 1.07
array of 16000 items push then pop 206.52 ns/op 206.76 ns/op 1.00
LinkedList of 16000 items push then pop 16.689 ns/op 15.812 ns/op 1.06
array of 24000 items push then shift 4.5661 us/op 4.5515 us/op 1.00
LinkedList of 24000 items push then shift 21.835 ns/op 18.592 ns/op 1.17
array of 24000 items push then pop 182.52 ns/op 183.40 ns/op 1.00
LinkedList of 24000 items push then pop 18.942 ns/op 17.370 ns/op 1.09

by benchmarkbot/action

@dapplion dapplion merged commit 5076ea4 into master Apr 13, 2022
@dapplion dapplion deleted the dapplion/produce-attestation-once branch April 13, 2022 11:14
@dapplion dapplion mentioned this pull request Apr 13, 2022
Merged
@nflaig
Copy link
Member

nflaig commented Mar 10, 2023 •
edited
Loading

@dapplion The DVT compatibility issue mentioned in #5103 is due to the changes in this PR. This also introduced a deviation compared to other CL clients as those send a request per committee index.

I am wondering if we really need this optimization as there is an upper bound of 64 requests due to MAX_COMMITTEES_PER_SLOT but of course it will put more load on the BN either way.

I really want to keep the optimization while also fixing the compatibility issues we have.
As described in the issue, my idea is to introduce a flag --distributed (also already considering aggregate committee selections) and if set the attestation service behaves differently (disables the optimization). My main concern with all of this is that we introduce branching logic the attestation flow which I really want to avoid.

Considering all of that, instead of adding branches to the current attenstation service I think we might just be best of creating a 2nd "distributed" attestation service.

Edit: opened a PR to propose and discuss a possible solution #5258

@nflaig nflaig mentioned this pull request Mar 11, 2023
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Call produceAttestationData once for all validators
3 participants
@dapplion @nflaig @twoeths