Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update default builder selection strategy to MaxProfit #5220

Merged
merged 1 commit into from
Mar 1, 2023
Merged

Update default builder selection strategy to MaxProfit #5220

merged 1 commit into from
Mar 1, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Mar 1, 2023 •
edited by wemeetagain
Loading

Shift builder vs execution default selection strategy to max profit

making this as a default UX makes more sense for a retail user who has to supply less args 👍

@g11tech g11tech requested a review from a team as a code owner March 1, 2023 09:33
@g11tech g11tech enabled auto-merge (squash) March 1, 2023 09:33
@github-actions
Copy link
Contributor

github-actions bot commented Mar 1, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 351d630 Previous: ec78af1 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 595.42 us/op 561.77 us/op 1.06
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 56.553 us/op 47.246 us/op 1.20
BLS verify - blst-native 1.2362 ms/op 1.2165 ms/op 1.02
BLS verifyMultipleSignatures 3 - blst-native 2.5161 ms/op 2.4828 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.4413 ms/op 5.3460 ms/op 1.02
BLS verifyMultipleSignatures 32 - blst-native 19.754 ms/op 19.246 ms/op 1.03
BLS aggregatePubkeys 32 - blst-native 26.265 us/op 25.855 us/op 1.02
BLS aggregatePubkeys 128 - blst-native 103.40 us/op 100.60 us/op 1.03
getAttestationsForBlock 53.453 ms/op 57.051 ms/op 0.94
isKnown best case - 1 super set check 264.00 ns/op 267.00 ns/op 0.99
isKnown normal case - 2 super set checks 260.00 ns/op 264.00 ns/op 0.98
isKnown worse case - 16 super set checks 248.00 ns/op 253.00 ns/op 0.98
CheckpointStateCache - add get delete 5.4620 us/op 5.8040 us/op 0.94
validate gossip signedAggregateAndProof - struct 2.8233 ms/op 2.7830 ms/op 1.01
validate gossip attestation - struct 1.3437 ms/op 1.3270 ms/op 1.01
pickEth1Vote - no votes 1.3452 ms/op 1.3372 ms/op 1.01
pickEth1Vote - max votes 12.359 ms/op 9.9704 ms/op 1.24
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.5514 ms/op 9.0075 ms/op 1.06
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 15.555 ms/op 14.644 ms/op 1.06
pickEth1Vote - Eth1Data fastSerialize value x2048 741.23 us/op 670.85 us/op 1.10
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.9877 ms/op 7.2826 ms/op 0.96
bytes32 toHexString 607.00 ns/op 502.00 ns/op 1.21
bytes32 Buffer.toString(hex) 487.00 ns/op 350.00 ns/op 1.39
bytes32 Buffer.toString(hex) from Uint8Array 739.00 ns/op 581.00 ns/op 1.27
bytes32 Buffer.toString(hex) + 0x 574.00 ns/op 345.00 ns/op 1.66
Object access 1 prop 0.23900 ns/op 0.16700 ns/op 1.43
Map access 1 prop 0.18000 ns/op 0.16800 ns/op 1.07
Object get x1000 11.699 ns/op 6.4880 ns/op 1.80
Map get x1000 0.79500 ns/op 0.64100 ns/op 1.24
Object set x1000 97.546 ns/op 54.609 ns/op 1.79
Map set x1000 71.893 ns/op 45.160 ns/op 1.59
Return object 10000 times 0.28790 ns/op 0.24940 ns/op 1.15
Throw Error 10000 times 4.6879 us/op 4.2335 us/op 1.11
fastMsgIdFn sha256 / 200 bytes 3.8560 us/op 3.4850 us/op 1.11
fastMsgIdFn h32 xxhash / 200 bytes 329.00 ns/op 281.00 ns/op 1.17
fastMsgIdFn h64 xxhash / 200 bytes 493.00 ns/op 386.00 ns/op 1.28
fastMsgIdFn sha256 / 1000 bytes 12.189 us/op 11.728 us/op 1.04
fastMsgIdFn h32 xxhash / 1000 bytes 451.00 ns/op 420.00 ns/op 1.07
fastMsgIdFn h64 xxhash / 1000 bytes 549.00 ns/op 477.00 ns/op 1.15
fastMsgIdFn sha256 / 10000 bytes 108.36 us/op 104.18 us/op 1.04
fastMsgIdFn h32 xxhash / 10000 bytes 2.0400 us/op 1.9900 us/op 1.03
fastMsgIdFn h64 xxhash / 10000 bytes 1.4990 us/op 1.4080 us/op 1.06
enrSubnets - fastDeserialize 64 bits 1.9520 us/op 1.3230 us/op 1.48
enrSubnets - ssz BitVector 64 bits 645.00 ns/op 491.00 ns/op 1.31
enrSubnets - fastDeserialize 4 bits 217.00 ns/op 175.00 ns/op 1.24
enrSubnets - ssz BitVector 4 bits 604.00 ns/op 516.00 ns/op 1.17
prioritizePeers score -10:0 att 32-0.1 sync 2-0 106.66 us/op 105.90 us/op 1.01
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 143.81 us/op 133.26 us/op 1.08
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 212.93 us/op 194.72 us/op 1.09
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 380.57 us/op 360.89 us/op 1.05
prioritizePeers score 0:0 att 64-1 sync 4-1 413.26 us/op 427.19 us/op 0.97
array of 16000 items push then shift 1.7644 us/op 1.6700 us/op 1.06
LinkedList of 16000 items push then shift 9.5130 ns/op 9.0440 ns/op 1.05
array of 16000 items push then pop 122.14 ns/op 113.36 ns/op 1.08
LinkedList of 16000 items push then pop 9.4580 ns/op 8.9670 ns/op 1.05
array of 24000 items push then shift 2.5065 us/op 2.3933 us/op 1.05
LinkedList of 24000 items push then shift 9.9780 ns/op 9.0640 ns/op 1.10
array of 24000 items push then pop 90.893 ns/op 85.442 ns/op 1.06
LinkedList of 24000 items push then pop 9.3670 ns/op 8.9200 ns/op 1.05
intersect bitArray bitLen 8 13.953 ns/op 13.523 ns/op 1.03
intersect array and set length 8 100.23 ns/op 85.537 ns/op 1.17
intersect bitArray bitLen 128 47.856 ns/op 44.771 ns/op 1.07
intersect array and set length 128 1.3631 us/op 1.1676 us/op 1.17
Buffer.concat 32 items 3.0560 us/op 3.0110 us/op 1.01
Uint8Array.set 32 items 2.9620 us/op 2.6580 us/op 1.11
pass gossip attestations to forkchoice per slot 2.4691 ms/op 3.4182 ms/op 0.72
computeDeltas 3.1169 ms/op 2.9936 ms/op 1.04
computeProposerBoostScoreFromBalances 1.8258 ms/op 1.7953 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 2.8643 ms/op 2.4709 ms/op 1.16
altair processAttestation - 250000 vs - 7PWei worstcase 4.0434 ms/op 3.8026 ms/op 1.06
altair processAttestation - setStatus - 1/6 committees join 148.44 us/op 143.13 us/op 1.04
altair processAttestation - setStatus - 1/3 committees join 287.98 us/op 277.61 us/op 1.04
altair processAttestation - setStatus - 1/2 committees join 403.67 us/op 374.68 us/op 1.08
altair processAttestation - setStatus - 2/3 committees join 484.80 us/op 471.64 us/op 1.03
altair processAttestation - setStatus - 4/5 committees join 706.91 us/op 658.81 us/op 1.07
altair processAttestation - setStatus - 100% committees join 788.64 us/op 764.08 us/op 1.03
altair processBlock - 250000 vs - 7PWei normalcase 17.761 ms/op 17.536 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 29.095 ms/op 27.269 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 53.850 ms/op 52.513 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase hashState 75.416 ms/op 73.430 ms/op 1.03
phase0 processBlock - 250000 vs - 7PWei normalcase 2.5151 ms/op 2.1754 ms/op 1.16
phase0 processBlock - 250000 vs - 7PWei worstcase 33.135 ms/op 29.569 ms/op 1.12
altair processEth1Data - 250000 vs - 7PWei normalcase 554.50 us/op 495.61 us/op 1.12
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 10.302 us/op 9.0240 us/op 1.14
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 31.659 us/op 28.640 us/op 1.11
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.293 us/op 11.659 us/op 0.88
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 8.1430 us/op 8.6220 us/op 0.94
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 106.02 us/op 111.52 us/op 0.95
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 669.19 us/op 653.44 us/op 1.02
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 915.08 us/op 918.87 us/op 1.00
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 931.16 us/op 887.03 us/op 1.05
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.4281 ms/op 2.4370 ms/op 1.00
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5528 ms/op 1.5014 ms/op 1.03
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.9540 ms/op 3.9518 ms/op 1.00
Tree 40 250000 create 325.11 ms/op 342.41 ms/op 0.95
Tree 40 250000 get(125000) 185.19 ns/op 197.64 ns/op 0.94
Tree 40 250000 set(125000) 982.99 ns/op 1.0334 us/op 0.95
Tree 40 250000 toArray() 21.419 ms/op 21.456 ms/op 1.00
Tree 40 250000 iterate all - toArray() + loop 20.513 ms/op 22.216 ms/op 0.92
Tree 40 250000 iterate all - get(i) 73.764 ms/op 73.074 ms/op 1.01
MutableVector 250000 create 11.494 ms/op 10.786 ms/op 1.07
MutableVector 250000 get(125000) 6.6130 ns/op 8.3130 ns/op 0.80
MutableVector 250000 set(125000) 266.49 ns/op 273.52 ns/op 0.97
MutableVector 250000 toArray() 3.6620 ms/op 3.2361 ms/op 1.13
MutableVector 250000 iterate all - toArray() + loop 2.9570 ms/op 3.2199 ms/op 0.92
MutableVector 250000 iterate all - get(i) 1.5793 ms/op 1.5485 ms/op 1.02
Array 250000 create 2.7648 ms/op 2.6946 ms/op 1.03
Array 250000 clone - spread 1.1821 ms/op 1.1564 ms/op 1.02
Array 250000 get(125000) 0.60700 ns/op 0.59800 ns/op 1.02
Array 250000 set(125000) 0.66200 ns/op 0.68800 ns/op 0.96
Array 250000 iterate all - loop 108.34 us/op 103.56 us/op 1.05
effectiveBalanceIncrements clone Uint8Array 300000 25.961 us/op 34.899 us/op 0.74
effectiveBalanceIncrements clone MutableVector 300000 372.00 ns/op 368.00 ns/op 1.01
effectiveBalanceIncrements rw all Uint8Array 300000 168.67 us/op 170.64 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 81.761 ms/op 84.774 ms/op 0.96
phase0 afterProcessEpoch - 250000 vs - 7PWei 118.62 ms/op 117.61 ms/op 1.01
phase0 beforeProcessEpoch - 250000 vs - 7PWei 68.619 ms/op 41.593 ms/op 1.65
altair processEpoch - mainnet_e81889 340.39 ms/op 309.16 ms/op 1.10
mainnet_e81889 - altair beforeProcessEpoch 64.956 ms/op 51.774 ms/op 1.25
mainnet_e81889 - altair processJustificationAndFinalization 17.319 us/op 30.888 us/op 0.56
mainnet_e81889 - altair processInactivityUpdates 6.0832 ms/op 6.5172 ms/op 0.93
mainnet_e81889 - altair processRewardsAndPenalties 53.384 ms/op 71.161 ms/op 0.75
mainnet_e81889 - altair processRegistryUpdates 4.5960 us/op 2.4440 us/op 1.88
mainnet_e81889 - altair processSlashings 602.00 ns/op 542.00 ns/op 1.11
mainnet_e81889 - altair processEth1DataReset 493.00 ns/op 540.00 ns/op 0.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2549 ms/op 1.2670 ms/op 0.99
mainnet_e81889 - altair processSlashingsReset 5.0890 us/op 4.1020 us/op 1.24
mainnet_e81889 - altair processRandaoMixesReset 4.5700 us/op 4.6280 us/op 0.99
mainnet_e81889 - altair processHistoricalRootsUpdate 601.00 ns/op 731.00 ns/op 0.82
mainnet_e81889 - altair processParticipationFlagUpdates 3.6170 us/op 2.8830 us/op 1.25
mainnet_e81889 - altair processSyncCommitteeUpdates 850.00 ns/op 486.00 ns/op 1.75
mainnet_e81889 - altair afterProcessEpoch 130.85 ms/op 128.57 ms/op 1.02
phase0 processEpoch - mainnet_e58758 370.32 ms/op 369.13 ms/op 1.00
mainnet_e58758 - phase0 beforeProcessEpoch 139.71 ms/op 129.18 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 16.963 us/op 16.714 us/op 1.01
mainnet_e58758 - phase0 processRewardsAndPenalties 60.108 ms/op 62.154 ms/op 0.97
mainnet_e58758 - phase0 processRegistryUpdates 10.043 us/op 7.4570 us/op 1.35
mainnet_e58758 - phase0 processSlashings 500.00 ns/op 438.00 ns/op 1.14
mainnet_e58758 - phase0 processEth1DataReset 638.00 ns/op 461.00 ns/op 1.38
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0699 ms/op 1.0003 ms/op 1.07
mainnet_e58758 - phase0 processSlashingsReset 4.0870 us/op 3.7600 us/op 1.09
mainnet_e58758 - phase0 processRandaoMixesReset 5.5370 us/op 3.9490 us/op 1.40
mainnet_e58758 - phase0 processHistoricalRootsUpdate 578.00 ns/op 543.00 ns/op 1.06
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.1700 us/op 4.2900 us/op 0.97
mainnet_e58758 - phase0 afterProcessEpoch 100.67 ms/op 99.409 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2781 ms/op 1.2587 ms/op 1.02
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5414 ms/op 1.5505 ms/op 0.99
altair processInactivityUpdates - 250000 normalcase 25.858 ms/op 24.038 ms/op 1.08
altair processInactivityUpdates - 250000 worstcase 27.797 ms/op 28.974 ms/op 0.96
phase0 processRegistryUpdates - 250000 normalcase 9.4190 us/op 7.2410 us/op 1.30
phase0 processRegistryUpdates - 250000 badcase_full_deposits 289.44 us/op 268.30 us/op 1.08
phase0 processRegistryUpdates - 250000 worstcase 0.5 134.92 ms/op 124.30 ms/op 1.09
altair processRewardsAndPenalties - 250000 normalcase 69.284 ms/op 70.938 ms/op 0.98
altair processRewardsAndPenalties - 250000 worstcase 71.499 ms/op 72.080 ms/op 0.99
phase0 getAttestationDeltas - 250000 normalcase 7.5001 ms/op 6.6074 ms/op 1.14
phase0 getAttestationDeltas - 250000 worstcase 9.2376 ms/op 6.5565 ms/op 1.41
phase0 processSlashings - 250000 worstcase 3.8788 ms/op 3.5393 ms/op 1.10
altair processSyncCommitteeUpdates - 250000 188.92 ms/op 177.19 ms/op 1.07
BeaconState.hashTreeRoot - No change 287.00 ns/op 350.00 ns/op 0.82
BeaconState.hashTreeRoot - 1 full validator 51.744 us/op 53.448 us/op 0.97
BeaconState.hashTreeRoot - 32 full validator 566.19 us/op 475.64 us/op 1.19
BeaconState.hashTreeRoot - 512 full validator 5.2316 ms/op 5.5657 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 66.595 us/op 62.201 us/op 1.07
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 920.93 us/op 887.62 us/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 13.209 ms/op 10.954 ms/op 1.21
BeaconState.hashTreeRoot - 1 balances 52.505 us/op 46.914 us/op 1.12
BeaconState.hashTreeRoot - 32 balances 463.28 us/op 460.53 us/op 1.01
BeaconState.hashTreeRoot - 512 balances 4.7712 ms/op 4.5365 ms/op 1.05
BeaconState.hashTreeRoot - 250000 balances 80.878 ms/op 71.983 ms/op 1.12
aggregationBits - 2048 els - zipIndexesInBitList 17.318 us/op 15.497 us/op 1.12
regular array get 100000 times 33.429 us/op 32.714 us/op 1.02
wrappedArray get 100000 times 33.436 us/op 43.072 us/op 0.78
arrayWithProxy get 100000 times 15.495 ms/op 15.425 ms/op 1.00
ssz.Root.equals 644.00 ns/op 570.00 ns/op 1.13
byteArrayEquals 562.00 ns/op 529.00 ns/op 1.06
shuffle list - 16384 els 7.0859 ms/op 6.7689 ms/op 1.05
shuffle list - 250000 els 105.43 ms/op 99.875 ms/op 1.06
processSlot - 1 slots 10.063 us/op 8.4910 us/op 1.19
processSlot - 32 slots 1.6754 ms/op 1.3940 ms/op 1.20
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 220.05 us/op 189.68 us/op 1.16
getCommitteeAssignments - req 1 vs - 250000 vc 3.0420 ms/op 2.9080 ms/op 1.05
getCommitteeAssignments - req 100 vs - 250000 vc 4.3830 ms/op 4.1601 ms/op 1.05
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5822 ms/op 4.4887 ms/op 1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.0200 ns/op 4.3300 ns/op 1.16
state getBlockRootAtSlot - 250000 vs - 7PWei 619.08 ns/op 549.24 ns/op 1.13
computeProposers - vc 250000 10.506 ms/op 10.139 ms/op 1.04
computeEpochShuffling - vc 250000 105.06 ms/op 107.69 ms/op 0.98
getNextSyncCommittee - vc 250000 177.33 ms/op 169.63 ms/op 1.05

by benchmarkbot/action

nflaig
nflaig reviewed Mar 1, 2023
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be helpful to have an explainer in the PR of what the reasoning is behind this change

why did we use BuilderAlways before instead of MaxProfit?

@g11tech
Copy link
Contributor Author

g11tech commented Mar 1, 2023

It would be helpful to have an explainer in the PR of what the reasoning is behind this change

why did we use BuilderAlways before instead of MaxProfit?

making this as a default UX makes more sense for a retail user who has to supply less args 👍

@wemeetagain wemeetagain changed the title Shift builder vs execution default selection strategy to max profit Update default builder selection strategy to max profit Mar 1, 2023
@wemeetagain wemeetagain changed the title Update default builder selection strategy to max profit Update default builder selection strategy to MaxProfit Mar 1, 2023
@g11tech g11tech merged commit 34e72ae into unstable Mar 1, 2023
@g11tech g11tech deleted the g11tech/builder-maxprofit branch March 1, 2023 15:56
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig left review comments

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@g11tech @wemeetagain @nflaig