test: fix the benchmark tests #6160
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nazarhussain
changed the title
nazarhussain
changed the title
|
| Benchmark suite | Current: 0a5afaf | Previous: 5201ac4 | Ratio |
|---|---|---|---|
| isKnown best case - 1 super set check | 921.00 ns/op | 287.00 ns/op | 3.21 |
| isKnown normal case - 2 super set checks | 970.00 ns/op | 287.00 ns/op | 3.38 |
Full benchmark results
| Benchmark suite | Current: 0a5afaf | Previous: 5201ac4 | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 873.31 us/op | 739.80 us/op | 1.18 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 155.08 us/op | 80.114 us/op | 1.94 |
| BLS verify - blst-native | 1.4156 ms/op | 1.2655 ms/op | 1.12 |
| BLS verifyMultipleSignatures 3 - blst-native | 2.7757 ms/op | 2.6535 ms/op | 1.05 |
| BLS verifyMultipleSignatures 8 - blst-native | 6.2216 ms/op | 5.8306 ms/op | 1.07 |
| BLS verifyMultipleSignatures 32 - blst-native | 23.266 ms/op | 21.319 ms/op | 1.09 |
| BLS verifyMultipleSignatures 64 - blst-native | 47.889 ms/op | 42.009 ms/op | 1.14 |
| BLS verifyMultipleSignatures 128 - blst-native | 96.235 ms/op | 83.378 ms/op | 1.15 |
| BLS deserializing 10000 signatures | 975.26 ms/op | 876.05 ms/op | 1.11 |
| BLS deserializing 100000 signatures | 10.883 s/op | 8.8350 s/op | 1.23 |
| BLS verifyMultipleSignatures - same message - 3 - blst-native | 1.9132 ms/op | 1.3712 ms/op | 1.40 |
| BLS verifyMultipleSignatures - same message - 8 - blst-native | 2.1902 ms/op | 1.4199 ms/op | 1.54 |
| BLS verifyMultipleSignatures - same message - 32 - blst-native | 2.9436 ms/op | 2.1971 ms/op | 1.34 |
| BLS verifyMultipleSignatures - same message - 64 - blst-native | 4.4690 ms/op | 3.2284 ms/op | 1.38 |
| BLS verifyMultipleSignatures - same message - 128 - blst-native | 6.3837 ms/op | 6.6548 ms/op | 0.96 |
| BLS aggregatePubkeys 32 - blst-native | 35.037 us/op | 25.238 us/op | 1.39 |
| BLS aggregatePubkeys 128 - blst-native | 130.28 us/op | 92.664 us/op | 1.41 |
| getAttestationsForBlock | 83.966 ms/op | 43.589 ms/op | 1.93 |
| getSlashingsAndExits - default max | 518.27 us/op | ||
| getSlashingsAndExits - 2k | 691.02 us/op | ||
| proposeBlockBody type=full, size=empty | 10.552 ms/op | ||
| isKnown best case - 1 super set check | 921.00 ns/op | 287.00 ns/op | 3.21 |
| isKnown normal case - 2 super set checks | 970.00 ns/op | 287.00 ns/op | 3.38 |
| isKnown worse case - 16 super set checks | 854.00 ns/op | 286.00 ns/op | 2.99 |
| CheckpointStateCache - add get delete | 8.9430 us/op | 5.4090 us/op | 1.65 |
| validate api signedAggregateAndProof - struct | 3.1797 ms/op | 2.6343 ms/op | 1.21 |
| validate gossip signedAggregateAndProof - struct | 3.7086 ms/op | 2.6328 ms/op | 1.41 |
| validate gossip attestation - vc 640000 | 1.6057 ms/op | 1.3189 ms/op | 1.22 |
| batch validate gossip attestation - vc 640000 - chunk 32 | 206.36 us/op | 156.35 us/op | 1.32 |
| batch validate gossip attestation - vc 640000 - chunk 64 | 180.50 us/op | 137.84 us/op | 1.31 |
| batch validate gossip attestation - vc 640000 - chunk 128 | 144.45 us/op | 126.79 us/op | 1.14 |
| batch validate gossip attestation - vc 640000 - chunk 256 | 133.49 us/op | 123.46 us/op | 1.08 |
| pickEth1Vote - no votes | 1.3308 ms/op | 1.1025 ms/op | 1.21 |
| pickEth1Vote - max votes | 10.050 ms/op | 8.9987 ms/op | 1.12 |
| pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 16.078 ms/op | 22.652 ms/op | 0.71 |
| pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 23.761 ms/op | 30.686 ms/op | 0.77 |
| pickEth1Vote - Eth1Data fastSerialize value x2048 | 737.58 us/op | 548.74 us/op | 1.34 |
| pickEth1Vote - Eth1Data fastSerialize tree x2048 | 5.3079 ms/op | 6.2145 ms/op | 0.85 |
| bytes32 toHexString | 571.00 ns/op | 461.00 ns/op | 1.24 |
| bytes32 Buffer.toString(hex) | 308.00 ns/op | 271.00 ns/op | 1.14 |
| bytes32 Buffer.toString(hex) from Uint8Array | 527.00 ns/op | 419.00 ns/op | 1.26 |
| bytes32 Buffer.toString(hex) + 0x | 320.00 ns/op | 275.00 ns/op | 1.16 |
| Object access 1 prop | 0.17100 ns/op | 0.15900 ns/op | 1.08 |
| Map access 1 prop | 0.14900 ns/op | 0.14400 ns/op | 1.03 |
| Object get x1000 | 7.5910 ns/op | 6.9940 ns/op | 1.09 |
| Map get x1000 | 0.75400 ns/op | 0.73000 ns/op | 1.03 |
| Object set x1000 | 53.078 ns/op | 48.248 ns/op | 1.10 |
| Map set x1000 | 40.293 ns/op | 37.520 ns/op | 1.07 |
| Return object 10000 times | 0.24050 ns/op | 0.23110 ns/op | 1.04 |
| Throw Error 10000 times | 4.1909 us/op | 3.7598 us/op | 1.11 |
| fastMsgIdFn sha256 / 200 bytes | 3.3030 us/op | 3.1640 us/op | 1.04 |
| fastMsgIdFn h32 xxhash / 200 bytes | 321.00 ns/op | 262.00 ns/op | 1.23 |
| fastMsgIdFn h64 xxhash / 200 bytes | 374.00 ns/op | 325.00 ns/op | 1.15 |
| fastMsgIdFn sha256 / 1000 bytes | 11.690 us/op | 10.905 us/op | 1.07 |
| fastMsgIdFn h32 xxhash / 1000 bytes | 427.00 ns/op | 392.00 ns/op | 1.09 |
| fastMsgIdFn h64 xxhash / 1000 bytes | 443.00 ns/op | 390.00 ns/op | 1.14 |
| fastMsgIdFn sha256 / 10000 bytes | 105.46 us/op | 99.572 us/op | 1.06 |
| fastMsgIdFn h32 xxhash / 10000 bytes | 2.0260 us/op | 1.8790 us/op | 1.08 |
| fastMsgIdFn h64 xxhash / 10000 bytes | 1.4400 us/op | 1.2940 us/op | 1.11 |
| send data - 1000 256B messages | 21.044 ms/op | 17.695 ms/op | 1.19 |
| send data - 1000 512B messages | 29.977 ms/op | 24.884 ms/op | 1.20 |
| send data - 1000 1024B messages | 46.377 ms/op | 39.230 ms/op | 1.18 |
| send data - 1000 1200B messages | 41.164 ms/op | 35.221 ms/op | 1.17 |
| send data - 1000 2048B messages | 44.767 ms/op | 42.159 ms/op | 1.06 |
| send data - 1000 4096B messages | 39.283 ms/op | 40.787 ms/op | 0.96 |
| send data - 1000 16384B messages | 116.00 ms/op | 107.85 ms/op | 1.08 |
| send data - 1000 65536B messages | 495.96 ms/op | 392.78 ms/op | 1.26 |
| enrSubnets - fastDeserialize 64 bits | 1.2430 us/op | 1.2440 us/op | 1.00 |
| enrSubnets - ssz BitVector 64 bits | 421.00 ns/op | 402.00 ns/op | 1.05 |
| enrSubnets - fastDeserialize 4 bits | 177.00 ns/op | 154.00 ns/op | 1.15 |
| enrSubnets - ssz BitVector 4 bits | 418.00 ns/op | 402.00 ns/op | 1.04 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 102.34 us/op | 99.907 us/op | 1.02 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 130.49 us/op | 117.11 us/op | 1.11 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 168.37 us/op | 159.60 us/op | 1.05 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 297.93 us/op | 317.38 us/op | 0.94 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 345.40 us/op | 333.56 us/op | 1.04 |
| array of 16000 items push then shift | 1.6017 us/op | 1.5818 us/op | 1.01 |
| LinkedList of 16000 items push then shift | 8.5570 ns/op | 8.7610 ns/op | 0.98 |
| array of 16000 items push then pop | 91.069 ns/op | 82.861 ns/op | 1.10 |
| LinkedList of 16000 items push then pop | 8.6720 ns/op | 8.5250 ns/op | 1.02 |
| array of 24000 items push then shift | 2.4162 us/op | 2.3338 us/op | 1.04 |
| LinkedList of 24000 items push then shift | 8.9620 ns/op | 8.7430 ns/op | 1.03 |
| array of 24000 items push then pop | 117.22 ns/op | 102.43 ns/op | 1.14 |
| LinkedList of 24000 items push then pop | 8.6150 ns/op | 8.4490 ns/op | 1.02 |
| intersect bitArray bitLen 8 | 6.2910 ns/op | 6.2420 ns/op | 1.01 |
| intersect array and set length 8 | 61.546 ns/op | 59.985 ns/op | 1.03 |
| intersect bitArray bitLen 128 | 33.777 ns/op | 33.098 ns/op | 1.02 |
| intersect array and set length 128 | 839.42 ns/op | 829.06 ns/op | 1.01 |
| bitArray.getTrueBitIndexes() bitLen 128 | 1.4130 us/op | 1.3580 us/op | 1.04 |
| bitArray.getTrueBitIndexes() bitLen 248 | 2.4490 us/op | 2.3270 us/op | 1.05 |
| bitArray.getTrueBitIndexes() bitLen 512 | 4.7130 us/op | 4.5050 us/op | 1.05 |
| Buffer.concat 32 items | 904.00 ns/op | 888.00 ns/op | 1.02 |
| Uint8Array.set 32 items | 1.5860 us/op | 1.5930 us/op | 1.00 |
| Set add up to 64 items then delete first | 4.3287 us/op | 4.2085 us/op | 1.03 |
| OrderedSet add up to 64 items then delete first | 5.4207 us/op | 5.2836 us/op | 1.03 |
| Set add up to 64 items then delete last | 4.8898 us/op | 4.5890 us/op | 1.07 |
| OrderedSet add up to 64 items then delete last | 5.8093 us/op | 5.6447 us/op | 1.03 |
| Set add up to 64 items then delete middle | 4.6632 us/op | 4.4479 us/op | 1.05 |
| OrderedSet add up to 64 items then delete middle | 7.1644 us/op | 6.8222 us/op | 1.05 |
| Set add up to 128 items then delete first | 9.5833 us/op | 9.1799 us/op | 1.04 |
| OrderedSet add up to 128 items then delete first | 13.179 us/op | 12.150 us/op | 1.08 |
| Set add up to 128 items then delete last | 9.1992 us/op | 9.1242 us/op | 1.01 |
| OrderedSet add up to 128 items then delete last | 11.697 us/op | 11.377 us/op | 1.03 |
| Set add up to 128 items then delete middle | 9.2193 us/op | 8.8459 us/op | 1.04 |
| OrderedSet add up to 128 items then delete middle | 17.127 us/op | 16.465 us/op | 1.04 |
| Set add up to 256 items then delete first | 18.756 us/op | 18.746 us/op | 1.00 |
| OrderedSet add up to 256 items then delete first | 25.502 us/op | 24.950 us/op | 1.02 |
| Set add up to 256 items then delete last | 18.604 us/op | 18.094 us/op | 1.03 |
| OrderedSet add up to 256 items then delete last | 23.772 us/op | 22.901 us/op | 1.04 |
| Set add up to 256 items then delete middle | 18.436 us/op | 17.600 us/op | 1.05 |
| OrderedSet add up to 256 items then delete middle | 45.371 us/op | 43.877 us/op | 1.03 |
| transfer serialized Status (84 B) | 1.7760 us/op | 1.6920 us/op | 1.05 |
| copy serialized Status (84 B) | 1.4640 us/op | 1.4580 us/op | 1.00 |
| transfer serialized SignedVoluntaryExit (112 B) | 1.8750 us/op | 2.0590 us/op | 0.91 |
| copy serialized SignedVoluntaryExit (112 B) | 1.5160 us/op | 1.7370 us/op | 0.87 |
| transfer serialized ProposerSlashing (416 B) | 1.9730 us/op | 2.8520 us/op | 0.69 |
| copy serialized ProposerSlashing (416 B) | 1.7660 us/op | 2.6770 us/op | 0.66 |
| transfer serialized Attestation (485 B) | 2.0160 us/op | 2.6390 us/op | 0.76 |
| copy serialized Attestation (485 B) | 1.7820 us/op | 2.9840 us/op | 0.60 |
| transfer serialized AttesterSlashing (33232 B) | 2.2630 us/op | 3.1810 us/op | 0.71 |
| copy serialized AttesterSlashing (33232 B) | 5.1940 us/op | 6.2910 us/op | 0.83 |
| transfer serialized Small SignedBeaconBlock (128000 B) | 2.5010 us/op | 2.8860 us/op | 0.87 |
| copy serialized Small SignedBeaconBlock (128000 B) | 14.465 us/op | 16.054 us/op | 0.90 |
| transfer serialized Avg SignedBeaconBlock (200000 B) | 3.0440 us/op | 3.2270 us/op | 0.94 |
| copy serialized Avg SignedBeaconBlock (200000 B) | 20.722 us/op | 22.421 us/op | 0.92 |
| transfer serialized BlobsSidecar (524380 B) | 2.9640 us/op | 2.8530 us/op | 1.04 |
| copy serialized BlobsSidecar (524380 B) | 85.052 us/op | 88.633 us/op | 0.96 |
| transfer serialized Big SignedBeaconBlock (1000000 B) | 2.9740 us/op | 3.1240 us/op | 0.95 |
| copy serialized Big SignedBeaconBlock (1000000 B) | 166.62 us/op | 161.75 us/op | 1.03 |
| pass gossip attestations to forkchoice per slot | 4.2464 ms/op | 4.0058 ms/op | 1.06 |
| forkChoice updateHead vc 100000 bc 64 eq 0 | 693.33 us/op | 719.43 us/op | 0.96 |
| forkChoice updateHead vc 600000 bc 64 eq 0 | 5.0236 ms/op | 5.1202 ms/op | 0.98 |
| forkChoice updateHead vc 1000000 bc 64 eq 0 | 7.0751 ms/op | 6.8886 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 320 eq 0 | 4.2346 ms/op | 3.9607 ms/op | 1.07 |
| forkChoice updateHead vc 600000 bc 1200 eq 0 | 4.2756 ms/op | 4.0298 ms/op | 1.06 |
| forkChoice updateHead vc 600000 bc 7200 eq 0 | 5.2515 ms/op | 4.9291 ms/op | 1.07 |
| forkChoice updateHead vc 600000 bc 64 eq 1000 | 11.071 ms/op | 10.471 ms/op | 1.06 |
| forkChoice updateHead vc 600000 bc 64 eq 10000 | 11.805 ms/op | 11.242 ms/op | 1.05 |
| forkChoice updateHead vc 600000 bc 64 eq 300000 | 15.656 ms/op | 14.930 ms/op | 1.05 |
| computeDeltas 500000 validators 300 proto nodes | 6.6165 ms/op | 6.4080 ms/op | 1.03 |
| computeDeltas 500000 validators 1200 proto nodes | 6.6912 ms/op | 6.5627 ms/op | 1.02 |
| computeDeltas 500000 validators 7200 proto nodes | 6.5125 ms/op | 6.2924 ms/op | 1.03 |
| computeDeltas 750000 validators 300 proto nodes | 9.8448 ms/op | 9.5432 ms/op | 1.03 |
| computeDeltas 750000 validators 1200 proto nodes | 10.071 ms/op | 9.5263 ms/op | 1.06 |
| computeDeltas 750000 validators 7200 proto nodes | 9.9916 ms/op | 9.5998 ms/op | 1.04 |
| computeDeltas 1400000 validators 300 proto nodes | 18.507 ms/op | 17.934 ms/op | 1.03 |
| computeDeltas 1400000 validators 1200 proto nodes | 19.495 ms/op | 17.833 ms/op | 1.09 |
| computeDeltas 1400000 validators 7200 proto nodes | 19.340 ms/op | 17.630 ms/op | 1.10 |
| computeDeltas 2100000 validators 300 proto nodes | 28.971 ms/op | 27.325 ms/op | 1.06 |
| computeDeltas 2100000 validators 1200 proto nodes | 29.541 ms/op | 27.603 ms/op | 1.07 |
| computeDeltas 2100000 validators 7200 proto nodes | 28.341 ms/op | 28.509 ms/op | 0.99 |
| computeProposerBoostScoreFromBalances 500000 validators | 3.7433 ms/op | 3.5905 ms/op | 1.04 |
| computeProposerBoostScoreFromBalances 750000 validators | 3.7370 ms/op | 3.6214 ms/op | 1.03 |
| computeProposerBoostScoreFromBalances 1400000 validators | 3.7294 ms/op | 3.6578 ms/op | 1.02 |
| computeProposerBoostScoreFromBalances 2100000 validators | 3.7158 ms/op | 3.6559 ms/op | 1.02 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 2.2931 ms/op | 2.2217 ms/op | 1.03 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 3.4806 ms/op | 3.2264 ms/op | 1.08 |
| altair processAttestation - setStatus - 1/6 committees join | 179.88 us/op | 171.14 us/op | 1.05 |
| altair processAttestation - setStatus - 1/3 committees join | 361.52 us/op | 332.49 us/op | 1.09 |
| altair processAttestation - setStatus - 1/2 committees join | 477.37 us/op | 446.79 us/op | 1.07 |
| altair processAttestation - setStatus - 2/3 committees join | 604.95 us/op | 550.07 us/op | 1.10 |
| altair processAttestation - setStatus - 4/5 committees join | 804.75 us/op | 770.34 us/op | 1.04 |
| altair processAttestation - setStatus - 100% committees join | 964.47 us/op | 870.17 us/op | 1.11 |
| altair processBlock - 250000 vs - 7PWei normalcase | 12.752 ms/op | 10.071 ms/op | 1.27 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 39.089 ms/op | 36.180 ms/op | 1.08 |
| altair processBlock - 250000 vs - 7PWei worstcase | 41.300 ms/op | 35.483 ms/op | 1.16 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 89.587 ms/op | 92.005 ms/op | 0.97 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 3.0840 ms/op | 3.4608 ms/op | 0.89 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 30.994 ms/op | 31.258 ms/op | 0.99 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 525.76 us/op | 539.71 us/op | 0.97 |
| getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 | 12.090 us/op | 14.236 us/op | 0.85 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 58.058 us/op | 94.485 us/op | 0.61 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 | 27.093 us/op | 16.762 us/op | 1.62 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 | 11.021 us/op | 17.887 us/op | 0.62 |
| getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 | 200.46 us/op | 212.96 us/op | 0.94 |
| getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 | 1.5742 ms/op | 2.0891 ms/op | 0.75 |
| getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 2.1608 ms/op | 2.8574 ms/op | 0.76 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 1.8104 ms/op | 2.2329 ms/op | 0.81 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 4.0855 ms/op | 3.3195 ms/op | 1.23 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.5053 ms/op | 2.1714 ms/op | 1.15 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 6.4040 ms/op | 4.6929 ms/op | 1.36 |
| Tree 40 250000 create | 414.92 ms/op | 347.23 ms/op | 1.19 |
| Tree 40 250000 get(125000) | 202.19 ns/op | 189.35 ns/op | 1.07 |
| Tree 40 250000 set(125000) | 1.0595 us/op | 970.46 ns/op | 1.09 |
| Tree 40 250000 toArray() | 22.398 ms/op | 18.870 ms/op | 1.19 |
| Tree 40 250000 iterate all - toArray() + loop | 25.347 ms/op | 18.874 ms/op | 1.34 |
| Tree 40 250000 iterate all - get(i) | 78.097 ms/op | 66.009 ms/op | 1.18 |
| MutableVector 250000 create | 16.411 ms/op | 16.656 ms/op | 0.99 |
| MutableVector 250000 get(125000) | 6.7180 ns/op | 6.3550 ns/op | 1.06 |
| MutableVector 250000 set(125000) | 347.06 ns/op | 252.79 ns/op | 1.37 |
| MutableVector 250000 toArray() | 4.7855 ms/op | 3.2358 ms/op | 1.48 |
| MutableVector 250000 iterate all - toArray() + loop | 4.6422 ms/op | 3.2470 ms/op | 1.43 |
| MutableVector 250000 iterate all - get(i) | 1.5739 ms/op | 1.4793 ms/op | 1.06 |
| Array 250000 create | 3.0683 ms/op | 2.7956 ms/op | 1.10 |
| Array 250000 clone - spread | 1.2956 ms/op | 1.2731 ms/op | 1.02 |
| Array 250000 get(125000) | 1.0900 ns/op | 1.0200 ns/op | 1.07 |
| Array 250000 set(125000) | 4.1700 ns/op | 4.0770 ns/op | 1.02 |
| Array 250000 iterate all - loop | 166.75 us/op | 161.92 us/op | 1.03 |
| effectiveBalanceIncrements clone Uint8Array 300000 | 32.304 us/op | 26.647 us/op | 1.21 |
| effectiveBalanceIncrements clone MutableVector 300000 | 399.00 ns/op | 366.00 ns/op | 1.09 |
| effectiveBalanceIncrements rw all Uint8Array 300000 | 201.14 us/op | 194.48 us/op | 1.03 |
| effectiveBalanceIncrements rw all MutableVector 300000 | 89.156 ms/op | 82.126 ms/op | 1.09 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 118.41 ms/op | 109.44 ms/op | 1.08 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 55.304 ms/op | 53.800 ms/op | 1.03 |
| altair processEpoch - mainnet_e81889 | 519.64 ms/op | 519.62 ms/op | 1.00 |
| mainnet_e81889 - altair beforeProcessEpoch | 86.962 ms/op | 78.556 ms/op | 1.11 |
| mainnet_e81889 - altair processJustificationAndFinalization | 17.862 us/op | 22.938 us/op | 0.78 |
| mainnet_e81889 - altair processInactivityUpdates | 6.3264 ms/op | 5.4911 ms/op | 1.15 |
| mainnet_e81889 - altair processRewardsAndPenalties | 64.253 ms/op | 72.962 ms/op | 0.88 |
| mainnet_e81889 - altair processRegistryUpdates | 3.5730 us/op | 3.9670 us/op | 0.90 |
| mainnet_e81889 - altair processSlashings | 565.00 ns/op | 717.00 ns/op | 0.79 |
| mainnet_e81889 - altair processEth1DataReset | 584.00 ns/op | 766.00 ns/op | 0.76 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.4397 ms/op | 2.5904 ms/op | 0.56 |
| mainnet_e81889 - altair processSlashingsReset | 4.3390 us/op | 6.2860 us/op | 0.69 |
| mainnet_e81889 - altair processRandaoMixesReset | 5.4400 us/op | 8.1360 us/op | 0.67 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 2.6650 us/op | 916.00 ns/op | 2.91 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 2.2450 us/op | 3.8570 us/op | 0.58 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 1.0230 us/op | 1.0630 us/op | 0.96 |
| mainnet_e81889 - altair afterProcessEpoch | 121.78 ms/op | 116.83 ms/op | 1.04 |
| capella processEpoch - mainnet_e217614 | 2.4898 s/op | 2.2645 s/op | 1.10 |
| mainnet_e217614 - capella beforeProcessEpoch | 574.81 ms/op | 452.75 ms/op | 1.27 |
| mainnet_e217614 - capella processJustificationAndFinalization | 47.531 us/op | 18.230 us/op | 2.61 |
| mainnet_e217614 - capella processInactivityUpdates | 43.883 ms/op | 19.164 ms/op | 2.29 |
| mainnet_e217614 - capella processRewardsAndPenalties | 623.91 ms/op | 412.86 ms/op | 1.51 |
| mainnet_e217614 - capella processRegistryUpdates | 35.926 us/op | 28.128 us/op | 1.28 |
| mainnet_e217614 - capella processSlashings | 1.2520 us/op | 824.00 ns/op | 1.52 |
| mainnet_e217614 - capella processEth1DataReset | 1.2470 us/op | 720.00 ns/op | 1.73 |
| mainnet_e217614 - capella processEffectiveBalanceUpdates | 16.629 ms/op | 4.9102 ms/op | 3.39 |
| mainnet_e217614 - capella processSlashingsReset | 8.8950 us/op | 3.9550 us/op | 2.25 |
| mainnet_e217614 - capella processRandaoMixesReset | 11.817 us/op | 6.9890 us/op | 1.69 |
| mainnet_e217614 - capella processHistoricalRootsUpdate | 1.5420 us/op | 979.00 ns/op | 1.58 |
| mainnet_e217614 - capella processParticipationFlagUpdates | 6.6220 us/op | 3.3360 us/op | 1.99 |
| mainnet_e217614 - capella afterProcessEpoch | 383.09 ms/op | 286.22 ms/op | 1.34 |
| phase0 processEpoch - mainnet_e58758 | 729.42 ms/op | 471.45 ms/op | 1.55 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 279.98 ms/op | 151.88 ms/op | 1.84 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 38.221 us/op | 25.387 us/op | 1.51 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 88.556 ms/op | 60.222 ms/op | 1.47 |
| mainnet_e58758 - phase0 processRegistryUpdates | 27.869 us/op | 14.165 us/op | 1.97 |
| mainnet_e58758 - phase0 processSlashings | 1.3530 us/op | 861.00 ns/op | 1.57 |
| mainnet_e58758 - phase0 processEth1DataReset | 1.1840 us/op | 657.00 ns/op | 1.80 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 1.7413 ms/op | 2.1616 ms/op | 0.81 |
| mainnet_e58758 - phase0 processSlashingsReset | 6.6720 us/op | 4.3730 us/op | 1.53 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 9.0840 us/op | 6.3850 us/op | 1.42 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 854.00 ns/op | 653.00 ns/op | 1.31 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 6.1410 us/op | 4.9200 us/op | 1.25 |
| mainnet_e58758 - phase0 afterProcessEpoch | 102.37 ms/op | 94.514 ms/op | 1.08 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.4161 ms/op | 2.4886 ms/op | 0.57 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 1.7766 ms/op | 2.6895 ms/op | 0.66 |
| altair processInactivityUpdates - 250000 normalcase | 28.814 ms/op | 27.062 ms/op | 1.06 |
| altair processInactivityUpdates - 250000 worstcase | 26.191 ms/op | 31.394 ms/op | 0.83 |
| phase0 processRegistryUpdates - 250000 normalcase | 9.6300 us/op | 11.032 us/op | 0.87 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 404.60 us/op | 617.43 us/op | 0.66 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 130.91 ms/op | 122.35 ms/op | 1.07 |
| altair processRewardsAndPenalties - 250000 normalcase | 62.464 ms/op | 65.473 ms/op | 0.95 |
| altair processRewardsAndPenalties - 250000 worstcase | 63.701 ms/op | 63.254 ms/op | 1.01 |
| phase0 getAttestationDeltas - 250000 normalcase | 8.9502 ms/op | 8.4964 ms/op | 1.05 |
| phase0 getAttestationDeltas - 250000 worstcase | 11.754 ms/op | 8.5149 ms/op | 1.38 |
| phase0 processSlashings - 250000 worstcase | 110.65 us/op | 132.70 us/op | 0.83 |
| altair processSyncCommitteeUpdates - 250000 | 169.32 ms/op | 156.13 ms/op | 1.08 |
| BeaconState.hashTreeRoot - No change | 259.00 ns/op | 232.00 ns/op | 1.12 |
| BeaconState.hashTreeRoot - 1 full validator | 191.75 us/op | 143.72 us/op | 1.33 |
| BeaconState.hashTreeRoot - 32 full validator | 1.7966 ms/op | 1.6988 ms/op | 1.06 |
| BeaconState.hashTreeRoot - 512 full validator | 19.265 ms/op | 18.722 ms/op | 1.03 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 182.28 us/op | 209.62 us/op | 0.87 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 2.4377 ms/op | 2.9605 ms/op | 0.82 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 41.137 ms/op | 40.666 ms/op | 1.01 |
| BeaconState.hashTreeRoot - 1 balances | 164.79 us/op | 172.29 us/op | 0.96 |
| BeaconState.hashTreeRoot - 32 balances | 1.6334 ms/op | 1.5157 ms/op | 1.08 |
| BeaconState.hashTreeRoot - 512 balances | 12.996 ms/op | 13.488 ms/op | 0.96 |
| BeaconState.hashTreeRoot - 250000 balances | 223.94 ms/op | 226.13 ms/op | 0.99 |
| aggregationBits - 2048 els - zipIndexesInBitList | 26.450 us/op | 16.840 us/op | 1.57 |
| byteArrayEquals 32 | 80.956 ns/op | 71.868 ns/op | 1.13 |
| Buffer.compare 32 | 56.266 ns/op | 53.054 ns/op | 1.06 |
| byteArrayEquals 1024 | 2.0899 us/op | 1.9609 us/op | 1.07 |
| Buffer.compare 1024 | 72.049 ns/op | 67.485 ns/op | 1.07 |
| byteArrayEquals 16384 | 33.028 us/op | 31.237 us/op | 1.06 |
| Buffer.compare 16384 | 270.07 ns/op | 261.56 ns/op | 1.03 |
| byteArrayEquals 123687377 | 250.95 ms/op | 242.66 ms/op | 1.03 |
| Buffer.compare 123687377 | 7.4373 ms/op | 6.1094 ms/op | 1.22 |
| byteArrayEquals 32 - diff last byte | 73.291 ns/op | 71.367 ns/op | 1.03 |
| Buffer.compare 32 - diff last byte | 56.513 ns/op | 58.118 ns/op | 0.97 |
| byteArrayEquals 1024 - diff last byte | 2.0096 us/op | 1.9693 us/op | 1.02 |
| Buffer.compare 1024 - diff last byte | 72.167 ns/op | 68.669 ns/op | 1.05 |
| byteArrayEquals 16384 - diff last byte | 32.551 us/op | 31.432 us/op | 1.04 |
| Buffer.compare 16384 - diff last byte | 289.09 ns/op | 249.55 ns/op | 1.16 |
| byteArrayEquals 123687377 - diff last byte | 244.58 ms/op | 230.83 ms/op | 1.06 |
| Buffer.compare 123687377 - diff last byte | 7.4797 ms/op | 6.0457 ms/op | 1.24 |
| byteArrayEquals 32 - random bytes | 5.5180 ns/op | 4.9260 ns/op | 1.12 |
| Buffer.compare 32 - random bytes | 60.578 ns/op | 56.810 ns/op | 1.07 |
| byteArrayEquals 1024 - random bytes | 5.1810 ns/op | 4.8440 ns/op | 1.07 |
| Buffer.compare 1024 - random bytes | 61.984 ns/op | 56.289 ns/op | 1.10 |
| byteArrayEquals 16384 - random bytes | 5.3000 ns/op | 4.8370 ns/op | 1.10 |
| Buffer.compare 16384 - random bytes | 60.683 ns/op | 56.641 ns/op | 1.07 |
| byteArrayEquals 123687377 - random bytes | 8.8100 ns/op | 7.6600 ns/op | 1.15 |
| Buffer.compare 123687377 - random bytes | 66.740 ns/op | 59.080 ns/op | 1.13 |
| regular array get 100000 times | 45.331 us/op | 41.319 us/op | 1.10 |
| wrappedArray get 100000 times | 45.360 us/op | 41.412 us/op | 1.10 |
| arrayWithProxy get 100000 times | 14.304 ms/op | 14.652 ms/op | 0.98 |
| ssz.Root.equals | 54.391 ns/op | 53.055 ns/op | 1.03 |
| byteArrayEquals | 53.492 ns/op | 52.170 ns/op | 1.03 |
| Buffer.compare | 11.726 ns/op | 10.652 ns/op | 1.10 |
| shuffle list - 16384 els | 7.0704 ms/op | 6.7463 ms/op | 1.05 |
| shuffle list - 250000 els | 104.26 ms/op | 99.956 ms/op | 1.04 |
| processSlot - 1 slots | 17.696 us/op | 18.299 us/op | 0.97 |
| processSlot - 32 slots | 3.2935 ms/op | 3.3333 ms/op | 0.99 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 61.034 ms/op | 59.612 ms/op | 1.02 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 2.4808 ms/op | 2.4814 ms/op | 1.00 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 3.6522 ms/op | 3.6492 ms/op | 1.00 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 4.0016 ms/op | 3.9909 ms/op | 1.00 |
| findModifiedValidators - 10000 modified validators | 525.84 ms/op | 542.18 ms/op | 0.97 |
| findModifiedValidators - 1000 modified validators | 430.33 ms/op | 412.23 ms/op | 1.04 |
| findModifiedValidators - 100 modified validators | 402.87 ms/op | 352.91 ms/op | 1.14 |
| findModifiedValidators - 10 modified validators | 392.86 ms/op | 385.94 ms/op | 1.02 |
| findModifiedValidators - 1 modified validators | 404.15 ms/op | 397.51 ms/op | 1.02 |
| findModifiedValidators - no difference | 414.89 ms/op | 416.14 ms/op | 1.00 |
| compare ViewDUs | 4.4441 s/op | 5.0722 s/op | 0.88 |
| compare each validator Uint8Array | 1.2632 s/op | 1.5912 s/op | 0.79 |
| compare ViewDU to Uint8Array | 1.1563 s/op | 1.0794 s/op | 1.07 |
| migrate state 1000000 validators, 24 modified, 0 new | 775.43 ms/op | 897.63 ms/op | 0.86 |
| migrate state 1000000 validators, 1700 modified, 1000 new | 1.0990 s/op | 1.1896 s/op | 0.92 |
| migrate state 1000000 validators, 3400 modified, 2000 new | 1.4701 s/op | 1.4153 s/op | 1.04 |
| migrate state 1500000 validators, 24 modified, 0 new | 948.40 ms/op | 898.74 ms/op | 1.06 |
| migrate state 1500000 validators, 1700 modified, 1000 new | 1.1946 s/op | 1.1975 s/op | 1.00 |
| migrate state 1500000 validators, 3400 modified, 2000 new | 1.3322 s/op | 1.3649 s/op | 0.98 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 4.2500 ns/op | 4.0200 ns/op | 1.06 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 540.69 ns/op | 926.66 ns/op | 0.58 |
| computeProposers - vc 250000 | 9.0991 ms/op | 8.5250 ms/op | 1.07 |
| computeEpochShuffling - vc 250000 | 103.37 ms/op | 100.77 ms/op | 1.03 |
| getNextSyncCommittee - vc 250000 | 148.69 ms/op | 147.26 ms/op | 1.01 |
| computeSigningRoot for AttestationData | 22.672 us/op | 27.681 us/op | 0.82 |
| hash AttestationData serialized data then Buffer.toString(base64) | 2.3094 us/op | 2.2381 us/op | 1.03 |
| toHexString serialized data | 1.0818 us/op | 1.0545 us/op | 1.03 |
| Buffer.toString(base64) | 215.21 ns/op | 212.35 ns/op | 1.01 |
by benchmarkbot/action
nazarhussain
changed the title
twoeths
reviewed
Dec 6, 2023
| @@ -17,6 +17,7 @@ on: | |||
| branches: | |||
| - stable | |||
| - unstable | |||
| workflow_dispatch: | |||
There was a problem hiding this comment.
I think it is just to be able to trigger it manually, might be useful to have
There was a problem hiding this comment.
In some discussions @dapplion suggested to open draft PRs to test some idea. For that purpose it's required sometime to run some workflows manually on branches.
nflaig
approved these changes
Dec 6, 2023
| @@ -17,6 +17,7 @@ on: | |||
| branches: | |||
| - stable | |||
| - unstable | |||
| workflow_dispatch: | |||
There was a problem hiding this comment.
I think it is just to be able to trigger it manually, might be useful to have
jeluard
pushed a commit
that referenced
this pull request
Dec 7, 2023
* Update the number of validators * Run the benchmark workflow manually * Add comments explaining the function side effects
|
🎉 This PR is included in v1.13.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Fix benchmark workflow.
Description
Closes #issue_number
Steps to test or reproduce