Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add eth_multicall , support array of eth_call for simulation across multiple blocks #383

Closed
wants to merge 76 commits into from
Closed
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
5556499
Add eth_multicall
epheph Feb 20, 2023
95822af
Remove chain-id from call-type (no replay protection necessary for ca…
epheph Feb 20, 2023
602d2bf
Merge remote-tracking branch 'origin/main' into eth_multicall
epheph Mar 27, 2023
6ce85e2
Add tests
epheph Mar 28, 2023
2fcbf1d
Add test error case
epheph Mar 28, 2023
046502a
Fix account override adding address to target
epheph Mar 30, 2023
e4ec926
Add example resetting WETH to 1ETH totalSupply and assigning that 1ET…
epheph Mar 30, 2023
0fbdf40
Add another account delta
epheph Mar 30, 2023
ccfc70b
Add example resetting WETH to 1ETH totalSupply and assigning that 1ET…
epheph Apr 3, 2023
7590b80
block override with simple contract that just returns block number
epheph Apr 5, 2023
bbf34e8
Change response to keep the block-based structure similar to request …
epheph May 11, 2023
c147f92
Update src/schemas/execute.yaml
epheph Jun 6, 2023
23ef5b8
Update src/schemas/execute.yaml
epheph Jun 6, 2023
73897a7
Update src/schemas/execute.yaml
epheph Jun 6, 2023
152cf09
Update src/schemas/execute.yaml
epheph Jun 6, 2023
c8abc82
Get rid of version, consider standard solution in #408 for the future
epheph Jun 8, 2023
14e9b35
Even though it's only 1 block you are overriding in this context, you…
epheph Jun 8, 2023
94786e2
Addressing feedback related to return being reserved word
epheph Jun 8, 2023
4b63f49
if no logs, empty array.
epheph Jun 8, 2023
dc38150
Most restrictive data type for blockNumber i could find elsewhere
epheph Jun 8, 2023
9463ba6
Most restrictive data type for gas I could find elsewhere in repo
epheph Jun 8, 2023
51fad35
Apply suggestions from code review
epheph Jun 8, 2023
1802812
Update src/eth/execute.yaml
epheph Jun 8, 2023
6bd8e50
Update src/eth/execute.yaml
epheph Jun 8, 2023
2cf1b8e
Add Trace Transfers argument
epheph Jun 15, 2023
ff39b4a
Add moveToAddress, a way to displace code or precompile at a specific…
epheph Jun 15, 2023
81e6730
Add tiny documentation inline concerning requirement that block numbe…
epheph Jun 15, 2023
4a42aa2
Apply suggestions from code review
epheph Jun 22, 2023
7d159f8
Merge remote-tracking branch 'origin/main' into eth_multicall
epheph Jul 5, 2023
7787ce8
Using `errors` style, as seen in more modern api error definitions. S…
epheph Jul 5, 2023
d8ca0a4
Merge remote-tracking branch 'df/eth_multicall' into eth_multicall
epheph Jul 5, 2023
90016c0
Update src/schemas/execute.yaml
MicahZoltu Jul 20, 2023
d596116
Update src/schemas/execute.yaml
KillariDev Jul 21, 2023
0f8426a
Removes `int` type.
MicahZoltu Jul 25, 2023
6928d24
Typo: errors -> error
MicahZoltu Jul 25, 2023
9528d89
Fixes error object type for CallResultFailure.
MicahZoltu Jul 25, 2023
c6a3397
Changes BlockOverrides.number to a uint64.
MicahZoltu Jul 25, 2023
b587f7f
Another `errors => error` typo
MicahZoltu Jul 25, 2023
818ef4a
Timestamp to uint64
MicahZoltu Jul 25, 2023
3c7c14e
Timestamp to uint64
MicahZoltu Jul 25, 2023
1814075
Block number to uint64
MicahZoltu Jul 25, 2023
c3999c2
Fixes bug in spec (error was both object and list).
MicahZoltu Jul 25, 2023
1a92538
add multicall object
KillariDev Jul 25, 2023
41213eb
fix error unions
KillariDev Jul 25, 2023
e785349
add descriptions
KillariDev Jul 25, 2023
024958c
fix account override
KillariDev Jul 25, 2023
385791c
use blockStateCalls instead of of "calls"
KillariDev Jul 25, 2023
557038a
Now builds without errors, but fails lint script / schema checks
0xjimmy Jul 26, 2023
e2ae78f
Apply suggestions from code review
MicahZoltu Jul 26, 2023
d1cce31
Update src/schemas/execute.yaml
KillariDev Jul 28, 2023
dfdab76
move errors to root, add couple standard errors, add more clarifying …
KillariDev Jul 31, 2023
2dde369
clarify statediff/state/moveToAddress params
KillariDev Aug 1, 2023
dd0fe15
make StateOverrides a dictionary
KillariDev Aug 3, 2023
6cde1ab
Update execute.yaml
KillariDev Aug 3, 2023
62ab430
update prevRandao, baseFeePerGas and feeRecipient back. Add descripti…
KillariDev Aug 7, 2023
bde1830
change `moveToAddress` to `movePrecompileToAddress`
KillariDev Sep 11, 2023
8d8725a
Add suggested default values for transactions
KillariDev Sep 11, 2023
dc4017b
fix according to comments
KillariDev Oct 11, 2023
a3f89ad
move call invalid errors to top level of multicall
KillariDev Oct 20, 2023
28d995f
multicall notes
KillariDev Oct 24, 2023
50c09b8
update document
KillariDev Oct 24, 2023
be26261
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
0630df2
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
e861a4a
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
cba6d4e
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
2196a29
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
40daadb
Update docs/multicall-notes.md
KillariDev Oct 24, 2023
d831c5f
fix according to comments
KillariDev Oct 25, 2023
995acb2
add new error for gas limit and block number limits
KillariDev Oct 25, 2023
88f1f7b
add failures comment
KillariDev Oct 25, 2023
235c40c
Update docs/multicall-notes.md
KillariDev Oct 30, 2023
e817b74
Update docs/multicall-notes.md
KillariDev Oct 30, 2023
78b3a92
Update docs/multicall-notes.md
KillariDev Oct 30, 2023
cb5d419
Update docs/multicall-notes.md
KillariDev Oct 30, 2023
8d3b4d4
add info about overriding default values and restrictions on them
KillariDev Nov 3, 2023
5f09367
add note
KillariDev Nov 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
216 changes: 216 additions & 0 deletions docs/multicall-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
# Multicall
This document contains some extra information that couldn't be fit to the specification document directly.

## Default block values
Unlike `eth_call`, `eth_multicallV1`'s calls are conducted inside blocks. We don't require user to define all the fields of the blocks so here are the defaults that are assumed for blocks parameters:

| parameter name | default value |
-----------------|-----------------------
| prevRandao | 0x0 |
| feeRecipient | 0x0 |
| mixHash | 0x0 |
| nonce | 0x0 |
| extraData | 0x0 |
KillariDev marked this conversation as resolved.
Show resolved Hide resolved
| difficulty | The same as the base block defined as the second parameter in the call |
| gasLimit | The same as the base block defined as the second parameter in the call |
| hash | calculated normally, except for phantom blocks, see below Phantom block section |
| parentHash | previous blocks hash (the real hash, or phantom blocks hash) |
| timestamp | the timestamp of previous block + 1 |
| baseFeePerGas | calculated on what it should be according to ethereum's spec. Note: Phantom blocks do not have transactions so this is always adjusted downwards on them |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we decided to ignore phantom blocks. I.e. in the base fee calculation the parent is the last simulated or real block.

This comment was marked as resolved.

| sha3Uncles | empty trie root |
| withdrawals | empty array |
| uncles | empty array |
| logsBloom | calculated normally. ETH logs are not part of the calculation |
| receiptsRoot | calculated normally |
| transactionsRoot | calculated normally |
| number | calculated normally |
| size | calculated normally |
| withdrawalsRoot | calculated normally |
| gasUsed | calculated normally |
| stateRoot | calculated normally |

An interesting note here is that we decide timestamp as `previous block timestamp + 1`, while `previous block timestamp + 12` could also be an assumed default. The reasoning to use `+1` is that it's the minimum amount we have to increase the timestamp to keep them valid. While `+12` is what Mainnet uses, there are other chains that use some other values, and we didn't want to complicate the specification to consider all networks.

## Phantom blocks
The multicall allows you to define on what block number your calls or transactions are being executed on. Eg, consider following call:
```json
{
"jsonrpc": "2.0",
"id": 1,
"method": "eth_multicallV1",
"params": [
{
"blockStateCalls": [
{
"blockOverrides": {
"number": "0xb"
},
"calls": [
{
"from": "0xc000000000000000000000000000000000000000",
"input": "0x4360005260206000f3"
}
]
},
{
"blockOverrides": {
"number": "0xc"
},
"calls": [
{
"from": "0xc100000000000000000000000000000000000000",
"input": "0x4360005260206000f3"
}
]
}
]
},
"0xa"
]
}
```

Here we want our calls to be executed in blocks 11 (0xb) and in 12 (0xc). The block numbers can be anything as long as they are increasing and higher than the block we are building from 10 (0xa). So, we could set the blocks to be 100 and 200 for example. Now we end up in a situation where there exists block ranges 13-99 and 101-199 that are not defined anywhere. These blocks are called "phantom blocks". What happens if you try to request block hash of any of such blocks in the EVM? How can we calculate the block hash of future blocks when we don't know the block hash of the previous block?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend just having block numbers 100 and 200 in the JSON example rather than having sequential numbers and then using text to describe how you might have something different.

This comment was marked as resolved.


Our solution to this problem is to define block hash of a phantom block to be:

```
phantom block at block #i = keccac(rlp(hash_of_previous_non_phantom_block, #i))
KillariDev marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like RLP encoding these is unnecessarily complex. Why not just keccak(hash_of_previous_non_phantom_block, phantom_block_number_as_bytes)? Like many hashing algorithms, keccak can hash arbitrary byte array sequences, and you can do it in series with almost all libraries. No need to use RLP to first combine the two byte arrays.

```

So for example in our example, you could get block hash of block #142 as
```
phantom block at block #i = keccac(rlp(hash of block #12, #142))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
phantom block at block #i = keccac(rlp(hash of block #12, #142))
keccak(rlp(hash_of_block_12, 142))
Suggested change
phantom block at block #i = keccac(rlp(hash of block #12, #142))
keccak(hash_of_block_12, toBytes(142))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think solidity's keccac supports that for example?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, you are correct. However you can easily just concatenate the two prior to handing it over to keccak in such cases. Getting an RLP encoder into Solidity is significantly harder than getting a byte concatenator. If we want to optimize for Solidity simplicity, we could change the 142 to be 32-byte padded with zeros as that is the easiest thing to get into memory in Solidity.

```

The phantom blocks other properties are set to their default properties as defined by the multicall specification. We came to this definition by wanting phantom block hashes to be unique if things prior to the phantom block changes, so if tooling is storing block hashes somewhere, they should remain unique if things change in the simulation.

One other approach to this problem would be to really calculate the real block hashes for all the phantom blocks, but this would make generating blocks far in future really expensive, as to generate 100 phantom blocks, you would need to calculate 100 block hashes that all depend on each other. And in most cases, no one really cares about these blocks.

## Default values for transactions
As multicall is an extension to `eth_call` we want to enable the nice user experience that the user does not need to provide all required values for a transaction. We are assuming following defaults if the variable is not provided by the user:
| parameter name | description |
-----------------|-----------------------
| type | 0x2 |
| nonce | Defaults to correct nonce |
| to | 0x0 |
| from | 0x0 |
KillariDev marked this conversation as resolved.
Show resolved Hide resolved
| gas limit | defaults to gas used amount |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest something like: "block gas limit subtracted by amount of gas used in the block so far"

| value | 0x0 |
| input | no data |
| gasPrice | 0x0 |
| maxPriorityFeePerGas | 0x0 |
| maxFeePerGas | 0x0 |
| accessList | empty array |

## ETH transfer logs
When `traceTransfers` setting is enabled on `eth_multicallV1` The multical will return logs for ethereum transfers along with the normal logs sent by contracts. The ETH transfers are identical to ERC20 transfers, except the "sending contract" is address 0x0.

For example, here's a query that will simply send ether from one address to another (with a state override that gives us the ETH initially):
```json
{
"jsonrpc": "2.0",
"id": 1,
"method": "eth_multicallV1",
"params": [
{
"blockStateCalls": [
{
"stateOverrides": {
"0xc000000000000000000000000000000000000000": {
"balance": "0x7d0"
}
},
"calls": [
{
"from": "0xc000000000000000000000000000000000000000",
"to": "0xc100000000000000000000000000000000000000",
"value": "0x3e8"
}
]
}
],
"traceTransfers": true
},
"latest"
]
}
```

The output of this query is:
```json
{
"jsonrpc": "2.0",
"id": 1,
"result": [
{
"number": "0x4",
"hash": "0x859c932c5cf0dabf8d12eb2518e063966ac1a25e2fc49f1f02574a37f358d0b5",
"timestamp": "0x1f",
"gasLimit": "0x4c4b40",
"gasUsed": "0x5208",
"feeRecipient": "0x0000000000000000000000000000000000000000",
"baseFeePerGas": "0x2310a91d",
"prevRandao": "0x0000000000000000000000000000000000000000000000000000000000000000",
"calls": [
{
"returnData": "0x",
"logs": [
{
"address": "0x0000000000000000000000000000000000000000",
"topics": [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x000000000000000000000000c000000000000000000000000000000000000000",
"0x000000000000000000000000c100000000000000000000000000000000000000"
],
"data": "0x00000000000000000000000000000000000000000000000000000000000003e8",
"blockNumber": "0x4",
"transactionHash": "0xa4d41019e71335f8567e17746b708ddda8b975a9a61f909bd3df55f4866cc913",
"transactionIndex": "0x0",
"blockHash": "0x859c932c5cf0dabf8d12eb2518e063966ac1a25e2fc49f1f02574a37f358d0b5",
"logIndex": "0x0",
"removed": false
}
],
"gasUsed": "0x5208",
"status": "0x1"
}
]
}
]
}
```

Here the interesting part is:
```json
"address": "0x0000000000000000000000000000000000000000",
"topics": [
"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
"0x000000000000000000000000c000000000000000000000000000000000000000",
"0x000000000000000000000000c100000000000000000000000000000000000000"
],
"data": "0x00000000000000000000000000000000000000000000000000000000000003e8",
```
As can be seen, the sending address is the zero address, `0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef` corresponds to signature `Transfer(address,address,uint256)`, `"0x000000000000000000000000c000000000000000000000000000000000000000"` corresponds the sending address and `0x000000000000000000000000c100000000000000000000000000000000000000` is the receiving address. The amount of ETH moved is stored in the `data` field.

The ETH logs will contain following types of ETH transfers:
- Transfering ETH from EOA
- Transfering ETH via contract
- Selfdestructuring contract sending eth
KillariDev marked this conversation as resolved.
Show resolved Hide resolved

But not following ones:
- Gas fees
- Multicalls eth balance override

ETH logs are not part of the calculation for logs bloom filter. Also similarly to normal logs, if the transaction sends eth but the execution reverts, no log gets issued.
KillariDev marked this conversation as resolved.
Show resolved Hide resolved

## Validation
The multicall has a feature to enable or disable validation with setting `Validation`, by default, the validation is off, and the multicall mimics `eth_call` with reduced number of checks. Validation enabled mode is intended to give as close as possible simulation of real EVM block creation, except there's no checks for transaction signatures and we also allow one to send a direct transaction from a contract.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"we also allow one to send a direct transaction from a contract." why do we allow this in the strict mode again? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Validation mode is eth_multicall wide, not per-transaction, and one may have one transaction in the middle of a sequence of transactions that is meant to simulate some contract action, while the rest of the transactions are normal EOA sourced transactions that you want to fully validate.
  • You can't really accidentally call something from a contract, so it is a piece of validation that is unlikely to provide much protection from anything in reality. This is unlike things like gas limits, gas prices, ETH available, and nonce checking which are easy to get wrong especially when user input is involved.


## Version number
The method name for multicall ´eth_multicallV1` the intention is that after release of multicall, if new features are wanted the `eth_multicallV1` is kept as it is, and instead `eth_multicallV2` is published with the new wanted features.
KillariDev marked this conversation as resolved.
Show resolved Hide resolved

## Clients can set their own limits
Clients may introduce their own limits to prevent DOS attacks using the method. We have thought of two such standard limits
- How many blocks can be defined in `BlockStateCalls`. The suggested default for this si 256 blocks
KillariDev marked this conversation as resolved.
Show resolved Hide resolved
- a global gas limit (similar to the same limit for `eth_call`). The multicall cannot exceed the global gas limit over its lifespan
KillariDev marked this conversation as resolved.
Show resolved Hide resolved
16 changes: 16 additions & 0 deletions src/eth/execute.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,19 @@
gasUsed:
title: Gas used
$ref: '#/components/schemas/uint'
- name: eth_multicallV1
summary: Executes a sequence of message calls building on each other's state without creating transactions on the block chain, optionally overriding block and state data
params:
- name: Payload
required: true
schema:
type: '#/components/schemas/MultiCallPayload'
- name: Block tag
required: false
description: "default: 'latest'"
schema:
type: '#/components/schemas/BlockNumberOrTagOrHash'
result:
name: Result of calls
schema:
$ref: '#/components/schemas/MultiCallResult'
Loading