Optimistic sync changes for HTTP API

[paulhauner]

Overview

This issue replaces #2945. I've noticed that my opt sync PR is not granular enough regarding the changes required and it would be nice if clients were consistent (but not a must-have).

If this format turns out to be helpful, I might propose to add it to the opt sync spec.

Table of API changes for optimistic sync

The table lists all [beacon-APIs][https://github.com/ethereum/beacon-APIs/tree/4319e74dd0a17f864fced64cccbff910ca0c9f5e] endpoints. All the changes are described in more detail later in the document.

Path	Method	Change?
/eth/v1/beacon/genesis	GET	No change.
/eth/v1/beacon/states/{state_id}/root	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/fork	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/finality_checkpoints	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/validators	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/validators/{validator_id}	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/validator_balances	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/committees	GET	Change: block_id or state_id
/eth/v1/beacon/states/{state_id}/sync_committees	GET	Change: block_id or state_id
/eth/v1/beacon/headers	GET	Change: headers
/eth/v1/beacon/headers/{block_id}	GET	Change: block_id or state_id
/eth/v1/beacon/blocks	POST	Change: block publishing
/eth/v1/beacon/blocks/{block_id}	GET	Change: block_id or state_id
/eth/v2/beacon/blocks/{block_id}	GET	Change: block_id or state_id
/eth/v1/beacon/blocks/{block_id}/root	GET	Change: block_id or state_id
/eth/v1/beacon/blocks/{block_id}/attestations	GET	Change: block_id or state_id
/eth/v1/beacon/pool/attestations	GET	No change.
/eth/v1/beacon/pool/attestations	POST	No Change (see attestation publishing).
/eth/v1/beacon/pool/attester_slashings	GET/POST	No change.
/eth/v1/beacon/pool/proposer_slashings	GET/POST	No change.
/eth/v1/beacon/pool/sync_committees	GET/POST	No change (see sync publishing).
/eth/v1/beacon/pool/voluntary_exits	GET/POST	No change.
/eth/v1/debug/beacon/states/{state_id}	GET	Change: block_id or state_id
/eth/v2/debug/beacon/states/{state_id}	GET	Change: block_id or state_id
/eth/v1/debug/beacon/heads	GET	Change: debug heads
/eth/v1/node/identity	GET	No change.
/eth/v1/node/peers	GET	No change.
/eth/v1/node/peers/{peer_id}	GET	No change.
/eth/v1/node/peer_count	GET	No change.
/eth/v1/node/version	GET	No change.
/eth/v1/node/syncing	GET	Node syncing.
/eth/v1/node/health	GET	No change.
/eth/v1/config/fork_schedule	GET	No change.
/eth/v1/config/spec	GET	No change.
/eth/v1/config/deposit_contract	GET	No change.
/eth/v1/validator/duties/attester/{epoch}	POST	No change.
/eth/v1/validator/duties/proposer/{epoch}	GET	No change.
/eth/v1/validator/duties/sync/{epoch}	POST	No change.
/eth/v1/validator/blocks/{slot}	GET	Change: block production
/eth/v2/validator/blocks/{slot}	GET	As per v1 ^^
/eth/v1/validator/attestation_data	GET	Change: attestation production
/eth/v1/validator/aggregate_attestation	GET	Change: attestation production
/eth/v1/validator/aggregate_and_proofs	POST	Change: attestation publishing
/eth/v1/validator/beacon_committee_subscriptions	POST	No change.
/eth/v1/validator/sync_committee_subscriptions	POST	No change.
/eth/v1/validator/sync_committee_contribution	GET	Change: sync production
/eth/v1/validator/contribution_and_proofs	POST	Change: sync publishing

/eth/v1/events | SSE | Change: event stream

Change details

Change: block_id or state_id

• head: return the [latest_verified_ancestor][].
• genesis: always return the genesis block (it's never optimistic).
• finalized: if the finalized checkpoint is optimistic, return a 503.
• justified: if the justified checkpoint is optimistic, return a 503.
• <slot>: return a 503 if the block is optimistic.
• <root>: return a 503 if the block is optimistic.

Note, it might be onerous to determine if some ancient <slot> or <root>
blocks are optimistic or not. I'd say that if their prior to the finalized
checkpoint (i.e., no longer in fork choice) then just use the state of the
finalized checkpoint. I.e., if the finalized checkpoint is optimistic, assume
all prior block are as well.

Change: headers.

If query parameters are not provided, use the [latest_verified_ancestor][https://github.com/sigp/consensus-specs/blob/opt-sync-2/sync/optimistic.md#helpers].

If query parameters are provided, return a 503 if the matching block is optimistic.

Change: block publishing

If the block is valid and a candidate for optimistic import, import the block and return a 200 OK response.

Change: block production

Never return a block with an optimistic parent.

Ultimately, this should require no change since
engine_getPayloadV1
should return an error if the parent is unknown or invalid.

Change: attestation publishing

No change is required. Assume that if a validator signed a message then it was
confident of the validity of the head block.

Change: attestation production

For any AttestationData included in a request or response, return an error
where attestation_data.beacon_block_root is optimistic.

Change: sync publishing

No change is required. Assume that if a validator signed a message then it was
confident of the validity of the head block.

Change: sync production

No change is required. Assume that if a validator signed a message then it was
confident of the validity of the head block.

Change: node syncing

No change is required, however it might be prudent to add a is_optimistic: bool to the returned object. This would require an API to the specs repo.

Change: debug heads (WIP)

I'm not sure of this one. I'm tempted to leave it as-is since it's a debug endpoint and more useful that way.

Change: event stream (WIP)

I'm not sure of this one. Perhaps we should add an is_optimistic flag to indicate if a block or head is optimistic or not. This requires more thought.

[paulhauner]

This issue is not ready for implementation yet, there's still some more thinking to be done.

[paulhauner]

Another, completely different, approach to this is to allow the beacon node to return info about execution-optimistic chains as if they were fully valid. Perhaps a is_execution_optimistic: bool value could be added wherever the response involves an execution-optimistic assumption.

This approach is appealing because it doesn't break any of the mechanisms that allow validators to follow the head of the optimistic chain. I believe that validators should follow the duties of the canonical chain, even if it's optimistic. However, validators should never produce/sign a message about an optimistic chain. Ensuring validators can follow the duties of an optimistic canonical chain helps ensure that they will be able to do their duties as soon as the chain is fully verified.

[paulhauner]

I'm proposing changes to the API spec here: ethereum/beacon-APIs#190

[paulhauner]

Closed in favor of #3031