"PublishError.InsufficientPeers" error on submitPoolBlsToExecutionChange

[marioevz]

Describe the bug

On a hive testnet with Deneb genesis, sending a valid /eth/v1/beacon/pool/bls_to_execution_changes POST request to the Beacon API results in error:

Oct-26 22:02:04.910[api]             �[31merror�[39m: Error on submitPoolBlsToExecutionChange [0] validatorIndex=0 - PublishError.InsufficientPeers
Error: PublishError.InsufficientPeers
    at Eth2Gossipsub.publish (file:///usr/app/node_modules/@chainsafe/libp2p-gossipsub/dist/src/index.js:1547:19)
    at async NetworkCore.publishGossip (file:///usr/app/packages/beacon-node/lib/network/core/networkCore.js:290:32)

Even when there is a second lodestar node connected and blocks and attestations are being gossiped correctly.

The error occurs in regardless of the epoch in which the request is sent to the beacon node.

A similar test where the Deneb fork happens at epoch 1, and then the BLSToExecutionChanges are sent, does not exhibit the same issue, and the changes are broadcasted successfully.

The main difference in the lodestar logs between the two tests is that the successful test is subscribed to topics:

Subscribe to gossipsub topic topic=/eth2/<CAPELLA DIGEST>/bls_to_execution_change/ssz_snappy
Subscribe to gossipsub topic topic=/eth2/<DENEB DIGEST>/bls_to_execution_change/ssz_snappy

whereas the failing test is only subscribed to topic:

Subscribe to gossipsub topic topic=/eth2/<DENEB DIGEST>/bls_to_execution_change/ssz_snappy

And later in the logs of the successful test we can see that the message is indeed gossiped to topic /eth2/<CAPELLA DIGEST>/bls_to_execution_change/ssz_snappy, which leads me to believe that in fact none of the lodestar peers are listening to the topic where they both broadcast to.

Expected behavior

bls_to_execution_changes should work.

Steps to reproduce

Using hive branch https://github.com/marioevz/hive/tree/eth2-dencun, run the following command:

go build -v .
cl=lodestar; ./hive --client-file ./configs/cancun.yaml --client go-ethereum,${cl}-bn,${cl}-vc --sim eth2/dencun --sim.limit "eth2-deneb-sync/test-sync-from-deneb"

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

Branch deneb-builder

[g11tech]

thanks @marioevz for the snopsis, yea if the topic isn't subscribed this could lead to the failure ... will debug