Store Previously Requested ETH1.0 Logs Block Number

[rauljordan]

Hi all,

Currently our powchain service reads all historical logs from the deposit contract from ETH1.0. As the contract grows even more, we should be persisting the last requested block number for logs to disk in the powchain service to avoid looping through all of these logs every time the node is launched. a PR for this would store the last requested ETH1.0 block number for log fetching in BoltDB. Next time the node spins up, we can request logs started from this block number + 1. Thanks @nisdas for bringing this up.

[prestonvanloon]

Whoever takes this issue should outline a plan for caching these logs.

They need to consider how to do this after 2 million or 50 million PoW blocks without overwhelming the process memory.

[shayzluf]

I am working on this task

Looking into the current code first

[shayzluf]

@prestonvanloon caching the last block read index (big.Int) in boltdb and reading it shouldn't be an issue for the process memory . Am I missing something here?

[nisdas]

Clarified with @shayzluf in chat https://discordapp.com/channels/476244492043812875/483017808658169866/550915035895955466 , about the requirements for the issue

[prestonvanloon]

This is a mainnet config requirement. Reading 16k logs on start up takes several minutes.

[rauljordan]

Hey @prestonvanloon could you try this again with #4299

[prestonvanloon]

#4299 adds significant improvement from 3 minutes to under 1 minute, but this is still unsatisfactory for node restarts.

[rauljordan]

I believe this is resolved by #4329, right @nisdas?

[nisdas]

Yeap @rauljordan, this is resolved by #4329