Log provider edge cases - combined changes

[ferglor]

This PR is the complete changeset introduced across the following three PRs:

• Dequeue upkeeps using the current dequeue iteration #12852 (dequeue with iterations)
• Dequeue minimum commitment of logs first - rework #13470 (dequeue min commitment first)
• Handle reorgs in the provider - rework #13480 (handle re orgs)

Additional changes have been made on top of these three base PRs, but to summarise:

• We are now calculating the number of iterations needed to dequeue logs for all upkeeps in the buffer, and we dequeue a different set of upkeeps in every iteration
• We're also using a new dequeue coordinator to determine which block window should be dequeued from. The dequeue coordinator operates as follows:
  • Dequeue the windows for which we have not yet dequeued the minimum number of guaranteed logs for, going from the oldest window to the newest until all windows have had the minimum number of guaranteed logs dequeued
  • Dequeue the remaining logs from all windows as best effort, going from the oldest window to the newest, until all windows are completely dequeued
• When a reorg has been identified, the dequeue coordinator gets reset for the affected block windows

Testing

Unit testing

Multiple unit tests have been added around the provider, buffer, and dequeue coordinator. The provider tests cover multiple scenarios regarding block progression, how dequeues are distributed across upkeeps and within block windows, as well as reorgs and changes in the number of upkeeps.

Happy path tests (AUTO-9176)

• AUTO-8216 - 0.3% missed events vs 0.1% missed events in develop
• AUTO-8219 - 84% missed events vs 95% missed events in develop
• AUTO-8220 - 92% missed events vs 95% missed events in develop

[infiloop2]

now that this function accesses and changes buffer state, it seems we need to acquire a lock

[infiloop2]

discussed offline, we'll need to combine lock for blockStatistics and evictReorgdLogs (maybe combine those)
also would be good to think through concurrency of the whole function

[cl-sonarqube-production]

Quality Gate passed

Issues
2 New issues
1 Fixed issue
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube