Manage pending in memory and move it to synchroniser #1844
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
from
4d851b3
to
cf6de15
Compare
|
This also fixes #1450 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1844 +/- ##
==========================================
- Coverage 75.40% 75.36% -0.04%
==========================================
Files 100 100
Lines 8982 8915 -67
==========================================
- Hits 6773 6719 -54
+ Misses 1604 1600 -4
+ Partials 605 596 -9 ☔ View full report in Codecov by Sentry. |
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
2 times, most recently
from
5e2d6e6
to
cc6cf21
Compare
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
from
cc6cf21
to
09e6778
Compare
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
2 times, most recently
from
c9f49cd
to
81d8a1c
Compare
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
2 times, most recently
from
7cd5306
to
9220e49
Compare
kirugan
reviewed
Jul 31, 2024
| } | ||
| return db.ErrKeyNotFound | ||
| } | ||
|
|
There was a problem hiding this comment.
why for tx-s you moved logic to transactionByHash but here it's not part of receiptByHash anymore ?
There was a problem hiding this comment.
I don't think I understand your question.
I removed checking of pending transactions and receipt from both TransactionByHash and Receip
kirugan
reviewed
Jul 31, 2024
kirugan
reviewed
Jul 31, 2024
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
3 times, most recently
from
ca7b92b
to
7342444
Compare
rianhughes
reviewed
Aug 5, 2024
rianhughes
reviewed
Aug 5, 2024
rianhughes
approved these changes
Aug 5, 2024
Pending Block is now only managed in memory this is to make sure that pending block in the DB and in memory do not become out of sync. Before the pending block was managed in memory as a cache, however, since there is only one pending block at a given time it doesn't make sense to keep track of pending block in both memory and DB. To reduce the number of block not found errors while simulating transactions it was decided to store empty pending block, using the latest header to fill in fields such as block number, parent block hash, etc. This meant that any time we didn't have a pending block this empty pending block would be served along with empty state diff and classes. Every time a new block was added to the blockchain a new empty pending block was also added to the DB. The unforeseen side effect of this change was when the --poll-pending-interval flag was disabled the rpc would still serve a pending block. This is incorrect behaviour. As the blocks changed per new versions of starknet the empty block also needed to be changed and a storage diff with a special contract "0x1" needed to be updated in the state diff. This overhead is unnecessary and incorrectly informs the user that there is a pending block when there isn't one.
In order to moved handling of pending to synchroniser the following changes needed to be made: - Add database to synchroniser, so that pending state can be served - Blockchain and Events Filter have a pendingBlockFn() which returns the pending block. Due to import cycle pending struct could not be referenced, therefore, the anonymous function is passed. - Add PendingBlock() to return just the pending block, this was mainly added to support the pendingBlockFn(). - In rpc package the pending block and state is retrieved through synchroniser. Therefore, receipt and transaction handler now check the pending block for the requested transaction/receipt.
IronGauntlets
force-pushed
the
IronGauntlets/fix-pending-block-issues
branch
from
7342444
to
66f53e1
Compare
kirugan
reviewed
Aug 22, 2024
blockchain/blockchain.go
Outdated
| if existingPending.Block.TransactionCount >= pending.Block.TransactionCount { | ||
| return nil // ignore the incoming pending if it has fewer transactions than the one we already have | ||
| // ignore the incoming pending if it has fewer transactions than the one we already have |
There was a problem hiding this comment.
can you please elaborate in what case it is possible to have incoming block with smaller number of transactions ?
kirugan
reviewed
Aug 22, 2024
blockchain/blockchain.go
Outdated
| @@ -248,7 +248,7 @@ func (b *Blockchain) TransactionByHash(hash *felt.Felt) (core.Transaction, error | |||
| // not found in the canonical blocks, try pending | |||
| if errors.Is(err, db.ErrKeyNotFound) { | |||
| var pending Pending | |||
| pending, err = b.pendingBlock(txn) | |||
| pending, err = pendingBlock(txn) | |||
There was a problem hiding this comment.
should we check for ErrPendingBlockNotFound here?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes
parts of#1789.A migration to remove the pending from DB is still outstanding.It would be best to review the commits individually and in order.