Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the performance issue caused by track_entry_statics #4382

Merged
merged 4 commits into from
Mar 18, 2024

Conversation

chenyukang
Copy link
Collaborator

What problem does this PR solve?

When the number of entries is large(>= 100000), we observed a performance regression for _update_tx_pool_for_reorg, the delay may longer than a duration of 10s, the RPC send_transaction also has a noticeable delay when the number of entries increasing a lot.

What is changed and how it works?

The performance regression is caused by a slow implementation of track_entry_statics, which will be called when adding a new entry or changing entry status.

entries.get_by_status(&Status::Pending).len() is slow when entries number is large, _update_tx_pool_for_reorg will iterate the proposals to invoke track_entry_statics multiple times(1000+ in corner case).

This PR will add 3 counts to track the number of pending, gap, proposed to resolve the performance issue.

Related changes

  • PR to update owner/repo:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code ci-runs-only: [ quick_checks,linters ]

Release note

Title Only: Include only the PR title in the release note.

@chenyukang chenyukang requested a review from a team as a code owner March 14, 2024 13:43
@chenyukang chenyukang requested review from zhangsoledad and removed request for a team March 14, 2024 13:43
@chenyukang chenyukang force-pushed the yukang-perf-tuning branch 3 times, most recently from de37b6a to 4b715ff Compare March 14, 2024 17:06
@chenyukang chenyukang added this pull request to the merge queue Mar 18, 2024
Merged via the queue into nervosnetwork:develop with commit f9c0373 Mar 18, 2024
32 checks passed
@doitian doitian mentioned this pull request Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants