Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use multiple threads in CLI debug_account_versions and debug_unconfirmed_frontiers #2950

Merged
merged 5 commits into from
Sep 25, 2020
Merged

Use multiple threads in CLI debug_account_versions and debug_unconfirmed_frontiers #2950

merged 5 commits into from
Sep 25, 2020

Conversation

guilhermelawless
Copy link
Contributor

  • Add parallelized pending table traversal (thanks @wezrule)
  • Expose transaction in parallel traversal methods
  • Optimize CLI debug_account_versions
    • Use parallel table traversal for frontiers and pending
    • Merge sets to do a single lookup of opened accounts
    • Change to boost::unordered_set and unordered_map for performance in very large containers
  • Optimize CLI debug_unconfirmed_frontiers
    • ledger::unconfirmed_frontiers now traverses the accounts table in parallel.
    • The frontiers table is not traversed sequentially anymore, but this still results in a speed up from what I saw.
  • Add value_type alias to nano::locked

Comparisons in different systems (cold and warm) are appreciated

guilhermelawless and others added 5 commits September 17, 2020 11:26
@guilhermelawless @wezrule
Add parallelized pending table traversal
14da223 
Co-Authored-By: Wesley Shillingford <650038+wezrule@users.noreply.github.com>
@guilhermelawless
Optimize CLI debug_account_versions
661d787 
- Use parallel table traversal for frontiers and pending
- Merge sets to do a single lookup of opened accounts
- Change to boost::unordered_set and unordered_map for performance in very large containers
@guilhermelawless
Optimize CLI debug_unconfirmed_frontiers
a5780df 
ledger::unconfirmed_frontiers now traverses the accounts table in parallel.

The frontiers table is not traversed sequentially anymore, but this still results in a speed up.
@guilhermelawless
Add value_type alias to nano::locked
e302390
@guilhermelawless
Expose transaction in parallel traversal methods
2f8ec49
@guilhermelawless guilhermelawless added performance Performance/resource utilization improvement cli Changes related to the Command Line Interface labels Sep 17, 2020
@guilhermelawless guilhermelawless added this to the V22.0 milestone Sep 17, 2020
@guilhermelawless guilhermelawless self-assigned this Sep 17, 2020
@wezrule wezrule mentioned this pull request Sep 25, 2020
Merged
@guilhermelawless guilhermelawless merged commit 032165b into nanocurrency:develop Sep 25, 2020
@guilhermelawless guilhermelawless deleted the cli/ledger-parallelize branch September 25, 2020 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wezrule wezrule wezrule approved these changes

@SergiySW SergiySW Awaiting requested review from SergiySW

Assignees

@guilhermelawless guilhermelawless

Labels
cli Changes related to the Command Line Interface performance Performance/resource utilization improvement
Projects
None yet
Milestone
V22.0
Development

Successfully merging this pull request may close these issues.
2 participants
@guilhermelawless @wezrule