Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: set max_connections sqlx parameter #1508

Merged
merged 6 commits into from
Dec 8, 2023

Conversation

lostman
Copy link
Contributor

@lostman lostman commented Dec 7, 2023

Description

Fixes #1507.

The default value for max_connections in sqlx is 10. From some testing, our number of DB connections per indexer can go up to ~6 and so, running fuel-indexer with more than one indexer could lead to failures.

This PR sets the default to 100 and adds a CLI flag to modify it.

Also, a warning will be issued if the set config value exceeds what is set in the backend.

Testing steps

cargo run -p fuel-indexer-benchmarks --bin qa -- --network beta-4.fuel.network --blocks 500 --runs 4 --num-additional-indexers 10

Should succeed.

Example: with --num-additional-indexers 10we get up to43` connections

run: 1
    runtime:        0.3 minutes
    start block:    0
    end block:      500
    avg memory:     241.3kB
    stdv memory:    0.0kB
    avg cpu:        1.5%
    stdv cpu:       0.0%
    db connections: 43/100
    missing blocks: 0
    blocks/sec:     26.3
    index size:     8.089kB per block

Changelog

  • Add --max-db-connections CLI option
  • Set the corresponding sqlx parameter
  • Update QA script to track the number of active connections

@lostman lostman self-assigned this Dec 7, 2023
@lostman lostman changed the title set max_connections sqlx parameter bugfix: set max_connections sqlx parameter Dec 7, 2023
@lostman lostman marked this pull request as ready for review December 7, 2023 21:13
deekerno
deekerno previously approved these changes Dec 8, 2023
Copy link
Contributor

@deekerno deekerno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link
Contributor

@ra0x3 ra0x3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lostman these manifest files need to be corrected

@lostman lostman requested a review from ra0x3 December 8, 2023 13:06
Copy link
Contributor

@ra0x3 ra0x3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected. Awesome @lostman

@lostman lostman enabled auto-merge (squash) December 8, 2023 13:32
@lostman lostman merged commit 58cdb47 into develop Dec 8, 2023
1 check passed
@lostman lostman deleted the maciej/1507-sqlx-pool-timed-out branch December 8, 2023 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: Sqlx(PoolTimedOut) when running multiple indexers
3 participants