Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add database config: max db size #2948

Merged
merged 1 commit into from
Feb 27, 2024
Merged

Add database config: max db size #2948

merged 1 commit into from
Feb 27, 2024

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Feb 26, 2024

Introduced a new configuration maxDBSize for Database.
maxDBSize is the hard limit of how large (in terms of bytes of disk pages) the database can allocate.
The config value should be no less than 4MB in bytes and be power of 2.
This is temporarily added to let users get around with issues related to not able to mmap the default 8TB virtual memory address space in some runtime environment.

     * @param maxDBSize The maximum size of the database in bytes. Note that this is introduced
     * temporarily for now to get around with the default 8TB mmap address space limit some
     * environment. This will be removed once we implemente a better solution later. The value is
     * default to 1 << 43 (8TB) under 64-bit environment and 1GB under 32-bit one (see
     * `DEFAULT_VM_REGION_MAX_SIZE`).

There are several todos left: reflect the changes in Java/NodeJS/Rust APIs, and CLI.

Before our next major release, we should revisit this and change the mmap allocation to one page/frame group at a time, so we don't need to allocate a given size at once.

@ray6080 ray6080 requested a review from mewim February 26, 2024 18:44
@ray6080 ray6080 force-pushed the add-max-db-size branch 2 times, most recently from b8a48c9 to c7a8785 Compare February 26, 2024 18:53
Copy link

codecov bot commented Feb 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.53%. Comparing base (d2cfc65) to head (c7a8785).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2948      +/-   ##
==========================================
+ Coverage   93.51%   93.53%   +0.01%     
==========================================
  Files        1121     1121              
  Lines       42913    42924      +11     
==========================================
+ Hits        40131    40147      +16     
+ Misses       2782     2777       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants