-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ENH] Use foyer for block cache (#2431)
## Description of changes *Summarize the changes made by this PR.* - Improvements & Bug fixes - This PR fixes bugs that readers and writers share the cache. The fix is as follows: 1. fork calls get instead of getting data from the read cache 2. commit returns the newly created blocks 3. flush takes the newly created blocks as input. - Temporarily disable the shuttle test as there is a compatibility issue between tokio filesystem calls and shuttle runtime. Tokio filesystem call will panic if not wrapped in tokio runtime. - New functionality - This PR introduces cache with eviction policies. Foyer (https://github.com/MrCroxx/foyer) is used as the cache. In the future, Foyer will be used as a hybrid cache. - Introduced cache-related configs to be used by different components. ## Test plan *How are these changes tested?* - [ ] Tests pass locally with `pytest` for python, `yarn test` for js, `cargo test` for rust ## Documentation Changes *Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the [docs repository](https://github.com/chroma-core/docs)?* --------- Co-authored-by: Max Isom <codetheweb@users.noreply.github.com> Co-authored-by: Hammad Bashir <HammadB@users.noreply.github.com> Co-authored-by: Sanket Kedia <kediasanket11121993@gmail.com>
- Loading branch information
1 parent
2722be0
commit 4644217
Showing
28 changed files
with
1,433 additions
and
300 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.