Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache creation of states_token_mapping in the __init__ of RegexFSM #492

Merged
merged 4 commits into from
Jan 11, 2024

Conversation

RobinPicard
Copy link
Contributor

PR aimed at talking about the resolution of issue #483

I tried to do it in a way that requires as few modification to existing functions as possible. It implies modifying the type of several variables for them to be cacheable/json-serializeable. It looks a bit awkward and leads to the initialization of the RegexFSM instance taking a bit more time when not cached because of the type conversions, but I thought that could be the start of further discussions on the topic

@rlouf
Copy link
Member

rlouf commented Dec 30, 2023

Great! I wonder if using diskcache everywhere in the codebase instead of perscache could help make the code cleaner? See #441

@RobinPicard
Copy link
Contributor Author

RobinPicard commented Dec 30, 2023

I think it makes the code much cleaner in that case (and does not change the use of @cache in the OpenAI model)

@RobinPicard RobinPicard marked this pull request as ready for review December 31, 2023 08:15
@rlouf rlouf mentioned this pull request Jan 6, 2024
outlines/caching.py Outdated Show resolved Hide resolved
outlines/fsm/fsm.py Outdated Show resolved Hide resolved
outlines/fsm/fsm.py Show resolved Hide resolved
outlines/caching.py Outdated Show resolved Hide resolved
@lapp0
Copy link
Collaborator

lapp0 commented Jan 7, 2024

Good work!

outlines/fsm/fsm.py Outdated Show resolved Hide resolved
@rlouf
Copy link
Member

rlouf commented Jan 11, 2024

Thank you!

@rlouf rlouf merged commit 076bd98 into outlines-dev:main Jan 11, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants