Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage Caching #692

Open
philippeller opened this issue Feb 22, 2022 · 0 comments
Open

Stage Caching #692

philippeller opened this issue Feb 22, 2022 · 0 comments

Comments

@philippeller
Copy link
Collaborator

Currently there is (unsafe) ad hoc caching implemented for the KDE stage (PR #689).

We should make this safer, and on a more global level.

Ideas:

I think cleanest would be to implement stashing in general at the pipeline level.
So you would stay: stash at stage X, and then the pipeline will not run anything up to stage X, except when a parameter has changed.
It probably should then keep an internal copy of some keys and apply those, just to avoid side effects, but that is low cost.
we could make the user specify what keys to be cached! So this way would be explicit.
E.g. in the cfg specify: At stage X cache keys x, and y
We could do that in the [pipeline] section of the cfg most naturally.
Add a optional: stash = {5 : ['weights', 'blah', ...]} or so

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant