Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronize and prevent repeated package caching in all presets #1071

Merged
merged 4 commits into from
Aug 23, 2021

Conversation

HannesWell
Copy link
Contributor

With this PR I suggest to store pythonFile computed in the cpython.presets.python.cachePackage() in a static variable and to return it if it is already computed to prevent caching of the python packages multiple times.
I assume the result should be the same in each call?

This method is called twice when using the embeddedpython.Python class (in a stripped variant) and each call takes at least one third of the 2 seconds initialization time of embeddedpython.Python. So also caching the path would speed up the initialization significantly.

Actually this method could be synchronized to prevent concurrent caching of the packages. If you agree I can update this PR accordingly.

@saudet
Copy link
Member

saudet commented Aug 21, 2021

Right, that and synchronizing it make it more consistent with how Loader.load() works. Sounds like a good idea! While we're at it, could you do the same for numpy, scipy, tensorflow, gym, opencv, llvm, and tvm as well?

@HannesWell
Copy link
Contributor Author

HannesWell commented Aug 21, 2021

Right, that and synchronizing it make it more consistent with how Loader.load() works. Sounds like a good idea! While we're at it, could you do the same for numpy, scipy, tensorflow, gym, opencv, llvm, and tvm as well?

Great.
Sure. I have applied it to the other mentioned presets and also searched for other cachePackage() methods but found none. So I think those are all.

@HannesWell HannesWell changed the title [CPython] prevent repeated package caching Prevent repeated package caching in all presets Aug 21, 2021
@HannesWell HannesWell changed the title Prevent repeated package caching in all presets Synchronize and prevent repeated package caching in all presets Aug 21, 2021
@saudet saudet merged commit 6c2e0a1 into bytedeco:master Aug 23, 2021
@HannesWell HannesWell deleted the cpythonPackageCaching branch August 23, 2021 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants