-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] put.stream() with integrity check might not work #27
Comments
I tried to update my tool that uses cacache to 15.x and it looks like the issue is still present, so I'm currently stuck at 12.x. Is there anything I can do to help you fix the problem? |
I did some tests and the stream interface is no longer functional starting with 13.0.0. In other words, the same code that works with 12.x fails with 13.0. My current workaround was to use an internal buffer, but since the files that I need to install are large, like 150 MB or more, this is not an optimal solution, especially when running on small systems, like Raspberry Pis. |
The internal buffer workaround is far from great, for example it makes AppVeyor tests fail with out of memory:
Is there anything that can be done to fix stream support in cacache? |
It might be possible that the issue was introduced with f4c0962. |
What / Why
Apparently
put.stream()
with integrity checks leaves the cache in an inconsistent state, and all keys inserted with this procedure are purged by the nextverify()
.I tried to find some use cases of
put.stream()
, but I could not find anything in npm, nor any tests here, so, although I don't rule out an improper use case in my code, I guess we are facing a small bug that slipped due to incomplete testing.When
Always.
How
Current Behavior
I'm using
put.stream()
to add large archives to the cache, and I have the expected sha256 digest for each archive.If I set
opts.algorithms = ['sha256']
and I check the resultingintegrity[hashAlgorithm][0].source
I get the expected digest, and everything is fine.If I set
opts.integrity
to the expected digest, the call seems ok too, it does not throw any error, the archive is saved in the cache and decompressing it is fine.However, if I run a
verify()
, the associated entries are removed, as there would be something wrong with them.Steps to Reproduce
put.stream()
withopts.algorithms = ['sha256']
verify()
ls()
; the recently added key is thereput.stream()
with opts.integrity set to the digestverify()
ls()
; the key is no longer in the cacheNote: using sha256 might not be relevant, but I mentioned it to match my test.
Expected Behavior
Running
verify()
afterput.stream()
with integrity checks should not remove the entries from the cache.Who
References
The text was updated successfully, but these errors were encountered: