-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
By default avoiding generating files in temp directory #1609
Conversation
…euralmagic/sparseml into feature/damian/avoid_tmp_files
…euralmagic/sparseml into feature/damian/avoid_tmp_files
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM overall - should be noted for testing creating tempfiles is ok
@bfineran I think that there is a risk (currently low but will grow over time as we work with larger and larger models) of saving very large files to /tmp. This is something that QA already reported. I think that saving files to /tmp is fine, as long as they also get properly cleaned up. We have been a bit sloppy at concluding our unit tests with appropriate cleanups in the past; this PR is a partial remedy to that sloppiness (fun fact, the test that was writing the biggest amount of mbytes to /tmp and did not clean them up was written by the user d.bogunowicz ;)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great cleanup and updated test conditions!
This reverts commit 6dbf554.
Changes
conftests
so that after running any test target, the total size of files created after running the test suite can be: zero mb for the root directory, and up to 1mb for the temp directory./src
and/tests
/tmp
from 200mb and 800mb respectively down to practically zero mb.Audit Plan
Auditing the pathways that contain keywords:
tempfile
anddownload
to make sure that we are not being sloppy with saving or downloading files to the temporary directory.Auditing the
/src
Auditing the
/tests
Note: I consider saving to tempfile benign if it is either done within a
context
(using a context manager), or using thetmp_path
fixture or factory within thepytest
framework.