Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compressor fixes #3142

Merged
merged 6 commits into from
Mar 22, 2024
Merged

Compressor fixes #3142

merged 6 commits into from
Mar 22, 2024

Conversation

mbway
Copy link
Contributor

@mbway mbway commented Mar 22, 2024

What does this PR do?

Fixes the issue encountered here #3118 (comment) where lzma is refusing to decompress a file in the docker container.

The version of lzma provided by the lzma package has a bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700681
where it refuses to decompress if the file extension does not match, causing the test to fail. This could be worked around but the xz-utils package already provides a version of lzma that does not have this issue, so that can be used instead.

Before submitting

  • Have you read the contributor guidelines?
  • Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
  • Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
  • Did you update any related docs and document your change?
  • Did you update any related tests and add any new tests related to your change? (see testing)
  • Did you run the tests locally to make sure they pass?
  • Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

The version of lzma provided by the lzma package has a bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700681
where it refuses to decompress if the file extension does not match,
causing the test to fail. This could be worked around but the `xz-utils`
package already provides a version of `lzma` that does not have this
issue, so that can be used instead.
@mbway mbway requested a review from a team as a code owner March 22, 2024 12:53
@mbway
Copy link
Contributor Author

mbway commented Mar 22, 2024

I think the reason for the test failure is that the docker image being used isn't updated until the PR is merged? I can implement the workaround into the test (use the proper file extension for the test file) and that should fix it

Copy link
Contributor

@mvpatel2000 mvpatel2000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks :)

@mvpatel2000 mvpatel2000 merged commit 00a07fa into mosaicml:dev Mar 22, 2024
13 of 26 checks passed
j316chuck pushed a commit that referenced this pull request May 16, 2024
* add notes on running tests in docker

* improve error handling of CliCompressor

* remove problematic lzma package from docker image

The version of lzma provided by the lzma package has a bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=700681
where it refuses to decompress if the file extension does not match,
causing the test to fail. This could be worked around but the `xz-utils`
package already provides a version of `lzma` that does not have this
issue, so that can be used instead.

* run as current user in docker container

* use compressor extension for test file

* improved documentation and error messages of CliCompressor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants