Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

✨ Add hash check to data download #284

Merged
merged 3 commits into from
Apr 29, 2022
Merged

✨ Add hash check to data download #284

merged 3 commits into from
Apr 29, 2022

Conversation

ashwinvaidya17
Copy link
Collaborator

Description

Changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My code follows the pre-commit style and check guidelines of this project.
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing tests pass locally with my changes

@ashwinvaidya17 ashwinvaidya17 added Enhancement New feature or request Data labels Apr 26, 2022
samet-akcay
samet-akcay requested changes Apr 26, 2022
View reviewed changes
anomalib/data/mvtec.py Outdated
@@ -384,6 +384,8 @@ def prepare_data(self) -> None:
filename=self.root / dataset_name,
reporthook=progress_bar.update_to,
)
logger.info("Checking hash")
hash_check(self.root / dataset_name, "eefca59f2cede9c3fc5b6befbfec275e")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quite minor, but btech has zip_filename, would be good if the two has consistent implementation, (ie., both has zip_filename or path)

djdameln
djdameln requested changes Apr 26, 2022
View reviewed changes
anomalib/data/utils/download.py Outdated
with open(file_path, "rb") as hash_file:
assert (
hashlib.md5(hash_file.read()).hexdigest() == expected_hash
), f"Downloaded file {file_path} does match the required hash."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be "does not match"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙃

@samet-akcay samet-akcay self-requested a review April 29, 2022 07:24
@samet-akcay samet-akcay merged commit 9076a95 into development Apr 29, 2022
@samet-akcay samet-akcay deleted the feature/av/file_hash branch April 29, 2022 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@samet-akcay samet-akcay samet-akcay approved these changes

@djdameln djdameln djdameln approved these changes

Assignees
No one assigned
Labels
Data Enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Check hash of downloaded MVTec and BTech datasets
3 participants
@ashwinvaidya17 @samet-akcay @djdameln