Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move empty directory removal to sparseml instead of sparsify #273

Merged
merged 1 commit into from
Jul 21, 2023

Conversation

dsikka
Copy link
Contributor

@dsikka dsikka commented Jul 20, 2023

  • Default behaviour involves the user passing in just the data arg and this always results in imagefolder being populated in for the dataset arg, for both custom datasets and datasets where we download public data (this shouldn't be happening/should be separated further in a follow-up PR)
  • However, we only want to check the existence of folders for custom datasets, not public datasets we haven't pulled down.
  • Therefore, remove the check and error
  • This should also just be done in sparseml, since click.Path() is where the folder is created.

Tested locally using the following command:
sparsify.run sparse-transfer --use-case image-classification --data imagenette --optim-level 0.5

@dsikka dsikka requested a review from rahul-tuli July 20, 2023 21:59
@dsikka dsikka merged commit e1c5088 into main Jul 21, 2023
3 checks passed
@dsikka dsikka deleted the imagenet_dir_fix branch July 21, 2023 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants