-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor of Transformers SparseML CLI and integrations #536
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Really like these changes, overall the code looks good, left a few nits,
main required change would be
- fix the name of
sparseml.sparsification
package, right now it is misspelled assparsfiication
need to rename and update usages.
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
… for transformers trainer
src/sparseml/transformers/export.py
Outdated
|
||
# create fake model input | ||
inputs = tokenizer( | ||
"", return_tensors="pt", padding=PaddingStrategy.MAX_LENGTH.value | ||
).data # Dict[Tensor] | ||
_LOGGER.info(f"Created sample inputs for the ONNX export process: {inputs}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the inputs here are coming from a tokenized/padded empty string and will be just a large tensor of mostly 0s, ends up being more like log spam. could print out input shapes or names instead
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, left comment on export logging but not critical
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* Refactor of Transformers SparseML CLI and integrations * Refactor export.py to use new pathways, fix make quality * Update src/sparseml/optim/manager.py Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * Update src/sparseml/transformers/utils/model.py Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * fixes from review * fixes from review and testing * bug fixes and logging * bug fixes for export and distillation * review fixes, quality fixes, style fixes * fix dependency issue * fix distillation tests * fix distillation tests * fix distillation tests * fill in docs and update style * fix issue with distillation improperly updating students inputs * fix quality * Update src/sparseml/pytorch/optim/modifier_distillation.py * add in better logging for missing and unexpected keys in model reload for transformers trainer * fix logging for transformers export Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
* Refactor of Transformers SparseML CLI and integrations (#536) * Refactor of Transformers SparseML CLI and integrations * Refactor export.py to use new pathways, fix make quality * Update src/sparseml/optim/manager.py Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * Update src/sparseml/transformers/utils/model.py Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * fixes from review * fixes from review and testing * bug fixes and logging * bug fixes for export and distillation * review fixes, quality fixes, style fixes * fix dependency issue * fix distillation tests * fix distillation tests * fix distillation tests * fill in docs and update style * fix issue with distillation improperly updating students inputs * fix quality * Update src/sparseml/pytorch/optim/modifier_distillation.py * add in better logging for missing and unexpected keys in model reload for transformers trainer * fix logging for transformers export Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * Fix model load bug and add logging to catch potential future issues (#537) * Fix model load bug and add logging to catch potential future issues * initial migration to generalize module sparsification information * propagate ModuleSparsificationInfo * report type of input tensors in export.py * minor bug fixes * ModuleSparsificationInfo docs * export onnx bugfix * bug fixes * make style * bug fix for quantization * revert to use ScheduledOptimizer due to bug with torch LambdaLR * remove language_modeling script * add end model sparsification log Co-authored-by: Benjamin <ben@neuralmagic.com> Co-authored-by: Mark Kurtz <mark@neuralmagic.com> Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
No description provided.