Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ONNX Export for Refactored Modifiers #1769

Merged
merged 14 commits into from
Oct 20, 2023
Merged

Conversation

Satrat
Copy link
Contributor

@Satrat Satrat commented Oct 17, 2023

  • Updated QuantizationModifier to apply structure before initialization
  • New version of export script that is compatible with the new modifier and session framework.The main change is replacing Managers with SparseSessions

Example

sparseml.transformers.export_onnx_refactor --task text-generation --model_path path/to/model/and/recipe

or

python src/transformers/sparsification/obcq/export.py  --task text-generation --model_path path/to/model/and/recipe

@Satrat Satrat marked this pull request as ready for review October 17, 2023 19:54
bfineran
bfineran previously approved these changes Oct 18, 2023
Copy link
Member

@bfineran bfineran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - would potentially be good to add a way for the current default script sparseml.transfomers.export_onnx to route to this one if it detects one of the new recipes

src/sparseml/transformers/sparsification/obcq/export.py Outdated Show resolved Hide resolved
@Satrat Satrat requested a review from bfineran October 19, 2023 15:45
@mgoin mgoin merged commit c9c1c21 into main Oct 20, 2023
11 checks passed
@mgoin mgoin deleted the modifier_refactor_onnx_export branch October 20, 2023 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants