[Exporter Refactor] Standardize qat embedding #1270
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bfineran
approved these changes
Dec 16, 2022
dbogunowicz
approved these changes
Dec 19, 2022
…efactor' into standardize-qat-embedding
bfineran
pushed a commit
that referenced
this pull request
Dec 29, 2022
…refactor (#1192) * initial commit * [Exporter Refactor] `BaseTransform` and `OnnxTransform` (#1210) * initial commit * PR comments * initial commit * Delete test_fold_identity_initializers.py * Delete __init__.py * Delete __init__.py * Update src/sparseml/exporters/transforms/base_transform.py * fix docstrings * few improvements and tests Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> * [Exporter Refactor] Onnx sub graph matching (#1211) * Adding onnx graph structural matching * Styling * Adding missing init.py * Update src/sparseml/exporters/transforms/utils/matching.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Updating docstring of structural_matches * Adding __all__ * Addressing review comments * Removing extra file from merge Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Fixing match initializer logic * [Exporter Refactor] Fold Identity Initializers Transform (#1194) * initial commit * PR comments * initial commit * Delete test_fold_identity_initializers.py * Delete __init__.py * Delete __init__.py * Adding onnx graph structural matching * Styling * Update src/sparseml/exporters/transforms/base_transform.py * fix docstrings * Adding missing init.py * Update src/sparseml/exporters/transforms/utils/matching.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Updating docstring of structural_matches * Adding __all__ * ready for review * Update src/sparseml/exporters/transforms/fold_identity_initializers.py Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> * some nits according to Bens comments Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> * Adding ConstantsToInitializers pass (#1227) * Adding UnwrapBatchNorms transform (#1230) * [Export Refactor] Adding InitializersToUint8 transform (#1228) * Adding InitializersToUint8 transform * Update src/sparseml/exporters/transforms/initializers_to_uint8.py * [Exporter Refactor] Quantizable Conv Integer (#1220) * Adding onnx graph structural matching * Styling * Adding missing init.py * Update src/sparseml/exporters/transforms/utils/matching.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Updating docstring of structural_matches * Adding __all__ * initial commit * Update convert_quantizable_conv_integer.py Co-authored-by: Corey Lowman <corey@neuralmagic.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> * [Exporter Refactor] Convert Quantizable Matmul (#1219) * Adding onnx graph structural matching * Styling * Adding missing init.py * Update src/sparseml/exporters/transforms/utils/matching.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Updating docstring of structural_matches * Adding __all__ * initial commit * Delete base_exporter.py * Update src/sparseml/exporters/transforms/convert_quantizable_matmul.py * beautify * check for initializers * add docstring Co-authored-by: Corey Lowman <corey@neuralmagic.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> * fix quality * [Export Refactor] Adding ConvToQLinearConv transform (#1221) * Adding ConvToQLinearConv transform * Responding to review comments * Respond to reviews * [Export Refactor] Adding FlattenQParams transform (#1229) * Adding FlattenQParams transform * Respond to review * Adding GemmToMatMulIntegerAddCastMul trasnform (#1237) * Adding MatMulToMatMulIntegerAddCastMul transform (#1238) * Adding FoldReLUQuants transform (#1240) * Adding PropagateEmbeddingQuantization transform (#1242) * Adding RemoveDuplicateQuantizeOps transform (#1243) * [Export Refactor]Adding GemmToQLinearMatMul transform (#1225) * Adding ConvToQLinearConv transform * Responding to review comments * Adding GemmToQLinearMatMul * Styling * [Export Refactor] Quantize QAT Embedding (#1234) * initial commit * intiial commit * PR comments * fix errors * Apply suggestions from code review * upadte heleprs * matching of conv integer pass * second implementation done, needs some polishing * Adding match_structure and iter_structural_matches * Using structural matching for quantizable_conv_integer * initial commit * Adding onnx graph structural matching * Styling * Adding missing init.py * Update src/sparseml/exporters/transforms/utils/matching.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Updating docstring of structural_matches * Adding __all__ * initial commit * ready for PR * beautify * Delete test_helpers.py * Delete base_exporter.py Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> * [Exporter Refactor] Adding FoldConvDivBn (#1235) * Adding FoldConvDivBn * Expanding docstring * Adding RemoveDuplicateQConvWeights transform (#1244) * [Export Refactor] Delete Trivial Onnx Adds (#1233) * initial commit * get transform into the correct format * ready for review * fix naming in test * Fixing trivial onnx adds Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com> * [Exporter Refactor] Adding QuantizeResiduals transform (#1245) * Adding QuantizeResiduals transform * Adding tests * Styling * Fixing conv-integer transform and add sorting to core ops (#1252) * Don't print out onnx model on validation error (#1253) * FoldReluQuants now modifies all children of relu node (#1254) * Adding shape check for weight comparison in duplicate-qconv-weights (#1255) * [Exporter Refactor] Adding DeleteRepeatedQdq transform (#1257) * Adding DeleteRepeatedQdq transform * Adding unit test for delete repeated qdq * Using assert_node_type * Update src/sparseml/exporters/transforms/delete_repeated_qdq.py * [Exporter Refactor] Adding SkipInputQuantize transform (#1256) * Adding SkipInputQuantize transform * add tests Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> * [Exporter Refactor] Fixing matching logic of qlinear transforms (#1251) * Fixing matching logic of qlinear transforms * Adding folding of input/output quants to qlinears * [Exporter Refactor] Base Exporter class and example implementations (#1249) * Initial comit of exporters * Styling * Fixing SkipInputQuantize * Adding validation methods * Clean up ONNXToDeepsparse * Moving TorchToONNX to pytorch * Adding inplace and saving pre optimized model to ONNXToDeepsparse * Adding sketch of tests * Regression tests against simple models * resnet50 regression tests passing * resnet50 exporters are all equivalent * Moving FoldConvDivBn under initializer folding * Adding yolov5 tests * Apply suggestions from code review Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Review response * Adding notes from review * uncomment asserts... oops * yolo & resnet tests passing Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * [Exporter Refactor] Adding `any_of` for get_structural_matches and MatchResult to str (#1262) * Addin any_of and MatchResult to str * Fixing docstring of get_structural_matches * Adding add_node_deferred and delete_node_deffered to OnnxTransform (#1263) * [Exporter Refactor] Standardize trivial transforms (#1264) * Standardization of some transforms * Adding logging methods to OnnxTransform class * [Exporter Refactor] Standardize non core transforms (#1265) * Standardizing transforms with node removals * Using log_match * [Exporter Refactor] Standardizing MatMulToQLinearMatMul (#1266) * Standardizing MatMulToQLinearMatMul * Using log_match * [Exporter Refactor] Standardizing ConvToConvIntegerAddCastMul (#1267) * Standardizing ConvToConvIntegerAddCastMul * Using log_match * [Exporter Refactor] Standardize qlinears (#1268) * Standardizing qlinear transforms * Using log_match * [ExporterRefactor] Standardizing XToMatMulIntegerAddCastMul transforms (#1269) * Standardizing MatMulIntegerAddCastMul transforms * Using log_match and any_of * [Exporter Refactor] Standardize qat embedding (#1270) * Standardizing QuantizeQATEmbedding * Add log_match * Using renamed versions of transforms (#1271) * Removing unused tests * [Exporters Refactor] Regression test Bert (#1258) * initial commit * Apply suggestions from code review * Update tests/sparseml/pytorch/test_torch_to_onnx_exporter.py * Fixing bert exporters Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com> * [Exporters Refactor] Fix failing base tests (#1260) * initial commit * PR edits * Delete recipe.yaml * fix onnx problem * Fixing torch import issue and numpy attr error * Another attempt at fixing get_numpy_dtype * Fix numpy.float usage Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com> Co-authored-by: bogunowicz@arrival.com <bogunowicz@arrival.com> Co-authored-by: corey-nm <109536191+corey-nm@users.noreply.github.com> Co-authored-by: Corey Lowman <corey@neuralmagic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as others