Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New Export API #1318

Merged
merged 57 commits into from
Aug 8, 2023
Merged

New Export API #1318

merged 57 commits into from
Aug 8, 2023

Conversation

BloodAxe
Copy link
Collaborator

@BloodAxe BloodAxe commented Jul 24, 2023 •
edited
Loading

What this PR does

In the nutshell, it allows to export detection models more easily:

model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")
model.export("yolo_nas_s.onnx", preprocessing=True, postprocessing=True, quantization_mode="int8")

Supported models are:

  • YoloX (FP32 & INT8)
  • PPYolo-E (All variants)
  • YoloNAS (All variants)

Not supported architectures:

  • SSD (All variants)
  • YoloX (FP16)

Changes in API

Potentially breaking changes

  • HasPredict do not inherit from Protocol. During testing it became clear that it messes with MRO and do not allow method overriding to work as intended. Using a pure mixin approach indeed worked fine.

Bugfixes

  • Fixed cache_anchors in YOLOX, PPYoloE & YoloNAS to respect the dtype & device of the surrounding modules of the model. That is if model is in fp16 mode and on cuda, cached anchors should be also placed there. Additionaly, if anchors were created first and then model.half() to be called - they should be converted to half as well, which was not the case and was fixed.

  • Supports only (some) detection models so far

Supports:

  • Export to onnxruntime & tensorrt
  • End-to-End export with NMS
  • Quantization with calibration

Still remaining to implement:

  • Remaining detection models
  • Inference of the preprocessing (normalization) params from the model itself
  • Docs & Tutorials

@BloodAxe
Designing export API
0414394
@BloodAxe
Export WIP
054152d
@BloodAxe
ONNX NMS
1872443
@BloodAxe
Export WIP
4a76131
@BloodAxe
Refactor test and move benchmark API to functino
fffcdc5
@BloodAxe
Export WIP
55afd70
@BloodAxe
Make the top_k a constant and not variable since TRT export does not …
ed8730b 
…work with dynamic top_k
@BloodAxe
Refactor test and move benchmark API to functino
cdc41d7
@BloodAxe
Added option to change the output format
72e7d66
@BloodAxe
Refactor test and move benchmark API to functino
015357d
@BloodAxe
Added option to change the output format
794fc73
@BloodAxe
Refactor test and move benchmark API to functino
da99bfa
@BloodAxe
Merge branch 'master' into feature/SG-1001
a18a19d
@BloodAxe
Fixing export to make it TRT friendly
ca6ab9b
@BloodAxe
Fixing export to make it TRT friendly
c7f123b
@BloodAxe
Fixing export to make it TRT friendly
f08ca47
@BloodAxe
Fixing export to make it TRT friendly
8c188ca
@BloodAxe
Remove unused classes
f7e6e93
@BloodAxe
Remove unused classes
56daabd
@BloodAxe
Remove unused classes
3ad8636
@BloodAxe
Remove unused classes
3717316
@BloodAxe
Fixing export to FP16
4ac7912
@BloodAxe
Fixing export to FP16
fa6167d
@BloodAxe
Improve output of the benchmark result
fdd6582
@BloodAxe
Improve device handling when exporting NMS
ff1fcff
@BloodAxe
Improve device handling when exporting NMS
16521a9
@BloodAxe
Fix nms format conversion modules export
46e5a6e
@BloodAxe
Revert unit test
6691b03
@BloodAxe
Improve model device handling
75bcd46
Louis-Dupont
Louis-Dupont reviewed Jul 31, 2023
View reviewed changes
Copy link
Contributor

@Louis-Dupont Louis-Dupont left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just left some comments, I'll review more in-depth once it's open

src/super_gradients/module_interfaces/exportable_detector.py Outdated Show resolved Hide resolved
src/super_gradients/module_interfaces/module_interfaces.py Show resolved Hide resolved
@BloodAxe BloodAxe marked this pull request as ready for review August 3, 2023 09:16
shaydeci
shaydeci approved these changes Aug 8, 2023
View reviewed changes
Copy link
Collaborator

@shaydeci shaydeci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BloodAxe BloodAxe merged commit 24a708f into master Aug 8, 2023
7 checks passed
@BloodAxe BloodAxe deleted the feature/SG-1001 branch August 8, 2023 12:57
@BloodAxe BloodAxe mentioned this pull request Aug 10, 2023
Closed
Louis-Dupont pushed a commit that referenced this pull request Aug 15, 2023
@BloodAxe @Louis-Dupont
New Export API (#1318)
3f53819 
* Designing export API

* Export WIP

* ONNX NMS

* Export WIP

* Refactor test and move benchmark API to functino

* Export WIP

* Make the top_k a constant and not variable since TRT export does not work with dynamic top_k

* Refactor test and move benchmark API to functino

* Added option to change the output format

* Refactor test and move benchmark API to functino

* Added option to change the output format

* Refactor test and move benchmark API to functino

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Remove unused classes

* Remove unused classes

* Remove unused classes

* Remove unused classes

* Fixing export to FP16

* Fixing export to FP16

* Improve output of the benchmark result

* Improve device handling when exporting NMS

* Improve device handling when exporting NMS

* Fix nms format conversion modules export

* Revert unit test

* Improve model device handling

* Adding docs

* Adding docs

* Adding docs

* Adding docs

* Address TODO's after code review

* Added check whether model is already quantized

* Install pytorch quantization package

* Added printin of user-friendly description on how to use the exported model

* Update docs

* Update docs

* Uninstall SG

* Added onnx_graphsurgeon

* Added onnx_graphsurgeon

* Put extra index url at the top

* Put extra index url before the package that requires it

* Fix --index-url to --extra-index-url

* get_requirements to handle --extra-index-url correctly

* Made method draw_box_title public

* Fix tests

* Fix missing HasPredict for BaseClassifier model

* Make quantization parameters overridable
Louis-Dupont added a commit that referenced this pull request Aug 22, 2023
@Louis-Dupont @BloodAxe
@pranoyr @ofrimasad @shaydeci
Feature/sg 1027 checkpoint directory refacto (#1401)
0b4710a 
* wip-S

* wip

* fix bug where resume would crash if latest run doesnt include latest_ckpt

* remove unwanted change + copy hydra

* minor changes

* fix test

* add tests

* fix test

* New Export API (#1318)

* Designing export API

* Export WIP

* ONNX NMS

* Export WIP

* Refactor test and move benchmark API to functino

* Export WIP

* Make the top_k a constant and not variable since TRT export does not work with dynamic top_k

* Refactor test and move benchmark API to functino

* Added option to change the output format

* Refactor test and move benchmark API to functino

* Added option to change the output format

* Refactor test and move benchmark API to functino

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Fixing export to make it TRT friendly

* Remove unused classes

* Remove unused classes

* Remove unused classes

* Remove unused classes

* Fixing export to FP16

* Fixing export to FP16

* Improve output of the benchmark result

* Improve device handling when exporting NMS

* Improve device handling when exporting NMS

* Fix nms format conversion modules export

* Revert unit test

* Improve model device handling

* Adding docs

* Adding docs

* Adding docs

* Adding docs

* Address TODO's after code review

* Added check whether model is already quantized

* Install pytorch quantization package

* Added printin of user-friendly description on how to use the exported model

* Update docs

* Update docs

* Uninstall SG

* Added onnx_graphsurgeon

* Added onnx_graphsurgeon

* Put extra index url at the top

* Put extra index url before the package that requires it

* Fix --index-url to --extra-index-url

* get_requirements to handle --extra-index-url correctly

* Made method draw_box_title public

* Fix tests

* Fix missing HasPredict for BaseClassifier model

* Make quantization parameters overridable

* Feature/sg 000 fix predict in pose estimation (#1358)

* Update readme

* Fix small bug in __repr__ implementation of KeypointsImageToTensor

* Test

* Test

* Test

* Test

* Test

* Test

* Make graphsurgeon an optional

* Make graphsurgeon an optional

* Properly handle imports of optional packages

* Added empty __init__.py files

* Do imports of gs inside the export call

* Do imports of gs inside the export call

* Fix DEKR's missing HasPredict interface

* Update notebook & example doc to reflect changes in imports & function names

* Update readme

* Put back images

* add model export (#1362)

* fix (#1367)

* fix

* add spacing

* Feature/sg 000 propagate imagenet dataset params (#1368)

* Propagate default dataset processing params for other classification models

* Fix bug in predict pipeline (Softmax was computed along batch dimension AFTER taking max along classes dimension)

* Added more classification models to test

* Doc changes (#1253)

* num classes specified was wrong

* wrong num_classes specified

---------

Co-authored-by: Ofri Masad <ofrimasad@users.noreply.github.com>
Co-authored-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>

* Summarize models, losses & metrics for segmentation (#1354)

* Summarize models, losses & metrics

* Added troubleshoothing section

* Feature/sg 000 fix import of onnx graphsurgeon (#1359)

* Update readme

* Fix small bug in __repr__ implementation of KeypointsImageToTensor

* Test

* Test

* Test

* Test

* Test

* Test

* Make graphsurgeon an optional

* Make graphsurgeon an optional

* Properly handle imports of optional packages

* Added empty __init__.py files

* Do imports of gs inside the export call

* Do imports of gs inside the export call

* Fix DEKR's missing HasPredict interface

* Update notebook & example doc to reflect changes in imports & function names

* Update readme

* Put back images

* Install onnx_graphsurgeon in CI

* Install onnx_graphsurgeon in CI

* Fix version of ONNX-GS installed in CI and installed on-demand

* Fix arange_cpu not implemented for Half

* Fix arange_cpu not implemented for Half

* Fix graph merging for old pytorch (1.12) that crashed because of nodes with duplicate names

* Feature/sg 1047 predict od with labels (#1365)

* cleanup start

* added docs

* added tests

* added tests + fix yolox

* fixed ppyoloe

* fixed ppyoloe

* small ppyoloe prep model for conversion fix

* small ppyoloe prep model for conversion fix

* fixed image_i_object_count ref docs

* alligned box thickness

* renamed vars in example

* changed statement and added len verification

* fixed predictions docs

* fixed pipelines docs

* removed gt text from plots

* removed gt text from plots

* refactored predict with labels to use show/save

* Feature/sg 1033 fix yolox anchors (#1369)

* Update readme

* Fix small bug in __repr__ implementation of KeypointsImageToTensor

* Test

* Test

* Test

* Test

* Test

* Test

* Make graphsurgeon an optional

* Make graphsurgeon an optional

* Properly handle imports of optional packages

* Added empty __init__.py files

* Do imports of gs inside the export call

* Do imports of gs inside the export call

* Fix DEKR's missing HasPredict interface

* Update notebook & example doc to reflect changes in imports & function names

* Update readme

* Put back images

* Install onnx_graphsurgeon in CI

* Install onnx_graphsurgeon in CI

* Working prototype of YoloX fix of Anchors that can load model weights as well

* Added more tests for detection predict() and yolox checkpoint loading

* Fix version of ONNX-GS installed in CI and installed on-demand

* Added docs

* Added docs

* Added docs

* Remove leftover

* Set ignore_errors=True to trainer test and declare why

* Fix bug in maybe_remove_module_prefix

* version bumped (#1374)

Co-authored-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>

* Adding pose estimation to the readme (#1375)

* Update readme

* Fix small bug in __repr__ implementation of KeypointsImageToTensor

* Update pose estimation image

* Fix link

* Remove broken link that we can't recover where it was pointing to (#1376)

* test on first run

* improve doc with new checkpoint run logic

* add doc for Trainer

* update doc

* test sweeper

* test sweeper

* test sweeper

* test sweeper

* test sweeper

* test sweeper

* test direcrlt call

* fix indentation

* fix indentation

* move test to integration test

* fix

* add makefiletest

* iupdate

* remove test from base ci test

---------

Co-authored-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Pranoy Radhakrishnan <pranoyalkr@gmail.com>
Co-authored-by: Ofri Masad <ofrimasad@users.noreply.github.com>
Co-authored-by: Shay Aharon <80472096+shaydeci@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Louis-Dupont Louis-Dupont Louis-Dupont left review comments

@shaydeci shaydeci shaydeci approved these changes

@ofrimasad ofrimasad Awaiting requested review from ofrimasad ofrimasad is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@BloodAxe @Louis-Dupont @shaydeci