[sparsezoo.analyze] Fix pathway such that it works for larger models #437

dsikka · 2024-02-08T20:33:50Z

Summary

For the following two tickets:

Analysis fails because we fail to start an inference session for basically any model with >2gb of data when starting the session with a ModelProto as opposed to using a path to the onnx model
To get around this, after making the required modifications, save the model and pass that to the inference session

src/sparsezoo/analyze_v2/model_analysis.py

src/sparsezoo/utils/onnx/external_data.py

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

…437) * fix analyze to work with larger models * update for failing tests; add comments * Update src/sparsezoo/utils/onnx/external_data.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> --------- Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom> Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

…437) (#438) * fix analyze to work with larger models * update for failing tests; add comments * Update src/sparsezoo/utils/onnx/external_data.py --------- Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom> Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* `RegistryMixin` improved alias management (#404) * initial commit * add docstrings * simplify * hardening * refactor * format registry lookup strings to be lowercases * standardise aliases * Move evaluator registry (#411) * More control over external data size (#412) * When splitting external data, avoid renaming `model.data` to `model.data.1` if only one external data file gets eventually saved (#414) * [model.download] fix function returning nothing (#420) * [BugFix] Path not expanded (#418) * [Fix] Allow for processing Path in the sparsezoo analysis (#417) * Raise TypeError instead of ValueError (#426) * Fix misleading docstring (#416) Add test * add support for benchmark.yaml (#415) * add support for benchmark.yaml recent zoo models use `benchmark.yaml` instead of `benchmarks.yaml`. adding this additional pathway so `benchmark.yaml` is downloaded in the bulk model download * update files filter * fix tests --------- Co-authored-by: dbogunowicz <damian@neuralmagic.com> * [BugFix] Add analyze to init (#421) * Add analyze to init * Move onnxruntime to deps * Print model analysis (#423) * [model.download] fix function returning nothing (#420) * [BugFix] Path not expanded (#418) * print model-analysis * [Fix] Allow for processing Path in the sparsezoo analysis (#417) * add print statement at the end of cli run --------- Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com> Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Omit scalar weight (#424) * ommit scalar weights: * remove unwanted files * comment * Update src/sparsezoo/utils/onnx/analysis.py Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> --------- Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> --------- Co-authored-by: George <george@neuralmagic.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com> Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> * update analyze help message for correctness (#432) * initial commit (#430) * [sparsezoo.analyze] Fix pathway such that it works for larger models (#437) * fix analyze to work with larger models * update for failing tests; add comments * Update src/sparsezoo/utils/onnx/external_data.py Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> --------- Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom> Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> * Delete hehe.py (#439) * Download deployment dir for llms (#435) * Download deployment dir for llms * Use path instead of download * only set save_as_external_data to true if the model originally had external data (#442) * Add Channel Wise Quantization Support (#441) * Chunk download (#429) * chunk download, break down into 10 * lint * threads download * draft * chunk download draft * job based download and combining/deleteing chunks * delete old code * lint * fix num jobs if file_size is less than the chunk size * doc string and return types * test * lint * fix type hints (#445) * fix bug if the value is a dict (#447) * [deepsparse.analyze] Fix v1 functionality to work with llms (#451) * fix equivalent changes made to analyze_v2 such that inference session works for llms; update wanrings to be debug printouts * typo * overwrite file (#450) Co-authored-by: 21 <a21@21s-MacBook-Pro.local> * Adds a `numpy_array_representer` to yaml (#454) on runtime, to avoid serialization issues * Avoid division by zero (#457) Avoid log of zero * op analysis total counts had double sparse counts (#461) * Rename legacy analyze to analyze_v1 (#459) * Fixing Quant % Calcuation (#462) * initial fix * style * Include Sparsity in Size Calculation (#463) * initial fix * style * incorporate sparsity into size calculation * quality * op analysis total counts had double sparse counts (#461) * Fixing Quant % Calcuation (#462) * initial fix * style * Include Sparsity in Size Calculation (#463) * initial fix * style * incorporate sparsity into size calculation * quality * Revert "Merge branch 'main' into analyze_cherry_picks" This reverts commit 509fa1a, reversing changes made to 08f94c4. --------- Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com> Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com> Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> Co-authored-by: dbogunowicz <damian@neuralmagic.com> Co-authored-by: George <george@neuralmagic.com> Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom> Co-authored-by: 21 <a21@21s-MacBook-Pro.local>

Dipika Sikka and others added 2 commits February 8, 2024 20:30

fix analyze to work with larger models

3105bfb

Merge branch 'main' into fix_analyze

c27bd28

bfineran reviewed Feb 8, 2024

View reviewed changes

src/sparsezoo/analyze_v2/model_analysis.py Show resolved Hide resolved

update for failing tests; add comments

d26b23b

dsikka force-pushed the fix_analyze branch from c6bcbd0 to d26b23b Compare February 8, 2024 20:58

dbogunowicz reviewed Feb 9, 2024

View reviewed changes

src/sparsezoo/utils/onnx/external_data.py Outdated Show resolved Hide resolved

dbogunowicz previously approved these changes Feb 9, 2024

View reviewed changes

bfineran previously approved these changes Feb 9, 2024

View reviewed changes

dsikka dismissed stale reviews from bfineran and dbogunowicz via a8c5cfb February 9, 2024 15:36

Update src/sparsezoo/utils/onnx/external_data.py

a8c5cfb

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

dsikka requested review from dbogunowicz and bfineran February 9, 2024 15:36

bfineran approved these changes Feb 9, 2024

View reviewed changes

horheynm approved these changes Feb 9, 2024

View reviewed changes

rahul-tuli approved these changes Feb 9, 2024

View reviewed changes

dsikka merged commit c1a096f into main Feb 9, 2024
4 checks passed

dsikka deleted the fix_analyze branch February 9, 2024 18:48

dsikka mentioned this pull request Feb 9, 2024

[Cherry Pick] Fix sparsezoo.analyze cli command to work with large llms #438

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sparsezoo.analyze] Fix pathway such that it works for larger models #437

[sparsezoo.analyze] Fix pathway such that it works for larger models #437

dsikka commented Feb 8, 2024 •

edited

Loading

[sparsezoo.analyze] Fix pathway such that it works for larger models #437

[sparsezoo.analyze] Fix pathway such that it works for larger models #437

Conversation

dsikka commented Feb 8, 2024 • edited Loading

Summary

dsikka commented Feb 8, 2024 •

edited

Loading