Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix] Allow for processing Path in the sparsezoo analysis #417

Merged
merged 1 commit into from
Feb 2, 2024
Merged

[Fix] Allow for processing Path in the sparsezoo analysis #417

merged 1 commit into from
Feb 2, 2024

Conversation

dbogunowicz
Copy link
Contributor

@dbogunowicz dbogunowicz commented Feb 1, 2024 •
edited
Loading

onnx.load() has very limited support for loading models from the Path objects, especially when loading involves properly searching for external data. This causes the issue: https://app.asana.com/0/1206109050183159/1206486977947981/f - onnx tries to look for external data in the cwd and not in the directory where model.onnx is.

The solution is to refrain from using onnx.load(...) in favor of using our internal helper func load_model. That is slightly enhanced version of the func, that also works with Path objects.

Now:

deepsparse.analyze /home/damian/.cache/sparsezoo/neuralmagic/codellama-7b-evolcodealpaca_codellama_pretrain-pruned60_quantized/deployment

Does not fail on model loading:

2024-02-01 15:51:49 __main__     INFO     Starting Analysis ...
INFO:__main__:Starting Analysis ...
INFO:root:Loading `model.onnx` from deployment directory /home/damian/.cache/sparsezoo/neuralmagic/codellama-7b-evolcodealpaca_codellama_pretrain-pruned60_quantized/deployment
EP Error narrowing_error when using ['CPUExecutionProvider']
Falling back to ['CPUExecutionProvider'] and retrying.
WARNING:sparsezoo.utils.node_inference:Extracting shapes using ONNX Runtime session failed: narrowing_error
WARNING:sparsezoo.utils.node_inference:Falling back to ONNX shape_inference
...

Note: it fails later on, but for that we have a separate ticket

@dbogunowicz dbogunowicz changed the title [Fix] Allow for processing Path in the sparsezoo analyse [Fix] Allow for processing Path in the sparsezoo analysis Feb 1, 2024
@bfineran bfineran merged commit 1a9ee4b into main Feb 2, 2024
4 checks passed
@bfineran bfineran deleted the fix/damian/silly_onnx_no_path branch February 2, 2024 15:29
@dbogunowicz dbogunowicz mentioned this pull request Feb 2, 2024
Merged
dbogunowicz added a commit that referenced this pull request Feb 2, 2024
@dbogunowicz
[Fix] Allow for processing Path in the sparsezoo analysis (#417)
53d8fb6
bfineran pushed a commit that referenced this pull request Feb 2, 2024
@dbogunowicz
[Fix] Allow for processing Path in the sparsezoo analysis (#417) (#425)
fe4d664
horheynm added a commit that referenced this pull request Feb 5, 2024
@horheynm @dsikka
@rahul-tuli @dbogunowicz
Print model analysis (#423)
df35785 
* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* print model-analysis

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* add print statement at the end of cli run

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
horheynm added a commit that referenced this pull request Feb 6, 2024
@rahul-tuli @horheynm
@dsikka @dbogunowicz @bfineran
[BugFix] Add analyze to init (#421)
fc00994 
* Add analyze to init

* Move onnxruntime to deps

* Print model analysis (#423)

* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* print model-analysis

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* add print statement at the end of cli run

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* Omit scalar weight (#424)

* ommit scalar weights:

* remove unwanted files

* comment

* Update src/sparsezoo/utils/onnx/analysis.py

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: George <george@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
rahul-tuli added a commit that referenced this pull request Feb 6, 2024
@rahul-tuli @horheynm
@dsikka @dbogunowicz @bfineran
[BugFix] Add analyze to init (#421)
fdabe2c 
* Add analyze to init

* Move onnxruntime to deps

* Print model analysis (#423)

* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* print model-analysis

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* add print statement at the end of cli run

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* Omit scalar weight (#424)

* ommit scalar weights:

* remove unwanted files

* comment

* Update src/sparsezoo/utils/onnx/analysis.py

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: George <george@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
bfineran added a commit that referenced this pull request Feb 6, 2024
@rahul-tuli @horheynm
@dsikka @dbogunowicz @bfineran
[BugFix] Add analyze to init (#421) (#434)
071cbf1 
* Add analyze to init

* Move onnxruntime to deps

* Print model analysis (#423)

* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* print model-analysis

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* add print statement at the end of cli run

---------





* Omit scalar weight (#424)

* ommit scalar weights:

* remove unwanted files

* comment

* Update src/sparsezoo/utils/onnx/analysis.py



---------



---------

Co-authored-by: George <george@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
Satrat added a commit that referenced this pull request Feb 22, 2024
@Satrat @dbogunowicz
@rahul-tuli @dsikka @bfineran @horheynm
[Cherry Picks] Analyze Bug Fixes (Updated) (#465)
edf177e 
* `RegistryMixin` improved alias management (#404)

* initial commit

* add docstrings

* simplify

* hardening

* refactor

* format registry lookup strings to be lowercases

* standardise aliases

* Move evaluator registry (#411)

* More control over external data size (#412)

* When splitting external data, avoid renaming `model.data` to `model.data.1` if only one external data file gets eventually saved (#414)

* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* Raise TypeError instead of ValueError (#426)

* Fix misleading docstring (#416)

Add test

* add support for benchmark.yaml (#415)

* add support for benchmark.yaml

recent zoo models use `benchmark.yaml` instead of `benchmarks.yaml`. adding this additional pathway so `benchmark.yaml` is downloaded in the bulk model download

* update files filter

* fix tests

---------

Co-authored-by: dbogunowicz <damian@neuralmagic.com>

* [BugFix] Add analyze to init (#421)

* Add analyze to init

* Move onnxruntime to deps

* Print model analysis (#423)

* [model.download] fix function returning nothing (#420)

* [BugFix] Path not expanded (#418)

* print model-analysis

* [Fix] Allow for processing Path in the sparsezoo analysis (#417)

* add print statement at the end of cli run

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* Omit scalar weight (#424)

* ommit scalar weights:

* remove unwanted files

* comment

* Update src/sparsezoo/utils/onnx/analysis.py

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

---------

Co-authored-by: George <george@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

* update analyze help message for correctness (#432)

* initial commit (#430)

* [sparsezoo.analyze] Fix pathway such that it works for larger models (#437)

* fix analyze to work with larger models

* update for failing tests; add comments

* Update src/sparsezoo/utils/onnx/external_data.py

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

---------

Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom>
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* Delete hehe.py (#439)

* Download deployment dir for llms (#435)

* Download deployment dir for llms

* Use path instead of download

* only set save_as_external_data to true if the model originally had external data (#442)

* Add Channel Wise Quantization Support (#441)

* Chunk download (#429)

* chunk download, break down into 10

* lint

* threads download

* draft

* chunk download draft

* job based download and combining/deleteing chunks

* delete old code

* lint

* fix num jobs if file_size is less than the chunk size

* doc string and return types

* test

* lint

* fix type hints (#445)

* fix bug if the value is a dict (#447)

* [deepsparse.analyze] Fix v1 functionality to  work with llms (#451)

* fix equivalent changes made to analyze_v2 such that inference session works for llms; update wanrings to be debug printouts

* typo

* overwrite file (#450)

Co-authored-by: 21 <a21@21s-MacBook-Pro.local>

* Adds a `numpy_array_representer` to yaml (#454)

on runtime, to avoid serialization issues

* Avoid division by zero (#457)

Avoid log of zero

* op analysis total counts had double sparse counts (#461)

* Rename legacy analyze to analyze_v1 (#459)

* Fixing Quant % Calcuation (#462)

* initial fix

* style

* Include Sparsity in Size Calculation (#463)

* initial fix

* style

* incorporate sparsity into size calculation

* quality

* op analysis total counts had double sparse counts (#461)

* Fixing Quant % Calcuation (#462)

* initial fix

* style

* Include Sparsity in Size Calculation (#463)

* initial fix

* style

* incorporate sparsity into size calculation

* quality

* Revert "Merge branch 'main' into analyze_cherry_picks"

This reverts commit 509fa1a, reversing
changes made to 08f94c4.

---------

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
Co-authored-by: dbogunowicz <damian@neuralmagic.com>
Co-authored-by: George <george@neuralmagic.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.coom>
Co-authored-by: 21 <a21@21s-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bfineran bfineran bfineran approved these changes

@rahul-tuli rahul-tuli Awaiting requested review from rahul-tuli

@derekk-nm derekk-nm Awaiting requested review from derekk-nm

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@dbogunowicz @bfineran