-
Notifications
You must be signed in to change notification settings - Fork 175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[V2 Pipeline] Middleware manager #1471
Conversation
…leware-manager-after-v2-merge
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM pending unresolved comment
c7fe1a7
to
f311e15
Compare
…ralmagic/deepsparse into v2/middleware-manager-after-v2-merge
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved - there are comments that need to be deleted however
also - as discussed we need to cut down on the number of times we instantiate the actual middleware callable classes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as other PRs, could we merge in pipeline.py and make sure everything works as expected?
Description
Adds ability to run middlewares for Pipeline and Operator callables
Problem
No existing way to just get the timings of each callable, add a generic manager for now, TimerManager in a separate pr
Solution
Create middleware manager and add a chain of callables to call middlewares before calling the callables
Design
Very similar to Starlette (Fastapi) middleware. Before calling the callables for Pipeline or Operator, add a "gadget" where the gadget logic is called before and after the callable.
One note is that Pipeline and Operator do not rely on the middleware. It doesnt know it exists. It is just a helper.
Usage
Check tests,
Testing
Checks the functionality of middleware and middleware managers work as expected