Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

wasi-nn: add named models #6854

Merged
merged 3 commits into from
Aug 23, 2023
Merged

wasi-nn: add named models #6854

merged 3 commits into from
Aug 23, 2023

Commits on Aug 22, 2023

  1. wasi-nn: add [named models] 

    This change adds a way to retrieve preloaded ML models (i.e., "graphs"
in wasi-nn terms) from a registry. The wasi-nn specification includes a
new function, `load_by_name`, that can be used to access these models
more efficiently than before; previously, a user's only option was to
read/download/etc. all of the bytes of an ML model and pass them to the
`load` function.

[named models]: WebAssembly/wasi-nn#36

In Wasmtime's implementation of wasi-nn, we call the registry that holds
the models a `GraphRegistry`. We include a simplistic `InMemoryRegistry`
for use in the Wasmtime CLI (more on this later) but the idea is that
production use will involve some more complex caching and thus a new
implementation of a registry--a `Box<dyn GraphRegistry>`--passed into
the wasi-nn context. Note that, because we now must be able to `clone` a
graph out of the registry and into the "used graphs" table, the OpenVINO
`BackendGraph` is updated to be easier to copy around.

To allow experimentation with this "preload a named model"
functionality, this change also adds a new Wasmtime CLI flag: `--graph
<encoding>:<host dir>`. Wasmtime CLI users can now preload a model from
a directory; the directory `basename` is used as the model name. Loading
models from a directory is probably not desired in Wasmtime embeddings
so it is cordoned off into a separate `BackendFromDir` extension trait.
    @abrown
    abrown committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    7bfc3cf View commit details
    Browse the repository at this point in the history

  2. wasi-nn: add "named model" test 

    Add a new example crate which loads a model by name and performs image
classification. It uses the same MobileNet model as the existing test
but a new version of the Rust bindings. The new crate is built and run
with the new CLI flag in the `ci/run-wasi-nn-example.sh` script.

prtest:full
    @abrown
    abrown committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    c03abac View commit details
    Browse the repository at this point in the history

  3. review: rename --graph to --wasi-nn-graph

    @abrown
    abrown committed Aug 22, 2023
    Configuration menu
    Copy the full SHA
    6cff07c View commit details
    Browse the repository at this point in the history