Skip to content

Commit

Permalink
Update documentation for vgg16 to use torch.compile (#3211)
Browse files Browse the repository at this point in the history
  • Loading branch information
ijkilchenko committed Jun 27, 2024
1 parent ce6be2c commit ffcbee9
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 0 deletions.
35 changes: 35 additions & 0 deletions examples/image_classifier/vgg_16/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,38 @@
#### TorchServe inference with torch.compile of vgg-16 model
This example shows how to take eager model of `vgg-16`, configure TorchServe to use `torch.compile` and run inference using `torch.compile`

Change directory to the root directory of this project.

`torch.compile` supports a variety of config and the performance you get can vary based on the config. You can find the various options [here](https://pytorch.org/docs/stable/generated/torch.compile.html).

Sample command to start torchserve with torch.compile:

```bash
wget https://download.pytorch.org/models/vgg16-397923af.pth
torch-model-archiver --model-name vgg16 --version 1.0 --model-file ./examples/image_classifier/vgg_16/model.py --serialized-file vgg16-397923af.pth --handler ./examples/image_classifier/vgg_16/vgg_handler.py --extra-files ./examples/image_classifier/index_to_name.json --config-file ./examples/image_classifier/vgg_16/model-config.yaml -f
mkdir model_store
mv vgg16.mar model_store/vgg16_compiled.mar
torchserve --start --model-store model_store --models vgg16=vgg16_compiled.mar
```

Now in another terminal, run

```bash
curl http://127.0.0.1:8080/predictions/densenet161 -T ../../image_classifier/kitten.jpg
```

which produces the output

```
{
"tiger_cat": 0.44697248935699463,
"tabby": 0.4408799707889557,
"Egyptian_cat": 0.05904558673501015,
"tiger": 0.02059638872742653,
"lynx": 0.009934586472809315
}
```

#### Sample commands to create a vgg-16 eager mode model archive, register it on TorchServe and run image prediction

Run the commands given in following steps from the parent directory of the root of the repository. For example, if you cloned the repository into /home/my_path/serve, run the steps from /home/my_path/serve
Expand Down
3 changes: 3 additions & 0 deletions examples/image_classifier/vgg_16/model-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
pt2:
compile:
enable: True

0 comments on commit ffcbee9

Please sign in to comment.