Skip to content

Commit

Permalink
Update image_classifier/densenet-161 to include torch.compile (#3200)
Browse files Browse the repository at this point in the history
* Update image_classifier/densenet-161 to include torch.compile

* minor

---------

Co-authored-by: Emma Liu <zichangliu@fb.com>
  • Loading branch information
lzcemma and Emma Liu committed Jun 21, 2024
1 parent 079ff7b commit 688f09e
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 2 deletions.
35 changes: 33 additions & 2 deletions examples/image_classifier/densenet_161/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,34 @@
#### TorchServe inference with torch.compile of densenet161 model
This example shows how to take eager model of `densenet161`, configure TorchServe to use `torch.compile` and run inference using `torch.compile`

Change directory to the examples directory
`cd examples/image_classifier/densenet_161`

`torch.compile` supports a variety of config and the performance you get can vary based on the config. You can find the various options [here](https://pytorch.org/docs/stable/generated/torch.compile.html).

Sample command to start torchserve with torch.compile:

```bash
wget https://download.pytorch.org/models/densenet161-8d451a50.pth
mkdir model_store
torch-model-archiver --model-name densenet161 --version 1.0 --model-file model.py --serialized-file densenet161-8d451a50.pth --export-path model_store --extra-files ../../image_classifier/index_to_name.json --handler image_classifier --config-file model-config.yaml -f
torchserve --start --ncs --model-store model_store --models densenet161.mar
curl http://127.0.0.1:8080/predictions/densenet161 -T ../../image_classifier/kitten.jpg
```

produces the output

```
{
"tabby": 0.4664836823940277,
"tiger_cat": 0.4645617604255676,
"Egyptian_cat": 0.06619937717914581,
"lynx": 0.0012969186063855886,
"plastic_bag": 0.00022856894065625966
}
```


#### Sample commands to create a densenet eager mode model archive, register it on TorchServe and run image prediction

Run the commands given in following steps from the parent directory of the root of the repository. For example, if you cloned the repository into /home/my_path/serve, run the steps from /home/my_path/serve
Expand Down Expand Up @@ -35,8 +66,8 @@ model.eval()
example_input = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example_input)
traced_script_module.save("densenet161.pt")
```
```

* Use following commands to register Densenet161 torchscript model on TorchServe and run image prediction

```bash
Expand Down
5 changes: 5 additions & 0 deletions examples/image_classifier/densenet_161/model-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pt2:
compile:
enable: True
backend: inductor
mode: reduce-overhead

0 comments on commit 688f09e

Please sign in to comment.