Skip to content

Commit

Permalink
squeezenet torch.compile example (#3277)
Browse files Browse the repository at this point in the history
Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>
  • Loading branch information
wdvr and agunapal authored Aug 2, 2024
1 parent 3233f44 commit 24e2492
Showing 1 changed file with 39 additions and 0 deletions.
39 changes: 39 additions & 0 deletions examples/image_classifier/squeezenet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,45 @@ torchserve --start --model-store model_store --models squeezenet1_1=squeezenet1_
curl http://127.0.0.1:8080/predictions/squeezenet1_1 -T examples/image_classifier/kitten.jpg
```

#### TorchServe inference with torch.compile of squeezenet model
This example shows how to take eager model of `squeezenet`, configure TorchServe to use `torch.compile` and run inference using `torch.compile`.

Change directory to the examples directory
Ex: `cd examples/image_classifier/squeezenet`

##### torch.compile config
`torch.compile` supports a variety of config and the performance you get can vary based on the config. You can find the various options [here](https://pytorch.org/docs/stable/generated/torch.compile.html).

In this example, we use the following config

```
echo "pt2 : {backend: inductor, mode: reduce-overhead}" > model-config.yaml
```

##### Sample commands to create a Squeezenet torch.compile model archive, register it on TorchServe and run image prediction

```bash
wget https://download.pytorch.org/models/squeezenet1_1-b8a52dc0.pth
torch-model-archiver --model-name squeezenet --version 1.0 --model-file model.py --serialized-file squeezenet1_1-b8a52dc0.pth --handler image_classifier --extra-files ../index_to_name.json --config-file model-config.yaml
mkdir model_store
mv squeezenet.mar model_store/
torchserve --start --model-store model_store --models squeezenet=squeezenet.mar
curl http://127.0.0.1:8080/predictions/squeezenet -T ../kitten.jpg
```

produces the output
```
{
"tabby": 0.2751994729042053,
"lynx": 0.2546878755092621,
"tiger_cat": 0.2425432652235031,
"Egyptian_cat": 0.22137290239334106,
"cougar": 0.0022544628009200096
}%
```



#### TorchScript example using alexnet image classifier:

* Save the Squeezenet1_1 model as an executable script module or a traced script:
Expand Down

0 comments on commit 24e2492

Please sign in to comment.