diff --git a/examples/image_classifier/vgg_16/README.md b/examples/image_classifier/vgg_16/README.md index 2a9459e372..fa52c4ced0 100644 --- a/examples/image_classifier/vgg_16/README.md +++ b/examples/image_classifier/vgg_16/README.md @@ -1,3 +1,38 @@ +#### TorchServe inference with torch.compile of vgg-16 model +This example shows how to take eager model of `vgg-16`, configure TorchServe to use `torch.compile` and run inference using `torch.compile` + +Change directory to the root directory of this project. + +`torch.compile` supports a variety of config and the performance you get can vary based on the config. You can find the various options [here](https://pytorch.org/docs/stable/generated/torch.compile.html). + +Sample command to start torchserve with torch.compile: + +```bash +wget https://download.pytorch.org/models/vgg16-397923af.pth +torch-model-archiver --model-name vgg16 --version 1.0 --model-file ./examples/image_classifier/vgg_16/model.py --serialized-file vgg16-397923af.pth --handler ./examples/image_classifier/vgg_16/vgg_handler.py --extra-files ./examples/image_classifier/index_to_name.json --config-file ./examples/image_classifier/vgg_16/model-config.yaml -f +mkdir model_store +mv vgg16.mar model_store/vgg16_compiled.mar +torchserve --start --model-store model_store --models vgg16=vgg16_compiled.mar +``` + +Now in another terminal, run + +```bash +curl http://127.0.0.1:8080/predictions/densenet161 -T ../../image_classifier/kitten.jpg +``` + +which produces the output + +``` +{ + "tiger_cat": 0.44697248935699463, + "tabby": 0.4408799707889557, + "Egyptian_cat": 0.05904558673501015, + "tiger": 0.02059638872742653, + "lynx": 0.009934586472809315 +} +``` + #### Sample commands to create a vgg-16 eager mode model archive, register it on TorchServe and run image prediction Run the commands given in following steps from the parent directory of the root of the repository. For example, if you cloned the repository into /home/my_path/serve, run the steps from /home/my_path/serve diff --git a/examples/image_classifier/vgg_16/model-config.yaml b/examples/image_classifier/vgg_16/model-config.yaml new file mode 100644 index 0000000000..974888d25b --- /dev/null +++ b/examples/image_classifier/vgg_16/model-config.yaml @@ -0,0 +1,3 @@ +pt2: + compile: + enable: True