Security documentation update

.github/workflows/docker-ci.yaml

@@ -29,7 +29,7 @@ jobs:
         working-directory: docker
         run: |
           IMAGE_TAG=test-image-${{ matrix.python-version }}
-          ./build_image.sh -py "${{ matrix.python-version }}" -t "${IMAGE_TAG}" -s -r
+          ./build_image.sh -py "${{ matrix.python-version }}" -t "${IMAGE_TAG}" -b ${GITHUB_HEAD_REF} -s
           echo "IMAGE_TAG=${IMAGE_TAG}" >> $GITHUB_OUTPUT
 
       - name: Container Healthcheck

README.md

@@ -1,5 +1,9 @@
+# ❗ANNOUNCEMENT: Security Changes❗
+TorchServe now enforces token authorization enabled and model API control disabled at start. These security features are designed to address the concern of unwanted API calls and to prevent potential malicious code from being introduced to the model server. Check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 # TorchServe
 
+
 ![Nightly build](https://github.com/pytorch/serve/actions/workflows/torchserve-nightly-build.yml/badge.svg)
 ![Docker Nightly build](https://github.com/pytorch/serve/actions/workflows/docker-nightly-build.yml/badge.svg)
 ![Benchmark Nightly](https://github.com/pytorch/serve/actions/workflows/benchmark_nightly.yml/badge.svg)

benchmarks/utils/system_under_test.py

@@ -116,7 +116,7 @@ def start(self):
         click.secho("*Starting local Torchserve instance...", fg="green")
 
         ts_cmd = (
-            f"torchserve --start --model-store {self.execution_params['tmp_dir']}/model_store --model-api-enabled --disable-token "
+            f"torchserve --start --model-store {self.execution_params['tmp_dir']}/model_store --enable-model-api --disable-token-auth "
             f"--workflow-store {self.execution_params['tmp_dir']}/wf_store "
             f"--ts-config {self.execution_params['tmp_dir']}/benchmark/conf/{self.execution_params['config_properties_name']} "
             f" > {self.execution_params['tmp_dir']}/benchmark/logs/model_metrics.log"
@@ -195,7 +195,7 @@ def start(self):
             f"docker run {self.execution_params['docker_runtime']} {backend_profiling} --name ts --user root -p "
             f"127.0.0.1:{inference_port}:{inference_port} -p 127.0.0.1:{management_port}:{management_port} "
             f"-v {self.execution_params['tmp_dir']}:/tmp {enable_gpu} -itd {docker_image} "
-            f'"torchserve --start --model-store /home/model-server/model-store --model-api-enabled --disable-token '
+            f'"torchserve --start --model-store /home/model-server/model-store --enable-model-api --disable-token-auth '
             f"\--workflow-store /home/model-server/wf-store "
             f"--ts-config /tmp/benchmark/conf/{self.execution_params['config_properties_name']} > "
             f'/tmp/benchmark/logs/model_metrics.log"'

docker/README.md

@@ -1,3 +1,6 @@
+## Security Changes
+TorchServe now enforces token authorization and model API control by default. Check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 ### Deprecation notice:
 [Dockerfile.neuron.dev](https://github.com/pytorch/serve/blob/master/docker/Dockerfile.neuron.dev) has been deprecated. Please refer to [deep learning containers](https://github.com/aws/deep-learning-containers/blob/master/available_images.md) repository for neuron torchserve containers.
 

docker/dockerd-entrypoint.sh

@@ -1,9 +1,10 @@
 #!/bin/bash
 set -e
 
+
 if [[ "$1" = "serve" ]]; then
     shift 1
-    torchserve --start --ts-config /home/model-server/config.properties --disable-token
+    torchserve --start --ts-config /home/model-server/config.properties --disable-token-auth
 else
     eval "$@"
 fi

docker/test_container_model_prediction.sh

@@ -19,7 +19,7 @@ torch-model-archiver \
     --handler=/home/model-server/mnist_handler.py \
     --export-path=/home/model-server/model-store
 
-torchserve --start --ts-config=/home/model-server/config.properties --models mnist=mnist.mar --disable-token
+torchserve --start --ts-config=/home/model-server/config.properties --models mnist=mnist.mar --disable-token-auth
 EOF
 
 echo "Starting container ${CONTAINER}"

docs/README.md

@@ -1,3 +1,6 @@
+# ❗ANNOUNCEMENT: Security Changes❗
+TorchServe now enforces token authorization enabled and model API control disabled at start. These security features are designed to address the concern of unwanted API calls and to prevent potential malicious code from being introduced to the model server. Check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 # TorchServe
 
 TorchServe is a performant, flexible and easy to use tool for serving PyTorch eager mode and torchscripted models.

docs/inference_api.md

@@ -2,6 +2,8 @@
 
 Inference API is listening on port 8080 and only accessible from localhost by default. To change the default setting, see [TorchServe Configuration](configuration.md).
 
+For all Inference API requests, TorchServe requires the correct Inference token to be included or token authorization must be disable. For more details see [token authorization documentation](./token_authorization_api.md)
+
 The TorchServe server supports the following APIs:
 
 * [API Description](#api-description) - Gets a list of available APIs and options

docs/management_api.md

@@ -8,9 +8,14 @@ TorchServe provides the following APIs that allows you to manage models at runti
 4. [Unregister a model](#unregister-a-model)
 5. [List registered models](#list-models)
 6. [Set default version of a model](#set-default-version)
+7. [Refresh tokens for token authorization](#token-authorization-api)
 
 The Management API listens on port 8081 and is only accessible from localhost by default. To change the default setting, see [TorchServe Configuration](./configuration.md).
 
+Management API for registering and deleting models is disabled by default. Add `--enable-model-api` to command line when running TorchServe to enable the use of these APIs. For more details and ways to enable see [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
+For all Management API requests, TorchServe requires the correct Management token to be included or token authorization must be disabled. For more details see [token authorization documentation](./token_authorization_api.md)
+
 Similar to the [Inference API](inference_api.md), the Management API provides a [API description](#api-description) to describe management APIs with the OpenAPI 3.0 specification.
 
 Alternatively, if you want to use KServe, TorchServe supports both v1 and v2 API. For more details please look into this [kserve documentation](https://github.com/pytorch/serve/tree/master/kubernetes/kserve)
@@ -19,6 +24,8 @@ Alternatively, if you want to use KServe, TorchServe supports both v1 and v2 API
 
 This API follows the [ManagementAPIsService.RegisterModel](https://github.com/pytorch/serve/blob/master/frontend/server/src/main/resources/proto/management.proto) gRPC API.
 
+To use this API after TorchServe starts, model API control has to be enabled. Add `--enable-model-api` to command line when running TorchServe to enable the use of this API. For more details see [model API control](./model_api_control.md)
+
 `POST /models`
 
 * `url` - Model archive download url. Supports the following locations:
@@ -441,6 +448,8 @@ print(customizedMetadata)
 
 This API follows the [ManagementAPIsService.UnregisterModel](https://github.com/pytorch/serve/blob/master/frontend/server/src/main/resources/proto/management.proto) gRPC API. It returns the status of a model in the ModelServer.
 
+To use this API after TorchServe starts, model API control has to be enabled. Add `--enable-model-api` to command line when running TorchServe to enable the use of this API. For more details see [model API control](./model_api_control.md)
+
 `DELETE /models/{model_name}/{version}`
 
 Use the Unregister Model API to free up system resources by unregistering specific version of a model from TorchServe:
@@ -522,3 +531,21 @@ curl -v -X PUT http://localhost:8081/models/noop/2.0/set-default
 ```
 
 The out is OpenAPI 3.0.1 json format. You use it to generate client code, see [swagger codegen](https://swagger.io/swagger-codegen/) for detail.
+
+## Token Authorization API
+
+TorchServe now enforces token authorization by default. Check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md).
+
+This API is used in order to generate a new key to replace either the management or inference key.
+
+Management Example:
+```
+curl localhost:8081/token?type=management -H "Authorization: Bearer {API Token}"
+```
+will replace the current management key in the key_file with a new one and will update the expiration time.
+
+Inference example:
+```
+curl localhost:8081/token?type=inference -H "Authorization: Bearer {API Token}"
+```
+will replace the current inference key in the key_file with a new one and will update the expiration time.

docs/model_control_mode.md → docs/model_api_control.md

@@ -1,28 +1,31 @@
-# Model Control Mode
+# Model API Control
 
-TorchServe now supports model control mode with two settings "none"(default) and "enabled"
+TorchServe now disables the use of model API (specifically registering and deleting models) by default. The use of these APIs can be enabled through command line or config.properties file.
+
+TorchServe disables the ability to register and delete models using API calls once servers are running. This is a security feature which addresses the concern of unwanted registering and deleting of models once the TorchServe servers have started. This is applicable in the scenario where a user may upload malicious code to the model server in the form of a model or where a user may delete a model that is being used. The default behavior prevents users from registering or deleting models once TorchServe is running, and then you can enable the model APIs to allow users to register and delete models whenever using the TorchServe model load APIs.
 
 ## Two ways to set Model Control
-1. Add `--model-api-enabled` to command line when running TorchServe to switch from none to enabled mode. Command line cannot be used to set mode to none, can only be used to set to enabled
-2. Add `model_api_enabled=false` or `model_api_enabled=true` to config.properties file
-    * `model_api_enabled=false` is default and prevents users from registering or deleting models once TorchServe is running
-    * `model_api_enabled=true` is not default and allows users to register and delete models using the TorchServe model load APIs
+1. Global environment variable: use `TS_ENABLE_MODEL_API` and set to `true` to enable and `false` to disable model API use. Note that `enable_envvars_config=true` must be set in config.properties for global environment variables to be used
+2. Add `--enable-model-api` to command line when running TorchServe to switch from disabled to enabled. Command line cannot be used to disabled, can only be used to enabled
+3. Add `enable_model_api=false` or `enable_model_api=true` to config.properties file
+    * `enable_model_api=false` is default and prevents users from registering or deleting models once TorchServe is running
+    * `enable_model_api=true` is not default and allows users to register and delete models using the TorchServe model load APIs
 
-Priority between cmd and config file follows the following [TorchServer standard](https://github.com/pytorch/serve/blob/c74a29e8144bc12b84196775076b0e8cf3c5a6fc/docs/configuration.md#advanced-configuration)
+Priority follows the following [TorchServer standard](https://github.com/pytorch/serve/blob/c74a29e8144bc12b84196775076b0e8cf3c5a6fc/docs/configuration.md#advanced-configuration)
 * Example 1:
-  * Config file: `model_api_enabled=false`
+  * Config file: `enable_model_api=false`
 
-    cmd line: `torchserve --start --ncs --model-store model_store --model-api-enabled`
+    cmd line: `torchserve --start --ncs --model-store model_store --enable-model-api`
 
     Result: Model api mode enabled
 * Example 2:
-  * Config file: `model_api_enabled=true`
+  * Config file: `enable_model_api=true`
 
     cmd line: `torchserve --start --ncs --model-store model_store`
 
     Result: Mode is enabled (no way to disable api mode through cmd)
 
-## Model Control Mode Default
+## Model API Control Default
 At startup TorchServe loads only those models specified explicitly with the `--models` command-line option. After startup users will be unable to register or delete models in this mode.
 
 ### Example default
@@ -40,11 +43,11 @@ ubuntu@ip-172-31-11-32:~/serve$ curl -X POST  "http://localhost:8081/models?url=
 ```
 
 ## Model Control API Enabled
-Setting model control to `enabled` allows users to load and unload models using the model load APIs.
+Setting model API to `enabled` allows users to load and unload models using the model load APIs.
 
 ### Example using cmd line to set mode to enabled
 ```
-ubuntu@ip-172-31-11-32:~/serve$ torchserve --start --ncs --model-store model_store --models resnet-18=resnet-18.mar --ts-config config.properties --model-api-enabled
+ubuntu@ip-172-31-11-32:~/serve$ torchserve --start --ncs --model-store model_store --models resnet-18=resnet-18.mar --ts-config config.properties --enable-model-api
 
 ubuntu@ip-172-31-11-32:~/serve$ curl -X POST  "http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/squeezenet1_1.mar"
 {

docs/token_authorization_api.md

@@ -2,18 +2,19 @@
 
 TorchServe now enforces token authorization by default
 
+TorchServe enforces the feature of token authorization which requires the correct token to be provided when calling an API. This is a security feature which addresses the concern of unwanted API calls. This is applicable in the scenario where an unauthorized user may try to access a running TorchServe server. The default behavior is to enable this feature which creates a key file with the appropriate tokens to be used for API calls. Users can disable this feature to prevent token authorization from being required for API calls ([how to disable](#how-to-set-and-disable-token-authorization)), however users are warned that this will open up the server to potential unwanted API calls.
 
 ## How to set and disable Token Authorization
 * Global environment variable: use `TS_DISABLE_TOKEN_AUTHORIZATION` and set to `true` to disable and `false` to enable token authorization. Note that `enable_envvars_config=true` must be set in config.properties for global environment variables to be used
-* Command line: Command line can only be used to disable token authorization by adding the `--disable-token` flag.
+* Command line: Command line can only be used to disable token authorization by adding the `--disable-token-auth` flag.
 * Config properties file: use `disable_token_authorization` and set to `true` to disable and `false` to enable token authorization.
 
 Priority between env variables, cmd, and config file follows the following [TorchServer standard](https://github.com/pytorch/serve/blob/master/docs/configuration.md)
 
 * Example 1:
   * Config file: `disable_token_authorization=false`
 
-    cmd line: `torchserve --start --ncs --model-store model_store --disable-token`
+    cmd line: `torchserve --start --ncs --model-store model_store --disable-token-auth`
 
     Result: Token authorization disabled through command line but enabled through config file, resulting in token authorization being disabled. Command line takes precedence
 * Example 2:

examples/README.md

@@ -1,5 +1,8 @@
 # [Examples showcasing TorchServe Features and Integrations](#torchserve-internals)
 
+## Security Changes
+TorchServe now enforces token authorization and model API control by default. This change will impact the current examples so please check the following documentation for more information: [Token Authorization](https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md), [Model API control](https://github.com/pytorch/serve/blob/master/docs/model_api_control.md)
+
 ## TorchServe Internals
 
 * [Creating mar file for an eager mode model](#creating-mar-file-for-eager-mode-model)

frontend/server/src/main/java/org/pytorch/serve/http/TokenAuthorizationHandler.java

@@ -85,7 +85,16 @@ public static void setupToken() {
             try {
                 token = new Token();
                 if (token.generateKeyFile("token")) {
-                    logger.info("Token Authorization Enabled");
+                    String loggingMessage =
+                            "\n######\n"
+                                    + "TorchServe now enforces token authorization by default.\n"
+                                    + "This requires the correct token to be provided when calling an API.\n"
+                                    + "Key file located at "
+                                    + ConfigManager.getInstance().getModelServerHome()
+                                    + "\n"
+                                    + "Check token authorization documenation for information: https://github.com/pytorch/serve/blob/master/docs/token_authorization_api.md \n"
+                                    + "######\n";
+                    logger.info(loggingMessage);
                 }
             } catch (IOException e) {
                 e.printStackTrace();