YAML settings file reference

This page contains description for each field of YAML generated by subcommand settings:

$ torchlambda settings

Fields have their specific properties:

• required: whether this field and value is required in order to create C++ code
• type: type of value
• default: whether there is a default value for this field. If this property exists user doesn't have to provide any value

Below is generated scheme with every field with some example values:

---
grad: true
validate_json: true
data: data
validate_data: true
model: /opt/model.ptc
inputs: [1, 3, width, height]
validate_inputs: true
cast: float
divide: 255
normalize:
  means: [0.485, 0.456, 0.406]
  stddevs: [0.229, 0.224, 0.225]
return:
  output:
    type: double
    name: output
    item: false
  result:
    operation: argmax
    arguments: 1
    type: long
    name: labels
    item: true

For exact description of each field see below

Fields

grad

If false disables PyTorch's autograd engine. It is almost always safe to set this value to false as it is not needed for inference. If you need it for some reason, set field below to true.

Properties:

• type: bool
• default: false

validate_json

If true check whether JSON passed as request was parsed successfully. If it wasn't, InvalidJSON will be sent as a response containing string: Failed to parse request JSON file.

Properties:

• type: bool
• default: true

data

Name of the JSON field which is necessary to pass during request and contains base64 encoded data (e.g. image).

Properties:

• type: non-empty str
• default: data

validate_data

If true check whether data_field exists in the request. If it doesn't InvalidJSON will be sent as a response containing string: Required field <name_of_field> was not provided.

Properties:

• type: bool
• default: true

model

Path to uploaded model within Lambda's filesystem. If model is uploaded as AWS Lambda Layer it will be unpacked to /opt and you should take it into consideration.

Properties:

• type: str
• default: /opt/model.ptc

inputs

Shape of tensor passed as base64 encoded string via data. Should be provided as list containing either int values or str values or both. For integers, shape will be hardcoded (for example 3 for three input channels). If str provided field with this name will be searched for in request and used as one of tensor dimensions.

Properties:

• type: List[str|int]
• default: /opt/model.ptc

validate_inputs

If true check whether all str values from inputs are passed in the request as JSON fields.

• If any field is missing InvalidJSON will be sent as a response containing string: Required input shape field: '<name_of_field>' was not provided
• If field exists but isn't of int type InvalidJSON will be sent as a response containing string: Required shape field: '<name_of_field>' is not of integer type.

Properties:

• type: bool
• default: true

cast

Type to which encoded byte data will be casted after tensor is created. Usually floating point types (half, float, double) should be used.

One of those values allowed:

• byte
• char
• short
• int
• long
• half
• float
• double

Properties:

• type: string
• default: float

divide

Value by which tensor will be divided after being casted. If user does not want division he should specify 1 as value.

255 is set as default because it will put [0, 255] range tensor after encoding into [0, 1] range (which is usually used for image predictions).

Properties:

• type: number
• default: 255 (if normalize specified)

normalize

If this nested field is defined tensor obtained from data will be normalized across channels using specified means and standard deviations.

⚠️ Under the hood uses torch::data::transforms::Normalize which requires means and stddevs to be any shape broadcastable to input tensor.

Example field (with ImageNet means and standard deviations):

normalize:
  means: [0.485, 0.456, 0.406]
  stddevs: [0.229, 0.224, 0.225]

For exact description of each field see below

means

Mean values used for normalization.

Properties:

• required: true (if normalize specified)
• type: list[number]

stddevs

Standard deviations used for normalization.

Properties:

• required: true (if normalize specified)
• type: list[number]

return

This is single most important (nested) field to understand correctly.

It is required for user to specify at least one of the fields below:

• output
• result

⚠️ if output and result specified both will be returned in JSON response!

output

If this field is specified output taken directly from neural network will be returned as JSON field. This field has additional properties user can specify in order to define what is returned.

Example field (including top level return):

return:
  output:
    name: output
    type: double
    item: true

For exact description of each field see below

type

Data type to be returned.

One of those values allowed:

• bool
• int
• long
• double

Properties:

• required: true
• type: str

name

Key (name) of field where the output will be placed in returned JSON.

Properties:

• type: str
• default: output

item

If true, output will be returned as a single item.

⚠️ output HAS TO BE SINGLE ITEM otherwise you will get silent error

Tensor will be flattened and .item<type>() will be applied on it. If your output is not a single value leave this with default false value.

Properties:

• type: bool
• default: false

result

result is created by some tensor operations applied on neural network output. For example, you can apply argmax operation on outputted logits along first dimension to get corresponding labels. You can describe those steps via .yaml setting like this:

return:
  result:
    operations: argmax
    arguments: 1
    name: labels
    type: long
    item: true

⚠️ you don't have to specify output for result to work!

For exact description of each field see below

operations

Name of operation described with str. To see available options please refer to libtorch documentation. All functions in at:: namespace are available but they have to be specified without at:: namespace.

Some available options could be: softmax, argmax, sigmoid.

Multiple consecutive operations can be applied if list is used.

In the example below first sigmoid will be applied on output and after that argmax to produce result:

return:
  result:
    operations: [sigmoid, argmax]
	...

Properties:

• required: true
• type: str or list[str]

arguments

Additional arguments beside output to be passed to operations (one or more). Can be specified as single argument of any type that will be passed as argument to first operation.

Example below will pass 1 as a dimension to softmax and nothing to argmax:

return:
  result:
    operations: [softmax, argmax]
    arguments: 1
	...

Example below will pass 1 as a dimension to second argument argmax and nothing to sigmoid:

return:
  result:
    operations: [sigmoid, argmax]
    arguments: [[], 1]
	...

You could also pass multiple arguments to each function, just place them in an appropriate list

Properties:

• default: None (no additional arguments beside output will be passed to operations
• type: any or list[any] or list[list[any] | any]

type

Data type to be returned.

One of those values allowed:

• bool
• int
• long
• double

Properties:

• required: true
• type: str

name

Key (name) of field where the output will be placed in returned JSON.

Properties:

• type: str
• default: output

item

If true, output will be returned as a single item.

⚠️ output HAS TO BE SINGLE ITEM otherwise you will get silent error

Tensor will be flattened and .item<type>() will be applied on it. If your output is not a single value leave this with default false value.

Properties:

• type: bool
• default: false