Skip to content

Commit

Permalink
feat: Document new MQTT Export Pre-connect capability (#1326)
Browse files Browse the repository at this point in the history 
Signed-off-by: Leonard Goodell <leonard.goodell@intel.com>
  • Loading branch information
Lenny Goodell authored Jan 8, 2024
1 parent bc469fd commit 43c8e08
Show file tree
Hide file tree
Showing 3 changed files with 73 additions and 29 deletions.
2 changes: 1 addition & 1 deletion docs_src/design/TOC.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,4 +52,4 @@
| [0026 Common Configuration](./adr/0026-Common Configuration.md) | Separate out the common configuration setting into a single source for all the services |
| [0027 URIs for Files](./adr/0027-URIs for Files.md) | Add capability to load service files from remote locations using URIs |
| [0028 Microservice communication security (token)](./adr/security/0028-authentication.md) | Microservice communication security / authentication (token-based) |
| [0029 Microservice communication security (E2EE)](./adr/security/0029-authentication.md) | Microservice communication security / authentication (end-to-end authentication) |
| [0029 Microservice communication security (E2EE)](./adr/security/0029-authentication-e2ee.md) | Microservice communication security / authentication (end-to-end authentication) |
23 changes: 22 additions & 1 deletion docs_src/microservices/application/sdk/api/BuiltInPipelineFunctions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,22 @@ The `URLFormatter` option allows you to override the default formatter with your
| NewMQTTSecretSender(mqttConfig MQTTSecretConfig, persistOnError bool) | This factory function returns a `MQTTSecretSender` instance initialized with the options specified in the `MQTTSecretConfig` and `persistOnError `. |
| NewMQTTSecretSenderWithTopicFormatter(mqttConfig MQTTSecretConfig, persistOnError bool, topicFormatter StringValuesFormatter) | This factory function returns a `MQTTSecretSender` instance initialized with the options specified in the `MQTTSecretConfig`, `persistOnError ` and `topicFormatter `. See [Topic Formatting](#topic-formatting) below for more details. |

| Method | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| ConnectToBroker(lc logger.LoggingClient, sp bootstrapInterfaces.SecretProvider, retryCount int, retryInterval time.Duration) | Pre-connects to the external MQTT Broker that data will be exported. If this function is not called, then lazy connection will be made when the first data needs to be exported. |

!!! example - "Pre-Connecting to MQTT Broker"
```go
...
export := transforms.NewMQTTSecretSender(mqttConfig, false)
export.ConnectToBroker(app.service.LoggingClient(), app.service.SecretProvider(), 5, time.Second*3)
err := app.service.SetDefaultFunctionsPipeline(export.MQTTSend)
...
```

!!! edgey - "EdgeX 3.2"
ConnectToBroker is new in EdgeX 3.2

```go
type MQTTSecretConfig struct {
// BrokerAddress should be set to the complete broker address i.e. mqtts://mosquitto:8883/mybroker
Expand All @@ -303,6 +319,8 @@ type MQTTSecretConfig struct {
SecretName string
// AutoReconnect indicated whether or not to retry connection if disconnected
AutoReconnect bool
// MaxReconnectInterval is the max duration for attempting to reconnect to the broker. Default to 60s if left blank.
MaxReconnectInterval string
// KeepAlive is the interval duration between client sending keepalive ping to broker
KeepAlive string
// ConnectTimeout is the duration for timing out on connecting to the broker
Expand Down Expand Up @@ -338,6 +356,9 @@ type WillConfig struct {
}
```

!!! edgey - "EdgeX 3.2"
MaxReconnectInterval setting is new in EdgeX 3.2

See [MQTT Last Will](https://cedalo.com/blog/mqtt-last-will-explained-and-example) for more details on MQTT Last Will capability.

!!! edgey "EdgeX 3.1"
Expand Down Expand Up @@ -479,7 +500,7 @@ There is one Tags transform included in the SDK that can be added to your pipeli
| Factory Method | Description |
|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| NewTags(tags `map[string]interface{}`) Tags | This factory function returns a `Tags` instance initialized with the passed in collection of generic tag key/value pairs. This `Tags` instance is used to access the following Tags function that will use the specified collection of tag key/value pairs. This allows for generic complex types for the Tag values. |


### Add Tags

Expand Down
77 changes: 50 additions & 27 deletions ...plication/services/AppServiceConfigurable/details/AvailablePipelineFunctions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -272,27 +272,31 @@ Please refer to the function's detailed documentation by clicking the function n

**Parameters**

| Name | Description |
|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| BrokerAddress | URL specify the address of the MQTT Broker |
| Topic | Topic to publish the data |
| ClientId | Id to use when connecting to the MQTT Broker |
| Qos | MQTT Quality of Service (QOS) setting to use (0, 1 or 2). Please refer [**here**](https://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/qos.html) for more details on QOS values |
| AutoReconnect | Boolean specifying if reconnect should be automatic if connection to MQTT broker is lost |
| Retain | Boolean specifying if the MQTT Broker should save the last message published as the “Last Good Message” on that topic |
| SkipVerify | Boolean indicating if the certificate verification should be skipped |
| PersistOnError | Indicates to persist the data if the POST fails. Store and Forward must also be enabled if this is set to "true" |
| AuthMode | Mode of authentication to use when connecting to the MQTT Broker. Valid values are: |
| | **none** - No authentication required |
| | **usernamepassword** - Use username and password authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `username` and `password` secrets |
| | **clientcert** - Use Client Certificate authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `clientkey` and `clientcert` secrets |
| | **cacert** - Use CA Certificate authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `cacert` secret |
| SecretName | Name of the secret in the SecretStore where authentication secrets are stored |
| WillEnabled | Enables Last Will Capability. See for [MQTT Last Will](https://cedalo.com/blog/mqtt-last-will-explained-and-example) more details. |
| WillTopic | Topic Last Will messages is publish |
| WillPayload | Last Will messages to be published when service disconnects from broker |
| WillRetain | Boolean specifying if the MQTT Broker should save the last message published as the “Last Good Message” on the Will topic |
| WillQos | MQTT Quality of Service (QOS) setting to use (0, 1 or 2) for Last Will Message. |
| Name | Description |
| ----------------------- | ------------------------------------------------------------ |
| BrokerAddress | URL specify the address of the MQTT Broker |
| Topic | Topic to publish the data |
| ClientId | Id to use when connecting to the MQTT Broker |
| Qos | MQTT Quality of Service (QOS) setting to use (0, 1 or 2). Please refer [**here**](https://www.eclipse.org/paho/files/mqttdoc/MQTTClient/html/qos.html) for more details on QOS values |
| AutoReconnect | Boolean specifying if reconnect should be automatic if connection to MQTT broker is lost. |
| MaxReconnectInterval | Time duration string that specifies the maximum duration to wait before trying to reconnect. Defaults to 60s if not specified. |
| Retain | Boolean specifying if the MQTT Broker should save the last message published as the “Last Good Message” on that topic |
| SkipVerify | Boolean indicating if the certificate verification should be skipped |
| PersistOnError | Indicates to persist the data if the POST fails. Store and Forward must also be enabled if this is set to "true" |
| AuthMode | Mode of authentication to use when connecting to the MQTT Broker. Valid values are: |
| | **none** - No authentication required |
| | **usernamepassword** - Use username and password authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `username` and `password` secrets |
| | **clientcert** - Use Client Certificate authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `clientkey` and `clientcert` secrets |
| | **cacert** - Use CA Certificate authentication. The Secret Store (Vault or [InsecureSecrets](../../../Configuration.md#writable)) must contain the `cacert` secret |
| SecretName | Name of the secret in the SecretStore where authentication secrets are stored |
| WillEnabled | Enables Last Will Capability. See for [MQTT Last Will](https://cedalo.com/blog/mqtt-last-will-explained-and-example) more details. |
| WillTopic | Topic Last Will messages is publish |
| WillPayload | Last Will messages to be published when service disconnects from broker |
| WillRetain | Boolean specifying if the MQTT Broker should save the last message published as the “Last Good Message” on the Will topic |
| WillQos | MQTT Quality of Service (QOS) setting to use (0, 1 or 2) for Last Will Message. |
| PreConnect | Boolean that indicates if the MQTT Broker connection should be established on initialization. Default is false which results in lazy connection when first data needs to be exported. |
| PreConnectRetryCount | Specifies the number of times to attempt to pre-connect to the MQTT Broker. If connection is never made, MQTT export reverts to using lazy connect. Defaults to 6 if not specified. |
| PreConnectRetryInterval | Time duration string that specifies the amount of time to wait between pre-connect attempts. Defaults to 10s if not specified. |

!!! note
`Authmode=cacert` is only needed when client authentication (e.g. `usernamepassword`) is not required, but a CA Cert is needed to validate the broker's SSL/TLS cert.
Expand All @@ -316,9 +320,9 @@ Please refer to the function's detailed documentation by clicking the function n
BrokerAddress: "tcps://my-broker-host.com:8883"
Topic: "mytopic"
ClientId: "myclientid"
Qos="2"
AutoReconnect="true"
Retain="true"
Qos: "2"
AutoReconnect: "true"
Retain: "true"
SkipVerify: "false"
PersistOnError: "true"
AuthMode: "usernamepassword"
Expand All @@ -331,9 +335,9 @@ Please refer to the function's detailed documentation by clicking the function n
BrokerAddress: "tcps://my-broker-host.com:8883"
Topic: "mytopic"
ClientId: "myclientid"
Qos="2"
AutoReconnect="true"
Retain="true"
Qos: "2"
AutoReconnect: "true"
Retain: "true"
SkipVerify: "false"
PersistOnError: "true"
AuthMode: "none"
Expand All @@ -343,6 +347,25 @@ Please refer to the function's detailed documentation by clicking the function n
WillRetained: "true"
WillTopic: "serviceX/last/will"
```
```yaml
# MQTT Export with pre-connect and MaxReconnectInterval
MQTTExport:
Parameters:
BrokerAddress: "tcps://my-broker-host.com:8883"
Topic: "mytopic"
ClientId: "myclientid"
Qos: "2"
AutoReconnect: "true"
MaxReconnectInterval: "15s"
Retain: "true"
SkipVerify: "false"
PersistOnError: "true"
AuthMode: "none"
PreConnect: "true"
PreConnectRetryCount: "10"
PreConnectRetryInterval: "2s"
```

## [SetResponseData](../../../sdk/api/BuiltInPipelineFunctions.md#set-response-data)

**Parameters**
Expand Down

0 comments on commit 43c8e08

Please sign in to comment.