Allow more characters in API Key names

[tjkirch]

This change allows for more characters to appear in our key names.
Specifically, "." and "/" are valid in Kubernetes labels and taints, but were
disallowed by the existing API system.

The size of this change reflects some of the complexity of keys:

• Their names are used commonly in discussions, documentation, user data, and
  code. We want the names to be convenient to reference.
• Their naming has security considerations. The character set of key names
  might not line up with the allowed character set in the data store
  implementation, for example dots and slashes in the filesystem.
• They're used everywhere in API-related code, so while we add functionality
  and security, we can't make them hard to use.

Here are the changes at 30,000 feet:

• Special characters in key names are percent-encoded ("URL-encoded") when used
  as filenames on the filesystem. For example, "%2E" for "."
• Key names are "TOML-quoted" elsewhere. This fits usage in user data, and is
  relatively intuitive. For example:
  settings.kubernetes.node-labels."group.name"
  This key name leads to "group.name" being a single segment instead of two,
  and therefore "group.name" being the name of a Kubernetes label.
• Key name segments in API requests and responses don't need to be quoted
  because they're already separated structurally, for example:
  {"settings": {"kubernetes": {"node-labels": {"group.name": "value"}}}}
• No existing key names or file paths have to change, because keys without
  special characters are still valid in both schemes.

Here are the changes at 10,000 feet:

• Key names are treated as a series of segments, rather than a single string.
  This allows for more precise specification and testing of names.
• Key names are parsed and encoded from segments, rather than checking them
  with a regex.
• Key name stripping / appending is done more carefully, by segment.
• Internal functions (for example in the data store) take Keys rather than
  strings, where feasible, to improve specificity.
  • Some, like to_pairs_with_prefix, continue to take strings, when it makes a simple use
    case more friendly.
  • migrations continue to use strings so they don't have to link to the data
    store, and to make writing them simpler.
• Keys no longer Deref to string, AsRef, etc., to reduce the possibility
  of a mixup resulting in mistaken/double quoting.

Other implementation notes:

• get_prefix had the strip_prefix parameter removed because it was no
  longer used, and would have had to change to taking segments/Key anyway.

---

Fixes #455.

Testing done:

Existing and new unit tests pass, for the whole workspace.

I added a node label with slashes and dots to my user data:

[settings.kubernetes.node-labels]
"tjk/group.name" = "hi there"

The instance launched OK and all services were running:

bash-5.0# systemctl status
  ip-192-168-127-167.us-west-2.compute.internal
    State: running
     Jobs: 0 queued
   Failed: 0 units

The setting shows up as you'd expect - no escaping needed in JSON format.

bash-5.0# apiclient -u /settings
{..., "kubernetes":{..., "node-labels":{"tjk/group.name":"hi there"},...}}

It won't connect to Kubernetes because spaces aren't valid in label values :)

Nov 13 21:17:03 ip-192-168-127-167.us-west-2.compute.internal kubelet[3439]: E1113 21:17:03.967064    3439 kubelet_node_status.go:94] Unable to register node "ip-192-168-127-167.us-west-2.compute.interna
l" with API server: Node "ip-192-168-127-167.us-west-2.compute.internal" is invalid: metadata.labels: Invalid value: "hi there": a valid label must be an empty string or consist of alphanumeric character
s, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')

Fix that:

[settings.kubernetes.node-labels]
"tjk/group.name" = "howdy"

Then the instance joins just fine, and you can see the label at the end here:

$ kubectl get nodes --show-labels
NAME                                           STATUS   ROLES    AGE   VERSION   LABELS
ip-192-168-105-42.us-west-2.compute.internal   Ready    <none>   44s   v1.14.6   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=c3.large,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/region=us-west-2,failure-domain.beta.kubernetes.io/zone=us-west-2c,kubernetes.io/arch=amd64,kubernetes.io/hostname=ip-192-168-105-42.us-west-2.compute.internal,kubernetes.io/os=linux,tjk/group.name=howdy

A busybox pod ran fine.

Here's how you can update settings; same JSON format:

bash-5.0# apiclient -m PATCH -u /settings -d '{"kubernetes": {"node-labels": {"tjk/group.idx": "42"}}}'

POSTing the changes returns a list of the changed settings, e.g. for thar-be-settings. We can see the dotted-key form here, which requires escaping of the inner quotes:

bash-5.0# apiclient -m POST -u /settings/commit_and_apply
["settings.kubernetes.node-labels.\"tjk/group.idx\""]

The new setting shows up fine, next to the one from (the earlier) user data:

bash-5.0# apiclient -u /settings
{..., "kubernetes":{..., "node-labels":{"tjk/group.name":"hi there","tjk/group.idx":"42"},...}}

[zmrow]

Whew! That was a journey! Nice work on this big change. Let's get it in and keep moving!

[bcressey]

Nicely done!

[tjkirch]

This push addresses @bcressey's concerns. Updated and new unit tests pass, and the manual testing above repeats correctly.

His request for additional testing exposed two bugs that are also fixed:

• Stripping the prefix wouldn't fail if it the prefix matched the key, as the docs said, because the appended . wouldn't match. (Yes, I see the irony/inconsistency here. Need to move to segments even more, but separately I think.)
• Keys created with from_segments weren't rejected for invalid characters, they were just quoted, which was only intended for segments with the . separator. Now it's checked, so new and from_segments check for the same invalid characters.

[bcressey]

🥇