Add encrypted file upload to gossip ring

[bookshelfdave]

This PR adds the hab file upload subcommand. In order for this command to work, you need a user key and a service key.

To encrypt, hab upload needs to be able to find the service public key and the user secret key in /hab/cache/keys (or override with HAB_CACHE_KEY_PATH). To decrypt a message, the supervisor needs a service secret key and a user public key.

If the supervisor receives the encrypted file via gossip, but is not the intended recipient, the file remains encrypted in memory. If the supervisor is the recipient and has service secret key + user public key, then the decrypted file will be written out to disk in the running service directory (ex: /hab/svc/redis/files/foo). If a service is the recipient but does not have the appropriate keys to decrypt, the GossipFileList will retry the write with exponential backoff.

Gossip file write failures are available in the sidecar:

root@d0b0b2323cca:/src/components/hab# curl http://localhost:9631/gossip | jq .file_write_retries
{
  "foo": {
    "file_name": "foo",
    "last_failure_reason": "Crypto error: Cannot find user key dparfitt-20160426212017",
    "total_retries": 4
  }
}

• HAB_ORG is honored if you don't specify --org
• ServiceGroup now has an organization component, and it's from_str can parse strings in "foo.bar" and "foo.bar@baz" format
• ConfigFile renamed to GossipFile
• ConfigFileList renamed to GossipFileList
• maximum file upload size is 4096 bytes
• remove dead key tests in sup

---

To play around with this feature:

hab user key generate someuser
hab service key generate redis.default xyz

Open 3 terminals:
/src/components/sup/target/debug/hab-sup start core/redis --permanent-peer  --topology leader --group default --org xyz
/src/components/sup/target/debug/hab-sup start core/redis --permanent-peer --peer 172.17.0.2:9634 --topology leader --group default --org xyz
/src/components/sup/target/debug/hab-sup start core/redis --permanent-peer --peer 172.17.0.2:9634 --topology leader --group default --org xyz
(note, you may need to change the `--peer` IP address, which you can find by running `ip a`)

date > foo
./target/debug/hab file upload redis.default ./foo 1 someuser  --org xyz

# on a supervisor node, you can cat the file:
cat /hab/svc/redis/files/foo

you can make this fail on the sup side by setting HAB_CACHE_KEY_PATH to a directory without keys:

export HAB_CACHE_KEY_PATH=/some/dir

and then uploading a new file. Check for failures with:

curl http://localhost:9631/gossip | jq .file_write_retries

and resolve the error by resetting:

export HAB_CACHE_KEY_PATH=/hab/cache/keys

---

TODO:

• receiving an encrypted payload without having appropriate keys to decrypt results in sup crash/restart. Slack discussion
  • bubble up severe error to sidecar
  • don't crash the supervisor and log an error
• more testing: GossipFile + GossipFileList definitely need more testing, but that will have to come in a later PR to make it easier to mock fs:svc_path*
• if we want "destination path" for an uploaded file, I'd like to do that in a separate PR Slack discussion

[thesentinels]

By analyzing the blame information on this pull request, we identified @fnichol, @adamhjk, @reset and @juliandunn to be potential reviewers

[bookshelfdave]

@habitat-sh/habitat-core-maintainers this is ready for review

[adamhjk]

@metadave it's worth squashing these three commits down into one, since the feature all goes together.

[adamhjk]

Need a little whitespace here.

[adamhjk]

Other than squashing the commits and adding one line of whitespace this looks great. The tests in particular are great. Nice work!

[bookshelfdave]

I'll squash and rebase against master

ping @davidwrede

[thesentinels]

☔ The latest upstream changes (presumably c86dfab) made this pull request unmergeable. Please resolve the merge conflicts.

[bookshelfdave]

@thesentinels r+

[thesentinels]

📌 Commit b9676d9 has been approved by metadave

[thesentinels]

⌛ Testing commit b9676d9 with merge 53f1526...

[thesentinels]

☀️ Test successful - travis