hashi_vault - add support for none auth type

[jlrgraham23]

SUMMARY

Support the use of a locally running Vault Agent process to interact with Vault. This relieves the user almost entirely have need to manage or interact with tokens.

Fixes #79

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

community.hashi_vault, authentication methods.

ADDITIONAL INFORMATION

More details in: #79

- name: authenticate with vault agent
  ansible.builtin.debug:
    msg: "{{ lookup('community.hashi_vault.hashi_vault', 'secret/hello:value', auth_method=agent, url='http://127.0.0.1:8100') }}"

Also related, older issue against the main Ansible repo: ansible/ansible#60728

[codecov]

Codecov Report

Merging #80 (44d178a) into main (a761953) will increase coverage by 0.08%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #80      +/-   ##
==========================================
+ Coverage   82.88%   82.97%   +0.08%     
==========================================
  Files          16       16              
  Lines         783      787       +4     
  Branches       78       78              
==========================================
+ Hits          649      653       +4     
  Misses        120      120              
  Partials       14       14              

Impacted Files	Coverage Δ
plugins/lookup/hashi_vault.py	63.84% <100.00%> (+0.69%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a761953...44d178a. Read the comment docs.

[briantist]

So first off, I want to thank you for the submission and apologize for the delay in reviewing. I've been on vacation (still am) so I may be slower to respond than usual.

As for the general idea, I have a few thoughts.

First, I want to confirm that agent auth can't already be achieved, by using "token" auth and passing anything for the token value, like

lookup('community.hashi_vault.hashi_vault', 'secret/hello:value', auth_method='token', token='fake')

I played around with agent a little in the past and I thought this worked. I ask not because that means we don't need or want something more appropriate, but because it's good to know workarounds/alternatives.

As for your addition, the method you're adding, called "agent" is not actually doing anything specifically, it's just passing nothing. So I'm wondering if this should be handled differently, like maybe having an "auth" method called none instead? After all it is also possible to use an agent with other auth methods, so calling this "agent" could be confusing imo.

The other major issue here is that there are no tests. I'm working hard to refactor the plugin, and to improve testing, and hopefully to make it easier to add tests for future contributions, but I'm hesitant to add new features that don't include any tests.

Related to naming, if we're calling the auth method "agent" then we're sort of saying it's expected to be used with agent, and that means we should test against agent. Since we don't yet test against a Vault agent it'll be some doing to get that going in CI.

Testing against Vault itself, and against other software (like tinyproxy which we use to test proxy operations), can be kind of pain, and I'm working on making that experience better right now.

On the other hand, if we call this auth method none, I think we can do with much less testing of it; basically we'd probably just against Vault as normal but we'd expect an access denied in our tests.

none will also flow better if there are other ways of running Vault server/agent in the future that make use of no authentication.

What do you think?

[jlrgraham23]

Hi @briantist! Thanks for the great feedback. Upfront, if you're on vacation no need at all to respond to this now! I've got a local build and distribution system in my environment where this change is needed, so it's not blocking usage. Just wanted to try and contribute back.

I like the idea of renaming the method to none, I'd just pulled agent out as it seemed to be more descriptive of what was actually happening -- that is, the user is declaring that they want to delegate the authentication, but at the same time, I guess that isn't actually really that different from just saying default to the token in ${HOME}/.vault-token. If that simplifies testing then great, I'll update for that convention.

I'm also happy to try and help with getting any CI tests in for Agent, but I'm new to this testing framework. I'll try to dig at some docs and see if I can figure out what's needed! :)

Some notes on if this is needed, unfortunately just passing in a fake token errors out when the plugin tries to validate the token.

lookup("community.hashi_vault.hashi_vault", "secret= secret/hello:value", auth_method="token", token="fake")

Yields:

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Invalid Hashicorp Vault Token Specified for hashi_vault lookup."}

And explicitly disabling validation

lookup("community.hashi_vault.hashi_vault", "secret=secret/hello:value", auth_method="token", token="fake", token_validate=false)

Yields:

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'community.hashi_vault.hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Forbidden: Permission Denied to secret 'secret/hello'."}```

I did a quick dig on the Vault Agent docs and didn't find an explicit answer (it was a cursory dig), but it appears that the behavior if a token is supplied it overrides Vault Agent's credentials.

Without VAULT_TOKEN set or with a ${HOME}/.vault-token file and the agent running

$ echo $VAULT_ADDR
http://127.0.0.1:8100
$ /usr/local/bin/vault kv get secret/hello
====== Metadata ======
Key              Value
---              -----
created_time     2021-03-29T03:38:36.993280771Z
deletion_time    n/a
destroyed        false
version          1

====== Data ======
Key         Value
---         -----
value       foo

With an invalid token:

$ VAULT_TOKEN=foo /usr/local/bin/vault kv get secret/hello
Error making API request.

URL: GET http://127.0.0.1:8100/v1/sys/internal/ui/mounts/secret/hello
Code: 403. Errors:

* permission denied

With a different valid token:

$ VAULT_TOKEN=<itsasecret> /usr/local/bin/vault kv get secret/hello
====== Metadata ======
Key              Value
---              -----
created_time     2021-03-29T03:38:36.993280771Z
deletion_time    n/a
destroyed        false
version          1

====== Data ======
Key         Value
---         -----
value       foo

[briantist]

Changes are looking good so far, how do you feel about trying to add the tests for the none method based on the tests for the other auth methods? I'd probably go off the token auth method but the difference would mainly be that you wouldn't be supplying a token and that you'd expect the calls to fail, so it should be simpler to set up.

[briantist]

Hi @jlrgraham23 I took the liberty of adding tests and a changelog fragment, and making a few other test updates, hope that's ok. Could you take a look and let me know if it looks good to you? And if possible to try it out with a Vault agent as you seem to have one set up already.

@Akasurde since I'm now a co-author would you be willing to give this another look so there's some additional review?

PR updated