hashi_vault lookup plugin require auth/token/lookup-self policy on the Vault token to read secrets

[rasta-rocket]

SUMMARY

Hello folks 👋

I found a behavior that seems weird to me: I'm using the hashi_vault lookup plugin with the token auth_method, I'm just setting a Vault token with a minimal set of vault policies (principle of least privilege) to my CI/CD to run the playbook that fetches secrets and the CI/CD system cannot fetches the secrets.

By using the token (with the given policies) with the Vault CLI I don't get any trouble.

My policies contains only sufficient rules to fetch the secrets that I want (read and list on specific vault path).

By investigating the code I came to that:

1. When using the token auth_method, the function auth_token is called.

2. This function call is_authenticated

3. That latest function only return True if the token have the capability to lookup-itself

So I came to the conclusion that if I want my playbook to run I have to add the lookup-itself capability in my token policies. I made the test and it works.

However Vault gives the capability to request its API by just setting the VAULT_TOKEN env variable, so why do we need that is_authenticated check for the token part ?

Thanks for your reply 🙏

ISSUE TYPE

• Bug Report

COMPONENT NAME

hashi_vault

ANSIBLE VERSION

ansible 2.10.3
  config file = /Users/***/***/ansible.cfg
  configured module search path = ['/Users/***/***/library']
  ansible python module location = /Users/***/venv/ansible/lib/python3.8/site-packages/ansible
  executable location = /Users/***/venv/ansible/bin/ansible
  python version = 3.8.6 (default, Oct  8 2020, 14:07:53) [Clang 11.0.0 (clang-1100.0.33.17)]

CONFIGURATION

DEFAULT_MODULE_PATH(env: ANSIBLE_LIBRARY) = ['/Users/***/library']
DEFAULT_ROLES_PATH(env: ANSIBLE_ROLES_PATH) = ['/Users/***/.ansible/roles', '/Users/***/***/roles']
DEFAULT_VAULT_PASSWORD_FILE(env: ANSIBLE_VAULT_PASSWORD_FILE) = /Users/***/.ansible/vault_password
DEPRECATION_WARNINGS(env: ANSIBLE_DEPRECATION_WARNINGS) = False
HOST_KEY_CHECKING(env: ANSIBLE_HOST_KEY_CHECKING) = False

OS / ENVIRONMENT

ProductName:	Mac OS X
ProductVersion:	10.14.6
BuildVersion:	18G95

STEPS TO REPRODUCE

 name: "Test"
  hosts: 127.0.0.1
  tasks:
  - debug:
      msg: "{{ lookup('community.general.hashi_vault', 'secret=kv/data/foo:secret')}}"

EXPECTED RESULTS

PLAY [Test] **************************************************************************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1]

TASK [debug] *************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
    "msg": "bar"

PLAY RECAP ***************************************************************************************************************************************************************************************************************************************************************************************************************************
127.0.0.1                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

ACTUAL RESULTS

LAY [Test] ************************************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************************
ok: [127.0.0.1]

TASK [debug] ***********************************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'community.general.hashi_vault'. Error was a <class 'ansible.errors.AnsibleError'>, original message: Invalid Hashicorp Vault Token Specified for hashi_vault lookup."}

PLAY RECAP *************************************************************************************************************************************************************************************
127.0.0.1                  : ok=1    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

[briantist]

Thanks @rasta-rocket this is a good find. By the way, the hashi_vault lookup is in the process of being moved to a community supported collection dedicated to HashiCorp Vault, community.hashi_vault: https://github.com/ansible-collections/community.hashi_vault

I've requested that this issue be moved there, so that should hopefully happen without you needing to resubmit it.

---

As for the issue at hand, first I want to sincerely thank you for the in-depth investigation; extremely helpful.

I looked back and found some relevant discussion in this issue raised in hvac (in 2015): hvac/hvac#29

It seems the issue was even worse at that time because Vault itself did not include lookup-self in default policies, but even then they declined to change is_authenticated(), so I doubt we'll see much movement in that library.

That leads us to figure out what to do with it here.

I agree that we shouldn't completely stop a valid token from working just for the sake of trying to give a better error message.

• We could remove that check, but the downside then is that we have no other way to test the validity other than trying to access the secret, at which point it's indistinguishable from any other access denied; that ends up being a poorer user experience when someone's valid tokens have lookup-self, but an incorrect or expired token was passed in.
• We could change it to a warning and continue on. If we do that, users like yourself will see an ugly (unactionable) warning on every run.
• We could add some sort of option that allows you to optionally turn off the self-lookup check. This one ends up fitting the bill best for everyone in my opinion. Its main downside is adding yet-another-option.

Let me know what you think and if you have any additional ideas.

[dmsimard]

Hi @rasta-rocket, FYI I've moved this issue to the community.hashi_vault collection repository.

[rasta-rocket]

Hi @briantist thanks for your quick answer

Your analysis of the situation sounds good to me 👍

I would say that the 3rd option (= extra option to disable the check) might be the best because as you said: it will fit all the use cases 😜

Let me know if it is doable for you and if you need a hand on that.

Cheers