Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(security): Remove Vault dependency on Consul by using file backend #2886

Merged

Conversation

jim-wang-intel
Copy link
Contributor

@jim-wang-intel jim-wang-intel commented Nov 18, 2020

Change the Vault backend storage configuration from Consul to filesystem so that we can remove Vault dependency on Consul.

The docker-compose file will also need to change to remove the -consul dependency from repo developer-script.

The current edgex-consul healthy check on Vault also need to be remove as that check becomes superfluous.

Closes: #2882

Signed-off-by: Jim Wang yutsung.jim.wang@intel.com

PR Checklist

Please check if your PR fulfills the following requirements:

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/master/.github/Contributing.md.

What is the current behavior?

Currently the Vault's backend storage is Consul and thus has the dependency on Consul.

Issue Number: #2882

What is the new behavior?

To break and remove the direct dependency on Consul. After this PR is merged, Vault won't directly depend on Consul.

Does this PR introduce a breaking change?

  • Yes
  • No

New Imports

  • Yes
  • No

Specific Instructions

Are there any specific instructions or things that should be known prior to reviewing?
Pending on the related ADR to be approved

Other information

To test it, one can use docker-compose file generated from developer-script and remove the dependency on consul for vault service like:

  vault:
    image: vault:${VAULT_VERSION}
    container_name: edgex-vault
    hostname: edgex-vault
    networks:
      - edgex-network
    ports:
      - "127.0.0.1:8200:8200"
    cap_add:
      - "IPC_LOCK"
    tmpfs:
      - /vault/config
    entrypoint: ["/vault/init/start_vault.sh"]
    environment:
      VAULT_ADDR: https://edgex-vault:8200
      VAULT_CONFIG_DIR: /vault/config
      VAULT_UI: "true"
    volumes:
      - vault-file:/vault/file:z
      - vault-logs:/vault/logs:z
      - vault-init:/vault/init:ro,z
      - /tmp/edgex/secrets/edgex-vault:/tmp/edgex/secrets/edgex-vault:ro,z
    depends_on:
      - security-secrets-setup

Note that the services list under the depends_on now does not have consul any more. In other words, it is changed from

   depends_on:
    -consul
    -security-secrets-setup

to

   depends_on:
    -security-secrets-setup

Run docker-compose up vault to test it and see there is no error in the docker logs regarding the Vault itself.

jim@jim-NUC7i5DNHE:~/go/src/github.com/edgexfoundry/developer-scripts/compose-builder$ docker logs edgex-vault
VAULT_LOCAL_CONFIG: listener "tcp" { address = "edgex-vault:8200" tls_disable = "0" cluster_address = "edgex-vault:8201" tls_min_version = "tls12" tls_client_ca_file ="/tmp/edgex/secrets/edgex-vault/ca.pem" tls_cert_file ="/tmp/edgex/secrets/edgex-vault/server.crt" tls_key_file = "/tmp/edgex/secrets/edgex-vault/server.key" tls_perfer_server_cipher_suites = "true" } backend "file" { path = "vault/file" } default_lease_ttl = "168h" max_lease_ttl = "720h"
Starting edgex-vault...
==> Vault server configuration:

                     Cgo: disabled
              Go Version: go1.14.7
              Listener 1: tcp (addr: "edgex-vault:8200", cluster address: "edgex-vault:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "enabled")
               Log Level: debug
                   Mlock: supported: true, enabled: true
           Recovery Mode: false
                 Storage: file
                 Version: Vault v1.5.3
             Version Sha: 9fcd81405feb320390b9d71e15a691c3bc1daeef

==> Vault server started! Log data will stream in below:

2020-11-18T21:45:43.297Z [INFO]  proxy environment: http_proxy= https_proxy= no_proxy=
2020-11-18T21:45:43.299Z [WARN]  no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2020-11-18T21:45:43.299Z [DEBUG] storage.cache: creating LRU cache: size=0
2020-11-18T21:45:43.341Z [DEBUG] cluster listener addresses synthesized: cluster_addresses=[192.168.16.3:8201]
2020-11-18T21:45:46.083Z [INFO]  core: security barrier not initialized
2020-11-18T21:45:46.131Z [INFO]  core: security barrier not initialized
2020-11-18T21:45:46.131Z [INFO]  core: security barrier initialized: stored=1 shares=5 threshold=3
2020-11-18T21:45:46.132Z [DEBUG] core: cluster name not found/set, generating new
2020-11-18T21:45:46.132Z [DEBUG] core: cluster name set: name=vault-cluster-f78a221b
2020-11-18T21:45:46.132Z [DEBUG] core: cluster ID not found, generating new
2020-11-18T21:45:46.132Z [DEBUG] core: cluster ID set: id=76cb4446-f698-4eea-2f5c-3acb26411626
2020-11-18T21:45:46.132Z [INFO]  core: post-unseal setup starting
2020-11-18T21:45:46.132Z [DEBUG] core: clearing forwarding clients
2020-11-18T21:45:46.132Z [DEBUG] core: done clearing forwarding clients
2020-11-18T21:45:46.132Z [DEBUG] core: persisting feature flags
2020-11-18T21:45:46.143Z [INFO]  core: loaded wrapping token key
2020-11-18T21:45:46.143Z [INFO]  core: successfully setup plugin catalog: plugin-directory=
2020-11-18T21:45:46.143Z [INFO]  core: no mounts; adding default mount table
2020-11-18T21:45:46.143Z [INFO]  core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2020-11-18T21:45:46.144Z [INFO]  core: successfully mounted backend: type=system path=sys/
2020-11-18T21:45:46.144Z [INFO]  core: successfully mounted backend: type=identity path=identity/
2020-11-18T21:45:46.145Z [INFO]  core: successfully enabled credential backend: type=token path=token/
2020-11-18T21:45:46.145Z [INFO]  core: restoring leases
2020-11-18T21:45:46.145Z [DEBUG] expiration: collecting leases
2020-11-18T21:45:46.145Z [DEBUG] expiration: leases collected: num_existing=0
2020-11-18T21:45:46.145Z [INFO]  rollback: starting rollback manager
2020-11-18T21:45:46.146Z [INFO]  expiration: lease restore complete
2020-11-18T21:45:46.147Z [DEBUG] identity: loading entities
2020-11-18T21:45:46.147Z [DEBUG] identity: entities collected: num_existing=0
2020-11-18T21:45:46.147Z [INFO]  identity: entities restored
2020-11-18T21:45:46.147Z [DEBUG] identity: identity loading groups
2020-11-18T21:45:46.147Z [DEBUG] identity: groups collected: num_existing=0
2020-11-18T21:45:46.147Z [INFO]  identity: groups restored
2020-11-18T21:45:46.147Z [INFO]  core: post-unseal setup complete
2020-11-18T21:45:46.147Z [INFO]  core: usage gauge collection is disabled
2020-11-18T21:45:46.148Z [INFO]  core: root token generated
2020-11-18T21:45:46.148Z [INFO]  core: pre-seal teardown starting
2020-11-18T21:45:46.148Z [DEBUG] expiration: stop triggered
2020-11-18T21:45:46.148Z [DEBUG] expiration: finished stopping
2020-11-18T21:45:46.148Z [INFO]  rollback: stopping rollback manager
2020-11-18T21:45:46.148Z [INFO]  core: pre-seal teardown complete
2020-11-18T21:45:46.149Z [DEBUG] core: unseal key supplied
2020-11-18T21:45:46.149Z [DEBUG] core: cannot unseal, not enough keys: keys=1 threshold=3 nonce=c04bbf35-f6e3-b124-b65e-760d26df066c
2020-11-18T21:45:46.150Z [DEBUG] core: unseal key supplied
2020-11-18T21:45:46.150Z [DEBUG] core: cannot unseal, not enough keys: keys=2 threshold=3 nonce=c04bbf35-f6e3-b124-b65e-760d26df066c
2020-11-18T21:45:46.150Z [DEBUG] core: unseal key supplied
2020-11-18T21:45:46.150Z [DEBUG] core: starting cluster listeners
2020-11-18T21:45:46.150Z [INFO]  core.cluster-listener.tcp: starting listener: listener_address=192.168.16.3:8201
2020-11-18T21:45:46.150Z [INFO]  core.cluster-listener: serving cluster requests: cluster_listen_address=192.168.16.3:8201
2020-11-18T21:45:46.151Z [INFO]  core: post-unseal setup starting
2020-11-18T21:45:46.151Z [DEBUG] core: clearing forwarding clients
2020-11-18T21:45:46.151Z [DEBUG] core: done clearing forwarding clients
2020-11-18T21:45:46.151Z [DEBUG] core: persisting feature flags
2020-11-18T21:45:46.151Z [INFO]  core: loaded wrapping token key
2020-11-18T21:45:46.151Z [INFO]  core: successfully setup plugin catalog: plugin-directory=
2020-11-18T21:45:46.151Z [INFO]  core: successfully mounted backend: type=system path=sys/
2020-11-18T21:45:46.151Z [INFO]  core: successfully mounted backend: type=identity path=identity/
2020-11-18T21:45:46.151Z [INFO]  core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2020-11-18T21:45:46.152Z [INFO]  core: successfully enabled credential backend: type=token path=token/
2020-11-18T21:45:46.152Z [INFO]  core: restoring leases
2020-11-18T21:45:46.153Z [DEBUG] identity: loading entities
2020-11-18T21:45:46.153Z [DEBUG] identity: entities collected: num_existing=0
2020-11-18T21:45:46.153Z [INFO]  rollback: starting rollback manager
2020-11-18T21:45:46.153Z [DEBUG] expiration: collecting leases
2020-11-18T21:45:46.153Z [DEBUG] expiration: leases collected: num_existing=0
2020-11-18T21:45:46.153Z [INFO]  identity: entities restored
2020-11-18T21:45:46.153Z [DEBUG] identity: identity loading groups
2020-11-18T21:45:46.153Z [DEBUG] identity: groups collected: num_existing=0
2020-11-18T21:45:46.153Z [INFO]  identity: groups restored
2020-11-18T21:45:46.153Z [DEBUG] core: request forwarding setup function
2020-11-18T21:45:46.153Z [DEBUG] core: clearing forwarding clients
2020-11-18T21:45:46.153Z [DEBUG] core: done clearing forwarding clients
2020-11-18T21:45:46.153Z [DEBUG] core: request forwarding not setup
2020-11-18T21:45:46.153Z [DEBUG] core: leaving request forwarding setup function
2020-11-18T21:45:46.153Z [INFO]  core: post-unseal setup complete
2020-11-18T21:45:46.153Z [INFO]  core: vault is unsealed
2020-11-18T21:45:46.153Z [INFO]  expiration: lease restore complete
2020-11-18T21:45:46.153Z [INFO]  core: usage gauge collection is disabled
2020-11-18T21:45:47.213Z [INFO]  core: root generation initialized: nonce=00a198cb-c281-954c-6204-7d45081cb709
2020-11-18T21:45:47.214Z [DEBUG] core: cannot generate root, not enough keys: keys=1 threshold=3
2020-11-18T21:45:47.215Z [DEBUG] core: cannot generate root, not enough keys: keys=2 threshold=3
2020-11-18T21:45:47.215Z [INFO]  core: root generation finished: nonce=00a198cb-c281-954c-6204-7d45081cb709
2020-11-18T21:45:47.226Z [INFO]  expiration: revoked lease: lease_id=auth/token/root/h602fa89a044d262007277be0107fe5540bc751f4478cfbda9a4ed5a9be99c72e
2020-11-18T21:45:47.445Z [INFO]  core: successful mount: namespace= path=secret/ type=kv
2020-11-18T21:45:48.398Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h289724c68798321bf3c27394fd3754ae889239367e340e3b4afdc2ad872428f0
2020-11-18T21:45:48.400Z [INFO]  expiration: revoked lease: lease_id=auth/token/root/h1180769418dec0c6f882aefeed795bc86b6b691579cf7c3b41ae9694404d270c
2020-11-18T22:45:47.379Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h99a06bbbd285fd59966d71abbe21376d6d8bf0bbd6db4b5432095fb8b9be95cc
2020-11-18T22:45:47.385Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h07f50ee80459db238eef5c782d83fae66c2a68fdcef25aae2a166f8e3fd590bb
2020-11-18T22:45:47.397Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h726fb5b24348474758d85ffdcd9aa60ddc23a793bc435016cc3ab7a805886604
2020-11-18T22:45:47.400Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h8d722daa4f3b72d934e60e7f3676384fdc3e72bec889e2bb972da70fed998e2c
2020-11-18T22:45:47.406Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h2acae2f64b48f9458dbff8b2710f4f9cd71da77bedd1bee671bc809ff3609e37
2020-11-18T22:45:47.412Z [INFO]  expiration: revoked lease: lease_id=auth/token/create/h99352a69d0bb3cdfa1a55d1ea9e8107f772e98e8849a1a4d163e197ad56b2cca

@codecov-io
Copy link

codecov-io commented Nov 18, 2020

Codecov Report

Merging #2886 (565a8fe) into master (b220847) will increase coverage by 0.40%.
The diff coverage is 58.27%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2886      +/-   ##
==========================================
+ Coverage   40.25%   40.66%   +0.40%     
==========================================
  Files         170      170              
  Lines       13873    14156     +283     
==========================================
+ Hits         5585     5756     +171     
- Misses       7924     8023      +99     
- Partials      364      377      +13     
Impacted Files Coverage Δ
internal/pkg/v2/infrastructure/redis/client.go 0.00% <0.00%> (ø)
internal/pkg/v2/infrastructure/redis/device.go 0.00% <0.00%> (ø)
...rnal/pkg/v2/infrastructure/redis/device_profile.go 0.00% <0.00%> (ø)
internal/pkg/v2/infrastructure/redis/event.go 0.00% <0.00%> (ø)
internal/pkg/v2/infrastructure/redis/queries.go 0.00% <0.00%> (ø)
internal/pkg/v2/infrastructure/redis/reading.go 0.00% <0.00%> (ø)
internal/core/data/v2/controller/http/reading.go 72.09% <76.00%> (+5.42%) ⬆️
...nternal/core/metadata/v2/controller/http/device.go 92.21% <77.77%> (-1.70%) ⬇️
.../core/metadata/v2/controller/http/deviceprofile.go 89.27% <78.57%> (-1.04%) ⬇️
...ernal/security/config/command/proxy/tls/command.go 84.39% <82.40%> (-15.61%) ⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 382321c...565a8fe. Read the comment docs.

@jim-wang-intel jim-wang-intel force-pushed the remove-vault-dependent-consul branch 2 times, most recently from 3a79bd6 to 7192dc2 Compare November 19, 2020 22:54
@jim-wang-intel jim-wang-intel marked this pull request as ready for review November 19, 2020 22:58
lenny-goodell
lenny-goodell previously approved these changes Nov 19, 2020
Copy link
Member

@lenny-goodell lenny-goodell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, moved to draft until ADR is Approved.

@lenny-goodell lenny-goodell marked this pull request as draft November 19, 2020 23:17
lenny-goodell
lenny-goodell previously approved these changes Nov 20, 2020
Copy link
Member

@lenny-goodell lenny-goodell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

bnevis-i
bnevis-i previously approved these changes Dec 1, 2020
lenny-goodell
lenny-goodell previously approved these changes Dec 1, 2020
@bnevis-i
Copy link
Collaborator

bnevis-i commented Dec 7, 2020

@jim-wang-intel Merge conflicts.

@jim-wang-intel
Copy link
Contributor Author

@jim-wang-intel Merge conflicts.

Conflicts resolved. Please check.

bnevis-i
bnevis-i previously approved these changes Dec 7, 2020
…nd instead of consul

 Change the Vault backend storage configuration from Consul to Filesystem so that we can remove the Vault dependency on Consul.

 The docker-compose file will also need to change to remove the -consul dependency from repo developer-script.

 The current edgex-consul healthy check on Vault also need to be remove as that check becomes superfluous.

 Closes: edgexfoundry#2882

Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
Resolve conflicts with master

Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
@sonarcloud
Copy link

sonarcloud bot commented Dec 10, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

redirect_addr = "http://edgex-vault:8200"
cluster_addr = "http://edgex-vault:8201"
}
backend "file" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this go once this service is removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will go to "bootstrapper" i guess and this one is taking the baby step. so in terms of removing is beyond the scope of this related issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue #2882

Copy link
Collaborator

@bnevis-i bnevis-i Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It goes straight into the docker-compose file as static configuration via env var (TLS was generating the variable parameters)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also check to see if the snap depends on this script to generate the config file, or uses something else.

Copy link
Contributor Author

@jim-wang-intel jim-wang-intel Dec 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lenny-intel please suggest a change. i do not know what you want me to do.

Copy link
Member

@lenny-goodell lenny-goodell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lenny-goodell lenny-goodell merged commit f9701ca into edgexfoundry:master Dec 10, 2020
jim-wang-intel added a commit to jim-wang-intel/edgex-go that referenced this pull request Dec 15, 2020
…nd (edgexfoundry#2886)

* feat(security): Remove Vault dependency on Consul by using file backend instead of consul

 Change the Vault backend storage configuration from Consul to Filesystem so that we can remove the Vault dependency on Consul.

 The docker-compose file will also need to change to remove the -consul dependency from repo developer-script.

 The current edgex-consul healthy check on Vault also need to be remove as that check becomes superfluous.

 Closes: edgexfoundry#2882
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3-high priority denoting release-blocking issues enhancement New feature or request ireland security-services
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove Vault dependency on Consul
4 participants