Managing Sensitive Data

Many of the services in the Hollowverse architecture require access to some sensitive data that cannot be exposed publicly (things like database passwords, private keys, API tokens and secret URLs).

We store these secrets in AWS Secrets Manager, which is a safe, centralized store for different types of sensitive data. A service in Hollowverse will usually fetch the secrets it needs from Secrets Manager on startup and store them in memory.

Restricting a certain service's access to only the secrets it needs is a good practice security-wise.

The integration between AWS Secrets Manager and AWS IAM service makes it easy to restrict the service access to specific secrets by limiting the list of resources that service can access in the IAM role policy.

This policy is defined in the Serverless framework configuration file, serverless.yml, of each repository in the Hollowverse organization.

For an example of an IAM role policy that lists the secrets required for a specific service, see serverlesss.yml in hollowverse/track-performance.

The secret names stored in AWS Secrets Manager follow a convention: each secret name must start with <stage>/ where <stage> is usually either production or development. This creates a clear separation between the development and production environments and helps avoid any unintended access to secrets designated for production which could negatively impact the live production services.

The helper functions readAwsSecretStringForStage and readAwsSecretStringsForStage provided by @hollowverse/utils help enforce this convention. They also provide a way to pass an alternative value for local development workflows where the developer cannot access the secrets.