Option to remove sensitive content from logs

[drewmullen]

Proposal

TRACE & DEBUG logs currently include authentication information. For AWS this is things like:

• account ids
• iam ARNs
• sts tokens

this is fine usually but if i want to post a log to a gist for a github issue, i have to manually scrape it for sensitive content. It would be great if there was a flag or environment variable to remove or obfuscate that content.

I'd be in favor of this flag being set by default when you include TF_LOG_PATH. But thats up to yall

Thanks for the great work :)

[bflad]

Hi @drewmullen! 👋 Thank you so much for raising this and for including specific use cases. Super helpful.

This is certainly an area we are interested in exploring, since as you mention, the experience of filtering logs is not the best currently. There will be a fun intersection between things that providers themselves consider sensitive, such as tokens, and practitioners themselves consider sensitive, such as an internal domain name (which in of itself would be interesting in how practitioners could declare these safely). Design-wise it seems like we will need to support both cases with this filtering capability.

If I had to napkin sketch something real quick, I think it would be along the lines where a provider can make a call into the provider logging systems, let's call it tflog.Filter(what, how) for simplicity, that then all log messages are processed through filtering entries to obfuscate that value or potentially skip the whole entry any time it is found before its written out.

Conceptually, this functionality is very early in its planning and design phases so if you or others have additional use cases or ideas, it is all game to discuss. Thanks again!

[drewmullen]

@detro Now that this is released (thank you!!!) Can we start using it immediately or does functionality need to be baked into the next version of Core?

[bflad]

@drewmullen the terraform-plugin-log Go module is unrelated to Terraform CLI ("core") and updating on the provider side will be able to use the new functionality. Typically provider codebases on GitHub will enable Dependabot for Go dependency, but to manually upgrade a provider:

go get github.com/hashicorp/terraform-plugin-log
go mod tidy

If there are any compilation errors, it is likely due to needing to do the following other dependencies as well:

# If the provider is written with terraform-plugin-framework
go get github.com/hashicorp/terraform-plugin-framework@v0.10.0
go get github.com/hashicorp/terraform-plugin-sdk/v2@v2.19.0
go mod tidy

For more information about the log filtering functionality available:

• tflog Go Documentation
• terraform.io Documentation

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.