[WIP] Data-driven Terraform Configuration #4961
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
24 tasks
|
@phinze here's the same changeset but in the main repo. Feel free to pick up the reins! I'll check in here again when things calm down for me and see if there's anything I can do to help finish this up. |
|
Thanks @apparentlymart! Will dig in soon. |
|
after @jen20 explained what this means, I am very excited to see it - this will mean that I can ditch so many custom modules that do lookups :) |
phinze
changed the title
|
This functionality is much required and looking forward to it! Do we have any rough release that we think this will make it into? |
|
@nickithewatt We're currently targeting 0.7 - Terraform's next major release. Not sure on the timeline quite yet, but suffice it to say this one is high on our list of core improvements! |
This was referenced Mar 25, 2016
apparentlymart
force-pushed
the
data-sources
branch
from
28d6921
to
342027c
Compare
This was referenced Apr 11, 2016
apparentlymart
force-pushed
the
data-sources
branch
2 times, most recently
from
0a769b3
to
6633d0c
Compare
3 tasks
This represents a data source configuration.
This allows the config loader to read "data" blocks from the config and turn them into DataSource objects. This just reads the data from the config file. It doesn't validate the data nor do anything useful with it.
This will undoubtedly evolve as implementation continues, but this is some initial documentation based on the design doc.
This is a breaking change to the ResourceProvider interface to add the new new methods DataSources and RefreshData, which are the two provider functions related to data sources. Along with breaking the interface, we also fix up the RPC client/server and helper/schema implementations of it, which are all of the callers used when provider plugins use helper/schema. This would be a breaking change for any provider plugin that directly implements the provider interface, but no known plugins do this and it is not recommended. The helper/schema implementation is currently a no-op stub which will be elaborated in a later change.
The namespaces of data sources and resources are separate, but at this layer they both share the "Resource" implementation, with data sources differing from resources only in that they are not permitted to provide Create, Update or Delete implementations.
This is a further breaking change to the ResourceProvider interface, and the updates to all of its significant implementations to support the new method.
In the "schema" layer a Resource is just any "thing" that has a schema and supports some or all of the CRUD operations. Data sources introduce a new use of Resource to represent read-only resources, which require some different InternalValidate logic.
Historically we've had some "read-only" and "logical" resources. With the addition of the data source concept these will gradually become data sources, but we need to retain backward compatibility with existing configurations that use the now-deprecated resources. This shim is intended to allow us to easily create a resource from a data source implementation. It adjusts the schema as needed and adds stub Create and Delete implementations. This would ideally also produce a deprecation warning whenever such a shimmed resource is used, but the schema system doesn't currently have a mechanism for resource-specific validation, so that remains just a TODO for the moment.
As a first example of a real-world data source, the pre-existing terraform_remote_state resource is adapted to be a data source. The original resource is shimmed to wrap the data source for backward compatibility.
apparentlymart
force-pushed
the
data-sources
branch
from
8d2b959
to
301e7e2
Compare
This allows ${data.TYPE.NAME.FIELD} interpolation syntax at the
configuration level, though there is not yet any implementation of
resolving these things in the core package.
24 tasks
|
I rebased this onto the |
24 tasks
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is where I'm working on the implementation of the proposal from #4169.
DataSourcetype in the config moduledatablocks from configuration filesdatablocksdata.TYPE.NAME.ATTR, with the same multi-capabilities as resourcesconfigpackage)terraformpackage)terraformmoduleRefreshDatamethod on theResourceProviderinterfaceRefreshDatasupport inhelper/schemacountimplementation to allow any non-computed variable, which will then include attributes of any data instances that do not depend on resource creationhelper/schema, allowing specific data source implementations to be aliased as resource implementationsterraform_remote_statedata source based on the existing resource, as an initial example