Skip to content

Commit

Permalink
incident_catalog_entries
Browse files Browse the repository at this point in the history 
Introduce a resource that can manage entries in bulk, which:

1. Solves terraform performance issues when dealing with a lot of
   entries
2. Reduces pressure on our API when working with entries

To achieve this we've forked the terraform sdk framework to improve
performance when logging, which is necessary to avoid massive plan times
on large nested objects.

Relevant links are:
- PR for logging change
  (hashicorp/terraform-plugin-framework#722)
- Issue in upstream
  (hashicorp/terraform-plugin-framework#721)
  • Loading branch information
@lawrencejones
lawrencejones committed Apr 11, 2023
1 parent 065ca94 commit 6d130be
Show file tree
Hide file tree
Showing 12 changed files with 6,749 additions and 338 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 1.4.0

- incident_catalog_entries for large entry counts

## 1.3.1

- Fix bug around omitted empty arrays
Expand Down
214 changes: 214 additions & 0 deletions docs/resources/catalog_entries.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,214 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "incident_catalog_entries Resource - terraform-provider-incident"
subcategory: ""
description: |-
This resource manages all entries for a given catalog type and should be used when
loading many (>100) catalog entries to ensure fast and reliable plans.
Please note that this resource is authoritative, in that it will delete all entries from
the catalog type that it doesn't manage, even those created outside of terraform.
If you have a catalog source such as Backstage or some custom catalog you'd like to sync
into incident.io, this is the recommended way of achieving that.
External IDs
As this resource loads content from an existing catalog source into the incident.io
catalog, it requires that each entry is given a stable identifier that can uniquely
identify it in the upstream system.
We call this the 'external ID' and might be something like:
The ID of the entry in a custom catalog, often the primary key of the entryAny stable human identifier (often called a slug) that uniquely reference the entry
This external ID is what we use as a map key for the entries attribute, and how we map
changes to one entry to an update to that same entry when the upstream changes.
---

# incident_catalog_entries (Resource)

This resource manages all entries for a given catalog type and should be used when
loading many (>100) catalog entries to ensure fast and reliable plans.

Please note that this resource is authoritative, in that it will delete all entries from
the catalog type that it doesn't manage, even those created outside of terraform.

If you have a catalog source such as Backstage or some custom catalog you'd like to sync
into incident.io, this is the recommended way of achieving that.

## External IDs

As this resource loads content from an existing catalog source into the incident.io
catalog, it requires that each entry is given a stable identifier that can uniquely
identify it in the upstream system.

We call this the 'external ID' and might be something like:

- The ID of the entry in a custom catalog, often the primary key of the entry
- Any stable human identifier (often called a slug) that uniquely reference the entry

This external ID is what we use as a map key for the entries attribute, and how we map
changes to one entry to an update to that same entry when the upstream changes.

## Example Usage

```terraform
# Load a catalog from a JSON file. You may also use an HTTP endpoint or some
# other data source, or prepare this file using a script before terraform runs.
#
# We'll use an example taken from Backstage:
# https://backstage.io/docs/features/software-catalog/descriptor-format
#
/*
{
"apiVersion": "backstage.io/v1alpha1",
"kind": "Component",
"metadata": {
"annotations": {
"backstage.io/managed-by-location": "file:/tmp/catalog-info.yaml",
"example.com/service-discovery": "artistweb",
"circleci.com/project-slug": "github/example-org/artist-website"
},
"description": "The place to be, for great artists",
"etag": "ZjU2MWRkZWUtMmMxZS00YTZiLWFmMWMtOTE1NGNiZDdlYzNk",
"labels": {
"example.com/custom": "custom_label_value"
},
"links": [
{
"url": "https://admin.example-org.com",
"title": "Admin Dashboard",
"icon": "dashboard",
"type": "admin-dashboard"
}
],
"tags": [
"java"
],
"name": "artist-web",
"uid": "2152f463-549d-4d8d-a94d-ce2b7676c6e2"
},
"spec": {
"lifecycle": "production",
"owner": "artist-relations-team",
"type": "website",
"system": "public-websites"
}
}
*/
locals {
catalog = {
for entry in jsondecode(file("catalog.json")) : entry["uid"] => entry
}
}
################################################################################
# Create the type
################################################################################
# Define the catalog type, creating attributes that map to the values in the
# catalog data source.
resource "incident_catalog_type" "service" {
name = "Service"
description = "All services that we run at Example Org"
}
resource "incident_catalog_type_attribute" "service_owner" {
catalog_type_id = incident_catalog_type.service.id
name = "Owner"
type = "String"
}
resource "incident_catalog_type_attribute" "service_description" {
catalog_type_id = incident_catalog_type.service.id
name = "Description"
type = "String"
}
resource "incident_catalog_type_attribute" "service_tags" {
catalog_type_id = incident_catalog_type.service.id
name = "Tags"
type = "String"
array = true
}
################################################################################
# Provision the entries
################################################################################
# This is where we create all the entries. If we have any entries in Service
# that are not defined in this resource, we will delete them.
resource "incident_catalog_entries" "services" {
id = incident_catalog_type.service.id
entries = {
# Map from the catalog external ID => entry value.
for external_id, entry in local.catalog :
# e.g. 2152f463-549d-4d8d-a94d-ce2b7676c6e2
external_id => {
# e.g. artist-web
name = entry["metadata"]["name"],
# In this catalog we know names are unique, so we can use them as a
# human-friendly unique alias. Other catalogs name may not be unique, in
# which case this would fail.
alias = entry["metadata"]["name"],
# Now build all attribute values for this entry, with an object
# comprehension that filters out any attributes that we are missing values
# for.
attribute_values = {
for attribute, binding in {
# Owner (e.g. artist-relations-team)
(incident_catalog_type_attribute.service_owner.id) = {
value = try(entry["spec"]["owner"], null)
},
# Description (e.g. The place to be, for great artists)
(incident_catalog_type_attribute.service_description.id) = {
value = try(entry["metadata"]["description"], null)
},
# Tags (e.g. ["java"])
(incident_catalog_type_attribute.service_tags.id) = {
array_value = try(entry["metadata"]["tags"], null)
},
} : attribute => binding if try(binding.value, binding.array_value) != null
}
}
}
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `entries` (Attributes Map) Map of external ID to entry in the catalog. (see [below for nested schema](#nestedatt--entries))
- `id` (String) ID of this catalog type

<a id="nestedatt--entries"></a>
### Nested Schema for `entries`

Required:

- `attribute_values` (Attributes Map) (see [below for nested schema](#nestedatt--entries--attribute_values))
- `name` (String) Name is the human readable name of this entry

Optional:

- `alias` (String) An optional alias that must uniquely identify this type
- `rank` (Number) When catalog type is ranked, this is used to help order things

Read-Only:

- `id` (String) ID of this resource

<a id="nestedatt--entries--attribute_values"></a>
### Nested Schema for `entries.attribute_values`

Optional:

- `array_value` (List of String) The value of this element of the array, in a format suitable for this attribute type.
- `value` (String) The value of this attribute, in a format suitable for this attribute type.


129 changes: 129 additions & 0 deletions examples/resources/incident_catalog_entries/resource.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Load a catalog from a JSON file. You may also use an HTTP endpoint or some
# other data source, or prepare this file using a script before terraform runs.
#
# We'll use an example taken from Backstage:
# https://backstage.io/docs/features/software-catalog/descriptor-format
#
/*
{
"apiVersion": "backstage.io/v1alpha1",
"kind": "Component",
"metadata": {
"annotations": {
"backstage.io/managed-by-location": "file:/tmp/catalog-info.yaml",
"example.com/service-discovery": "artistweb",
"circleci.com/project-slug": "github/example-org/artist-website"
},
"description": "The place to be, for great artists",
"etag": "ZjU2MWRkZWUtMmMxZS00YTZiLWFmMWMtOTE1NGNiZDdlYzNk",
"labels": {
"example.com/custom": "custom_label_value"
},
"links": [
{
"url": "https://admin.example-org.com",
"title": "Admin Dashboard",
"icon": "dashboard",
"type": "admin-dashboard"
}
],
"tags": [
"java"
],
"name": "artist-web",
"uid": "2152f463-549d-4d8d-a94d-ce2b7676c6e2"
},
"spec": {
"lifecycle": "production",
"owner": "artist-relations-team",
"type": "website",
"system": "public-websites"
}
}
*/
locals {
catalog = {
for entry in jsondecode(file("catalog.json")) : entry["uid"] => entry
}
}

################################################################################
# Create the type
################################################################################

# Define the catalog type, creating attributes that map to the values in the
# catalog data source.
resource "incident_catalog_type" "service" {
name = "Service"
description = "All services that we run at Example Org"
}

resource "incident_catalog_type_attribute" "service_owner" {
catalog_type_id = incident_catalog_type.service.id

name = "Owner"
type = "String"
}

resource "incident_catalog_type_attribute" "service_description" {
catalog_type_id = incident_catalog_type.service.id

name = "Description"
type = "String"
}

resource "incident_catalog_type_attribute" "service_tags" {
catalog_type_id = incident_catalog_type.service.id

name = "Tags"
type = "String"
array = true
}

################################################################################
# Provision the entries
################################################################################

# This is where we create all the entries. If we have any entries in Service
# that are not defined in this resource, we will delete them.
resource "incident_catalog_entries" "services" {
id = incident_catalog_type.service.id

entries = {
# Map from the catalog external ID => entry value.
for external_id, entry in local.catalog :

# e.g. 2152f463-549d-4d8d-a94d-ce2b7676c6e2
external_id => {
# e.g. artist-web
name = entry["metadata"]["name"],

# In this catalog we know names are unique, so we can use them as a
# human-friendly unique alias. Other catalogs name may not be unique, in
# which case this would fail.
alias = entry["metadata"]["name"],

# Now build all attribute values for this entry, with an object
# comprehension that filters out any attributes that we are missing values
# for.
attribute_values = {
for attribute, binding in {
# Owner (e.g. artist-relations-team)
(incident_catalog_type_attribute.service_owner.id) = {
value = try(entry["spec"]["owner"], null)
},

# Description (e.g. The place to be, for great artists)
(incident_catalog_type_attribute.service_description.id) = {
value = try(entry["metadata"]["description"], null)
},

# Tags (e.g. ["java"])
(incident_catalog_type_attribute.service_tags.id) = {
array_value = try(entry["metadata"]["tags"], null)
},
} : attribute => binding if try(binding.value, binding.array_value) != null
}
}
}
}
Loading

0 comments on commit 6d130be

Please sign in to comment.