Support test results as a view dbt-core PR #8653

[FishtownBuildBot]

dbt-labs/dbt-core#8653

At a minimum, update body to include a link to the page on docs.getdbt.com requiring updates and what part(s) of the page you would like to see updated.

Pages:

• update https://docs.getdbt.com/docs/build/tests#storing-test-failures
• create a new page similar to https://docs.getdbt.com/reference/resource-configs/store_failures
• maybe some other places too? 🤷

[runleonarun]

@dbeatty10 👋🏻
There's a ton of info in the linked issue and PR. Can you help us understand this change and which docs it affects? Here's my assessment so far:

It sounds like we're introducing this feature to provide a more lightweight storage than storing test failures in tables because that's resource heavy and possibly unnecessary.

Definition: Storing failures lets you examine records that cause tests to fail, so you can either repair the data or change the test as needed.

I'm guessing we need:

• a page similar to https://docs.getdbt.com/reference/resource-configs/store_failures
• Acceptable values are 'ephemeral', 'table', 'view'
• store_failures_as is an optional string.
• Setting store_failures_as overrides any existing setting for store_failures

1. Does store_failures_as apply to Specific test, Singular test, Generic test block, and Project level like store_failures does?
2. Can you help with some code examples too?

🙏🏻 thanks!

[dbeatty10]

tl;dr

For the test resource type, store_failures_as is an optional config that specifies how test failures should be stored in the database (and takes precedence over store_failures). The three valid values are:

1. ephemeral - nothing stored in the database (default)
2. table - test failures stored as a database table
3. view - test failures stored as a database view

Where it applies

It is configurable in all the same places as store_failures, including singular tests (.sql files), generic tests (.yml files), and dbt_project.yml. See examples below for details.

Examples

Singular test

Singular test in tests/singular/check_something.sql file

{{ config(store_failures_as="table") }}

-- custom singular test
select 1 as id
where 1=0

Generic test

Generic tests in models/_models.yml file

models:
  - name: my_model
    columns:
      - name: id
        tests:
          - not_null:
              config:
                store_failures_as: view
          - unique:
              config:
                store_failures_as: ephemeral

Project level

Config in dbt_project.yml

name: "my_project"
version: "1.0.0"
config-version: 2
profile: "sandcastle"

tests:
  my_project:
    +store_failures_as: table
    my_subfolder_1:
      +store_failures_as: view
    my_subfolder_2:
      +store_failures_as: ephemeral

"Clobbering" configs

Most configurations are "clobbered" when applied hierarchically, and store_failures_as is no different! Whenever a more-specific value is available, it will completely replace the less-specific value. Read more in the links below:

• Test configurations
• Test-specific configurations
• Configuring directories of models in dbt_project.yml
• Config inheritance