From efb35616a0c7d8c8b30fb23e16e2e637500d20e2 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 22 Oct 2024 10:11:49 +0100 Subject: [PATCH 1/8] add data type and time zone considerations Adding clarifying information about time zones and data types per [internal slack thread](https://dbt-labs.slack.com/archives/C03KHQRQUBX/p1729278831684459?thread_ts=1729023422.431269&cid=C03KHQRQUBX) --- website/docs/docs/build/metricflow-time-spine.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index e932fb36f53..f475e7b4eb1 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -463,9 +463,20 @@ For example, if you use a custom calendar in your organization, such as a fiscal - This is useful for calculating metrics based on a custom calendar, such as fiscal quarters or weeks. - Use the `custom_granularities` key to define a non-standard time period for querying data, such as a `retail_month` or `fiscal_week`, instead of standard options like `day`, `month`, or `year`. -- Ensure the the `standard_granularity_column` is a date time type. - This feature provides more control over how time-based metrics are calculated. +### Data types and time zone considerations +When working with custom calendars in MetricFlow, it's important to ensure: + +- Consistent data types — Both your dimension column and the time spine column should use the same data type to allow accurate comparisons. Functions like `DATE_TRUNC` don't change the data type of the input in some databases (like Snowflake). Using different data types can lead to mismatches and inaccurate results. + + We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support higher granularities like hours or minutes. + +- Consistent time zones — Ensure that all your time-related data uses the same time zone. MetricFlow currently doesn't perform any timezone manipulation and inconsistent time zones can cause unexpected results during aggregations and comparisons. + +For example: +If your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type. + ### Add custom granularities To add custom granularities, the Semantic Layer supports custom calendar configurations that allow users to query data using non-standard time periods like `fiscal_year` or `retail_month`. You can define these custom granularities (all lowercased) by modifying your model's YAML configuration like this: From 19199a36d651c95806b13e7d02fe9df1a6379b9a Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 22 Oct 2024 10:16:34 +0100 Subject: [PATCH 2/8] Update website/docs/docs/build/metricflow-time-spine.md --- website/docs/docs/build/metricflow-time-spine.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index f475e7b4eb1..2d523338fe0 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -465,7 +465,7 @@ For example, if you use a custom calendar in your organization, such as a fiscal - Use the `custom_granularities` key to define a non-standard time period for querying data, such as a `retail_month` or `fiscal_week`, instead of standard options like `day`, `month`, or `year`. - This feature provides more control over how time-based metrics are calculated. -### Data types and time zone considerations + When working with custom calendars in MetricFlow, it's important to ensure: - Consistent data types — Both your dimension column and the time spine column should use the same data type to allow accurate comparisons. Functions like `DATE_TRUNC` don't change the data type of the input in some databases (like Snowflake). Using different data types can lead to mismatches and inaccurate results. From e689c8ad58bb9a0b4a02af811a1e181bc43ae3f5 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 22 Oct 2024 10:16:47 +0100 Subject: [PATCH 3/8] Update website/docs/docs/build/metricflow-time-spine.md --- website/docs/docs/build/metricflow-time-spine.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index 2d523338fe0..94b67fb8832 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -476,7 +476,7 @@ When working with custom calendars in MetricFlow, it's important to ensure: For example: If your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type. - + ### Add custom granularities To add custom granularities, the Semantic Layer supports custom calendar configurations that allow users to query data using non-standard time periods like `fiscal_year` or `retail_month`. You can define these custom granularities (all lowercased) by modifying your model's YAML configuration like this: From 742e216a2790c173e264fc6a11a7082942abf817 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 22 Oct 2024 10:18:55 +0100 Subject: [PATCH 4/8] Update metricflow-time-spine.md --- website/docs/docs/build/metricflow-time-spine.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index 94b67fb8832..1bee191d55c 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -466,17 +466,19 @@ For example, if you use a custom calendar in your organization, such as a fiscal - This feature provides more control over how time-based metrics are calculated. + When working with custom calendars in MetricFlow, it's important to ensure: - Consistent data types — Both your dimension column and the time spine column should use the same data type to allow accurate comparisons. Functions like `DATE_TRUNC` don't change the data type of the input in some databases (like Snowflake). Using different data types can lead to mismatches and inaccurate results. We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support higher granularities like hours or minutes. -- Consistent time zones — Ensure that all your time-related data uses the same time zone. MetricFlow currently doesn't perform any timezone manipulation and inconsistent time zones can cause unexpected results during aggregations and comparisons. +- Consistent time zones — Ensure that all your time-related data uses the same time zone. MetricFlow supports UTC and currently doesn't perform any timezone manipulation. This means inconsistent time zones can cause unexpected results during aggregations and comparisons. + +For example, if your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type. -For example: -If your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type. + ### Add custom granularities To add custom granularities, the Semantic Layer supports custom calendar configurations that allow users to query data using non-standard time periods like `fiscal_year` or `retail_month`. You can define these custom granularities (all lowercased) by modifying your model's YAML configuration like this: From 74a351ed200aa84f67721cb2900a14f2b5dbc220 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 23 Oct 2024 09:39:32 +0100 Subject: [PATCH 5/8] Update website/docs/docs/build/metricflow-time-spine.md --- website/docs/docs/build/metricflow-time-spine.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index 1bee191d55c..96546152f7e 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -471,7 +471,7 @@ When working with custom calendars in MetricFlow, it's important to ensure: - Consistent data types — Both your dimension column and the time spine column should use the same data type to allow accurate comparisons. Functions like `DATE_TRUNC` don't change the data type of the input in some databases (like Snowflake). Using different data types can lead to mismatches and inaccurate results. - We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support higher granularities like hours or minutes. + We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support smaller granularities like hours or minutes. - Consistent time zones — Ensure that all your time-related data uses the same time zone. MetricFlow supports UTC and currently doesn't perform any timezone manipulation. This means inconsistent time zones can cause unexpected results during aggregations and comparisons. From 180cb85ae09418edeaf47110d5970f873f6be446 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 23 Oct 2024 09:40:34 +0100 Subject: [PATCH 6/8] Update website/docs/docs/build/metricflow-time-spine.md --- website/docs/docs/build/metricflow-time-spine.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/metricflow-time-spine.md b/website/docs/docs/build/metricflow-time-spine.md index 96546152f7e..9932a35839c 100644 --- a/website/docs/docs/build/metricflow-time-spine.md +++ b/website/docs/docs/build/metricflow-time-spine.md @@ -473,7 +473,7 @@ When working with custom calendars in MetricFlow, it's important to ensure: We recommend using `DATETIME` or `TIMESTAMP` data types for your time dimensions and time spine, as they support all granularities. The `DATE` data type may not support smaller granularities like hours or minutes. -- Consistent time zones — Ensure that all your time-related data uses the same time zone. MetricFlow supports UTC and currently doesn't perform any timezone manipulation. This means inconsistent time zones can cause unexpected results during aggregations and comparisons. +- Time zones — MetricFlow currently doesn't perform any timezone manipulation. When working with timezone-aware data, inconsistent time zones may lead to unexpected results during aggregations and comparisons. For example, if your time spine column is `TIMESTAMP` type and your dimension column is `DATE` type, comparisons between these columns might not work as intended. To fix this, convert your `DATE` column to `TIMESTAMP`, or make sure both columns are the same data type. From c3e9d37f5dee9b346a8a16a621b28de640db6de6 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 23 Oct 2024 09:51:50 +0100 Subject: [PATCH 7/8] update agg_time_dimension example updating `agg_time_dimension` key examples to match valid values. raised in [internal slack](https://dbt-labs.slack.com/archives/C050Z6UP4RK/p1729560227396339) --- website/docs/docs/build/measures.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/docs/build/measures.md b/website/docs/docs/build/measures.md index 1901c7d05b7..977b630fada 100644 --- a/website/docs/docs/build/measures.md +++ b/website/docs/docs/build/measures.md @@ -102,7 +102,7 @@ semantic_models: description: A record of every transaction that takes place. Carts are considered multiple transactions for each SKU. model: ref('schema.transactions') defaults: - agg_time_dimension: metric_time + agg_time_dimension: transaction_date # --- entities --- entities: @@ -167,7 +167,7 @@ semantic_models: # --- dimensions --- dimensions: - - name: metric_time + - name: transaction_date type: time expr: date_trunc('day', ts) # expr refers to underlying column ts type_params: @@ -204,7 +204,7 @@ semantic_models: description: A subscription table with one row per date for each active user and their subscription plans. model: ref('your_schema.subscription_table') defaults: - agg_time_dimension: metric_time + agg_time_dimension: subscription_date entities: - name: user_id @@ -212,7 +212,7 @@ semantic_models: primary_entity: subscription_table dimensions: - - name: metric_time + - name: subscription_date type: time expr: date_transaction type_params: From ed8289260a5ea72355801430813f556ec56a3e30 Mon Sep 17 00:00:00 2001 From: Mirna Wong <89008547+mirnawong1@users.noreply.github.com> Date: Wed, 23 Oct 2024 10:17:49 +0100 Subject: [PATCH 8/8] Update metricflow-commands.md add versionblocks and update syntax based on versions. raised in [internal slack thread](https://dbt-labs.slack.com/archives/C02NCQ9483C/p1729633657302969) --- website/docs/docs/build/metricflow-commands.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/website/docs/docs/build/metricflow-commands.md b/website/docs/docs/build/metricflow-commands.md index d010cd8bb17..466a61cc5bc 100644 --- a/website/docs/docs/build/metricflow-commands.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -34,10 +34,22 @@ For dbt Cloud CLI users, MetricFlow commands are embedded in the dbt Cloud CLI, You can install [MetricFlow](https://github.com/dbt-labs/metricflow#getting-started) from [PyPI](https://pypi.org/project/dbt-metricflow/). You need to use `pip` to install MetricFlow on Windows or Linux operating systems: + + 1. Create or activate your virtual environment `python -m venv venv` 2. Run `pip install dbt-metricflow` * You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `python -m pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `python -m pip install "dbt-metricflow[snowflake]"` + + + + +1. Create or activate your virtual environment `python -m venv venv` +2. Run `pip install dbt-metricflow` + * You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `python -m pip install "dbt-metricflow[adapter_package_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `python -m pip install "dbt-metricflow[dbt-snowflake]"` + + + **Note**, you'll need to manage versioning between dbt Core, your adapter, and MetricFlow. Something to note, MetricFlow `mf` commands return an error if you have a Metafont latex package installed. To run `mf` commands, uninstall the package.