fix(dbt-derived-metrics): Support raw metrics and Jinja syntax

[Vitor-Avila]

dbt allows the creation of derived metrics with just raw SQL, not involving any other dbt metric:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
       (SUM(price_each)) * 1.2
      {% endraw %}

These would currently fail to sync, as derived metrics don't have a model definition, so currently the CLI identifies the desired model by getting the models used by the metrics used in the syntax (which don't exist in this case).

To support this flow, the CLI now will get the model's unique_id from meta.superset.model for such metrics:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
       (SUM(price_each)) * 1.2
      {% endraw %}
    meta:
      superset:
        model: model.postgres.vehicle_sales

the {% raw %} macro also allows users to use Superset-specific Jinja in their metrics:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
        SUM(
          {% for x in filter_values('x_values') %}
            {{ + x_values }}
          {% endfor %}
        )
      {% endraw %}
    meta:
      superset:
        model: model.postgres.vehicle_sales
  - name: new_derived
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
        SUM(
          {% for x in filter_values('x_values') %}
            {{ + 
      {% endraw %}
              {{ metric('revenue_verbose_name_from_dbt')}}
      {% raw %}
            }}
          {% endfor %}
        )
      {% endraw %}

These were currently failing as we rely on sqlglot to parse the derived metrics and replace a metric name with its sql syntax. This PR changes the logic so that the SQL parsing is skipped in case the derived metric doesn't really include other metrics. In case it's a combination of Superset-Jinja + dbt Jinja (like the second example), the CLI will catch the ParseError exception and will perform a replace with regex for the metric (definitely not as stable as SQL parsing, but hoping to increase compatibility with these types of metrics).

These changes are currently aiming older versions of dbt -- going to also perform some tests with MetricFlow/new semantic layer to validate if these flows are supported in newer versions, to assess how to also support them with the CLI.

[betodealmeida]

I'm confused. 🙂 When you say

the {% raw %} macro also allows users to use Superset-specific Jinja in their metrics:

Is this something people actually want to do?

Why write this:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
       (SUM(price_each)) * 1.2
      {% endraw %}

Why not write directly:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
     (SUM(price_each)) * 1.2

?

[Vitor-Avila]

@betodealmeida ohh maybe that was a bad example -- my apologies. Your example should also work (in the sense that dbt would compile properly), however the CLI wouldn't be able to sync it since the metric doesn't have any dependency.

The {% raw %} macro would be needed when using the Superset-specific Jinja macros:

metrics:
  - name: new_derived_again
    label: test
    calculation_method: derived
    expression: |
      {% raw %}
        SUM(
          {% for x in filter_values('x_values') %}
            {{ + x_values }}
          {% endfor %}
        )
      {% endraw %}

The {% raw %} macro wouldn't persist to the manifest.json though.

[betodealmeida]

LGTM!