Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Make Jinja template applied in timestamp columns #17237

Merged
merged 3 commits into from
Oct 28, 2021
Merged

feat: Make Jinja template applied in timestamp columns #17237

merged 3 commits into from
Oct 28, 2021

Conversation

xingyc15
Copy link
Contributor

SUMMARY

Fix the timestamp expression in sqla model.py, adding jinja template processor to it.

TESTING INSTRUCTIONS

diff --git a/docker/pythonpath_dev/superset_config.py b/docker/pythonpath_dev/superset_config.py
index 6c58bec79..83ebedca1 100644
--- a/docker/pythonpath_dev/superset_config.py
+++ b/docker/pythonpath_dev/superset_config.py
@@ -99,6 +99,15 @@ WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL

 SQLLAB_CTAS_NO_LIMIT = True

+# Local env test
+FEATURE_FLAGS = {
+    "ENABLE_TEMPLATE_PROCESSING": True
+}
+
+JINJA_CONTEXT_ADDONS = {
+    'jinja_tmpl': 'date'
+}
+

Apply the above changes, run docker build -t apache/superset:latest-dev . and then use docker-compose to start the superset app. Go to localhost:8088 in browser and login to superset.

Go to dataset, find cleaned_sales_data and go to calculated columns, add a new item like this:
Screen Shot 2021-10-26 at 12 23 30 PM

Then go to the chart Proportion of Revenue by Product Line, choose the jinja_date as time column and run. The chart should works fine without error. View the query and the order_{{ jinja_tmpl }} is already replaced by "order_date".

ADDITIONAL INFORMATION

  • Has associated issue: JINJA Template does not work with calculated time column #16984
  • Required feature flags: "ENABLE_TEMPLATE_PROCESSING": True
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, really nice feature! I left a small proposal to make the feature support all query properties available.

Comment on lines 325 to 327
col = literal_column(self.expression, type_=type_)
tp = self.table.get_template_processor()
expression = tp.process_template(self.expression)
col = literal_column(expression, type_=type_)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure the template processor has access to all query parameters which are defined early in the get_sqla_query method (see

template_kwargs = {
"columns": columns,
"from_dttm": from_dttm.isoformat() if from_dttm else None,
"groupby": groupby,
"metrics": metrics,
"row_limit": row_limit,
"row_offset": row_offset,
"time_column": granularity,
"time_grain": time_grain,
"to_dttm": to_dttm.isoformat() if to_dttm else None,
"table_columns": [col.column_name for col in self.columns],
"filter": filter,
}
), it would be nice to pass the template processor (declared here:
template_processor = self.get_template_processor(**template_kwargs)
) as an optional parameter to get_timesstamp_expression that can then be used if available.

@codecov
Copy link

codecov bot commented Oct 27, 2021

Codecov Report

Merging #17237 (f0aec1a) into master (dd71035) will increase coverage by 0.00%.
The diff coverage is 75.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #17237   +/-   ##
=======================================
  Coverage   76.95%   76.96%           
=======================================
  Files        1038     1038           
  Lines       55612    55614    +2     
  Branches     7590     7590           
=======================================
+ Hits        42798    42803    +5     
+ Misses      12564    12561    -3     
  Partials      250      250           
Flag Coverage Δ
hive 81.52% <75.00%> (+0.01%) ⬆️
mysql 81.95% <75.00%> (+0.01%) ⬆️
postgres 81.96% <75.00%> (+0.01%) ⬆️
presto 81.82% <75.00%> (+0.01%) ⬆️
python 82.45% <75.00%> (+0.01%) ⬆️
sqlite 81.63% <75.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/connectors/sqla/models.py 87.39% <75.00%> (-0.09%) ⬇️
superset/annotation_layers/annotations/schemas.py 100.00% <0.00%> (+13.33%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd71035...f0aec1a. Read the comment docs.

@villebro villebro changed the title Fix: Make Jinja template applied in timestamp columns feat: Make Jinja template applied in timestamp columns Oct 27, 2021
Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the change that was spot-on with what I was hoping for!

@villebro villebro merged commit 5aaa333 into apache:master Oct 28, 2021
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/S 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants