Update Python models to include a way to change the Python version

[runleonarun]

Contributions

• I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/docs/build/python-models

What part(s) of the page would you like to see updated?

If you want use a later version of Python in dbt than 3.8, you'd need to configure the version.

See this community thread in #dbt-core-python-model that explains how to set the py version in the dbt.config) in the model

def model(dbt, session):

    # setting configuration
    dbt.config(
      materialized="table",
        python_version="3.9"
)

Additional information

No response

[runleonarun]

@dbeatty10 or @joellabes Is there a specific way you'd recommend we document this or does the example above look ok? I was thinking about adding this information to the Specific data platforms section and the examples.

1. Are there any differences between changing the version for Databricks vs BigQuery vs Snowflake?
2. Is there a best practice for setting a different version?
3. Any gotchas / Do we recommend against doing this?

thank you!

[dbeatty10]

@runleonarun I don't know if I can fully answer any of those questions 😅

After doing some very quick code searches, I didn't see the python_version config for any other adapters other than dbt-snowflake:

• https://github.com/search?q=repo%3Adbt-labs%2Fdbt-snowflake%20python_version&type=code
• https://github.com/search?q=repo%3Adbt-labs%2Fdbt-bigquery%20python_version&type=code
• https://github.com/search?q=repo%3Adbt-labs%2Fdbt-spark+python_version&type=code
• https://github.com/search?q=repo%3Adatabricks%2Fdbt-databricks+python_version&type=code

So it looks like that this example would only have any effect in dbt-snowflake:

def model(dbt, session):

    # setting configuration
    dbt.config(
      materialized="table",
        python_version="3.9"
)

For dbt-bigquery, it sounds like setting the PYSPARK_PYTHON environment variable is how to choose the default Python interpreter for your PySpark jobs, but that is not something I've tried out or verified personally.

[runleonarun]

Thanks so much for looking into this @dbeatty10! Sounds like we'll have to do some more research.

[xemiem]

Thanks for bringing this up @runleonarun! Looking forward to the docs being updated on this :)