- It's now possible to set export commands to generate YAML files with Unix EOL (#316).
- Calls to the Preset Manager API are now paginated (#320).
- It's now possible to trigger an import that creates missing dependencies but don't overwrite them (#321).
- It's now possible to link dbt metrics with Preset metrics that don't have the same name (#311).
- The dbt sync continues in case syncing columns for a dataset fails (#313).
- dbt Cloud versionless jobs are now supported (#314).
- It's now possible to bypass MetricFlow when syncing metrics from dbt Core (#305).
- Migrated the GraphQL query to fetch metrics from dbt Cloud to support jobs with an ID exceeding 32 bits (#306).
- It's now possible to sync a dbt Cloud project that doesn't have a semantic layer (#301).
- The dbt Core sync now supports syncing legacy metrics from dialects that are not supported by MetricFlow (#302).
- The dbt Cloud sync API endpoints were updated to support custom base domains (#303).
- The dbt sync now supports creating physical datasets in other catalogs (requires catalog Support in Superset) (#295 and #297).
- The dbt ModelSchema now supports models with
columns
set toNone
(#298). - It's now deprecated to trigger a dbt Core sync passing a
dbt_project.yml
file (support for passing the manifest.json file is no longer deprecated) (#299).
- Changed the dbt sync logic so that metrics that are not associated with the models being synced are ignored (#289).
- The dbt sync now indicates if the snowflake SQLAlchemy package is missing in the environment (#290).
- The dbt sync now removes Redshift-specific metatada from columns when refreshing a dataset (#291).
- Datasets used in dashboard filters are now included when importing assets individually with the
--split
flag (#292).
- The
profiles.yml
content is now rendered so that Jinja variables are handled properly (#280). - Added an upper bound limit to the
sqlglot
version that gets installed to avoid compatibility issues (#283). - The
sync native
command now tries to retrieve the DB connectionuuid
through the API first, to avoid exporting assets if not necessary (#284). - Added support for syncing derived metrics that rely on other derived metrics containing Superset-specific Jinja syntax (#285).
- Fixed an issue when syncing columns for datasets powered by BigQuery (#278).
- Added support for syncing derived metrics that don't rely on other metrics, and also metrics including Superset-Jinja specific syntax (#277).
- Improved metric parsing with sqlglot (#273 and #274).
- Fixed the dataset creation flow for the dbt sync (#275).
- Support for including the account and project IDs with the dbt Cloud command (#264).
- Support MetricFlow/new dbt Semantic Layer for the dbt Core sync (#265).
- New
--raise-failures
flag added to the dbt sync commands to end the execution with an error in case any model failed to sync (#266). - Syncing from dbt with the
--preserve-metadata
/--merge-metadata
flags now sync the dataset columns (#268). - Derived metrics for older dbt versions are now syncing properly (#270).
- Support for custom access URLs when connecting to dbt Cloud APIs (#262).
- Initial support for syncing metrics from dbt/MetricFlow (#256).
- The Jinja rendering/escaping logic for content migration was improved (#237).
- It's now possible to specify dbt models to be synced using the file name/path (#242).
- The CLI now has a re-try mechanism to address Session-related errors (#235).
- It's now possible to trigger a dbt sync and merge dbt metadata with Preset metadata (#238).
- The dbt sync now uses Superset updated endpoints to properly create a virtual dataset (#232).
- It's now possible to authenticate to Superset instances that require a CSRF token (#233).
- Further adjustments to dbt marshmallow schemas to avoid integration errors (#229).
- Further adjustments to dbt marshmallow schemas to avoid integration errors (#228).
- Export RLS rules is now compatible with Preset Cloud and older Superset installations (#227)
- Adjustments to dbt marshmallow schemas to avoid integration errors (#225).
certification
and additionalextra
information is now synced from dbt models (#213 and #215).- Improved the
exposures
sync (#221). - The
--preserve-columns
flag can now be used to preservegroupby
andfilterable
values for existing columns during a dbt sync (#221). - The search for roles during the
sync roles
command now usesEquals
comparison, instead ofStarts with
(#222).
- Fix for apache/superset#24067 (#211).
- Column descriptions and labels are now synced from dbt models (#72, #111, #195 and #197).
- CLI can now provision users directly to the team (doesn't require accepting the invitation) with the
import-users
command. Theexport-users
command can now also be used with Preset Workspaces. (#74, #100 and #148). - It's possible to export roles information using the
export-roles
command (#75 and #161). - Exported roles information can be imported via the
import-roles
command (#76, #167 and #179). - Improved session object logic (#77).
- Improved export/import logic for owernship and role information (#79).
- CLI can now add users to imported roles (#81).
- A JWT token can now be passed for authentication (#82).
- Added debug logging to API requests (#83 <https://github.com/preset-io/backend-sdk/pull/83>_).
- CLI can now export specific asset types, using the
--asset-type
flag (#84). - CLI can now export specific assets only, using the
--$asset_type-ids
(for example--dashboard-ids
) flag (#85 and #88). - CLI can now authenticate to Superset (On Premises) without CSRF token (#87).
- Workspace/Team prompt no longer happens in case
--help
was pased (#89). - Team Roles, Workspace Roles and DARs can now be synced to a Preset team based on a YAML file (#90).
- Added
--version
command to display the installed version (#91). - Fixed parent/child node selection in dbt Core for proper graph selection (#92).
- Improved logging for the dbt Client (#94).
- CLI now can create datasets for different databases (for DB Engines that supports multiple databases like Snowflake, BigQuery, etc) (#95).
- BQ connection can now successfully be created/updated from the
profiles.yml
information (#96). - Redshift connectons now get created with the
redshift+psycopg2
driver (#97). - YAML files outside of asset folders aren't imported in the native sync (#99).
- Improved BQ DB detection (#102).
- Reduced the maximum amount of files included in an export file (#105).
- Workspaces can now be defined as environment variables (#106).
- CLI can now create Snowflake connections authenticated via private key pair (#108).
- Improved the
--exclude
filter for the dbt sync (#109). - Improved database connection logic (#111).
- CLI can now create Snowflake connections authenticated with DUO MFA (#112).
- dbt target definition now defaults to the
profile.yml
if not specified (#114). - The dbt sync can now be triggered using the
dbt_project.yml
file rather than themanifest.json
(#115). - CLI now supports None as column type (#116).
- Database connection is now tested before triggering the import (#118).
- Added support for companion YAML templates (#120).
- YAML rendering logic is now improved (#121 and #205).
- DB connection password is no longer logged in case the connection fails (#122).
- Import assets is now performed through the
assets
endpoint (#124). - Large imports can be performed with the
--split
flag to prevent timeouts (#124). It also creates acheckpoint
in case it fails so the retry would ignore already imported assets (#137 and #139). - Preset Manager requests updated to use
api.app.preset.io
(#127). - CLI now prompts user for job information if not specified when triggering a sync from dbt Cloud (#128).
- dbt exposures now includes assets that were created by manual datasets, based on the schema and table name (#132).
- Added support for Python 3.11 (#133).
- CLI now refreshes JWT token if needed (#134).
- Import failures due to connection errors are automatically retried (#135).
- Improved Get Resources logic (#136).
- CLI no longer prompts user to enter the DB password in case the connection already exists (#140).
- It's now possible to trigger a sync only for exposures back to dbt, using the
--exposures-only
flag (#142). - CLI can be used to list SCIM groups and membership with the
list-group-membership
command (#143). - The dbt profile name is now used to look for an existing DB connection in the Workspace, instead of the project name (#151).
- Added support for dbt derived metrics (#154, #160, #196, #198 and #199).
- Fixed column configuration issues after a dbt sync (#156 and #165).
- Added support for dbt 1.3 (#159).
- Improved the
MetricSchema
loading (#159). - Added support for Secondary Contributor Workspace Role (#186).
- Use model table alias for dataset creation (#192).
- The dbt sync now only updates the DB connection in case
--import-db
is passed. It's also possible to trigger a sync without this flag (#193 and #200). - Added support for specifying a certification payload for dbt syncs (#203).
- dbt models can now be filtered using
config
options (#204). - It's now possible to disable Jinja syntax escaping during export, and Jinja syntax rendering during import (#205).
- File path is now passed to template as
filepath
in thesync native
command. - CLI can now invite users to Preset from a YAML file created by
export-users
. - Fix database update in the dbt sync.
- Initial release.