0.54.1

Release 0.54.1, includes a mix of updates new additions, and bug fixes. The most notable changes are the new production guide,
allowing multi-step VMs for the Skypilot orchestrator which allows you to configure a step to run on a specific VM or run the entire pipeline on a single VM, and some improvements to the Model Control Plane.

What's Changed

• Bump aquasecurity/trivy-action from 0.16.0 to 0.16.1 by @dependabot in #2244
• Bump crate-ci/typos from 1.16.26 to 1.17.0 by @dependabot in #2245
• Add YAML formatting standardisation to formatting & linting scripts by @strickvl in #2224
• Remove text annotation by @strickvl in #2246
• Add MariaDB migration testing by @strickvl in #2170
• Delete artifact links from model version via Client, ModelVersion and API by @avishniakov in #2191
• Default/Non-Default step params produce conflict with yaml ones as defaults are set in code by @avishniakov in #2247
• Prune of unused artifacts links via client by @avishniakov in #2192
• Rename nlp example by @safoinme in #2221
• Support refreshing service connector credentials in the Vertex step operator to support long-running jobs by @stefannica in #2198
• Refactor secrets stores to store all secret metadata in the DB by @stefannica in #2193
• Add latest_version_id to the ModelResponse by @avishniakov in #2266
• Remove link_artifact from docs for MCP by @strickvl in #2272
• Improve action by adding advice to KeyError when configured steps are not present in pipeline by @christianversloot in #2265
• Allow multi step configuration for skypilot by @safoinme in #2166
• Reworking the examples by @bcdurak in #2259
• A docs update for incorrect import in docs/book/user-guide/starter-guide/track-ml-models.md by @yo-harsh in #2279
• Allow sklearn versions > 1.3 by @Vishal-Padia in #2271
• Free sklearn dependency to allow all versions by @strickvl in #2281
• Misc CI bugfixes by @strickvl in #2260
• Fix yamlfix script to use --no-yamlfix flag by @strickvl in #2280
• Fix dependabot settings autoformatting by yamlfix by @strickvl in #2282
• Add advice for next step to error on AuthorizationException by @christianversloot in #2264
• Allow skypilot to configure step or run full pipeline in one VM by @safoinme in #2276
• A docs update with production guide + restructured advanced guide by @htahir1 in #2232
• Prepare release 0.54.1 by @safoinme in #2284
• Add Escan to teams.yml by @strickvl in #2283
• Fix typos from docs updates by @strickvl in #2287
• Fix to file path for notebook quickstart by @jacksandom in #2288
• Fix zenml-tutorial path assignment in cli.go() by @safoinme in #2291

New Contributors

• @yo-harsh made their first contribution in #2279
• @Vishal-Padia made their first contribution in #2271
• @jacksandom made their first contribution in #2288

Full Changelog: 0.54.0...0.54.1

0.54.0

This release brings a range of new features, bug fixes and documentation
updates. The Model Control Plane has received a number of small bugfixes and
improvements, notably the ability to change model and model version names.

We've also added a whole new starter guide that walks you through
how to get started with ZenML, from creating your first pipeline to fetching
objects once your pipelines have run and much more. Be sure to check it out if
you're new to ZenML!

Speaking of documentation improvements, the Model Control Plane now has its own
dedicated documentation section introducing the concepts and features of the
Model Control Plane.

As always, this release comes with number of bug fixes, docs additions and
smaller improvements to our internal processes.

Breaking Change

This PR introduces breaking changes in the areas of the REST API concerning secrets and tags. As a consequence, the ZenML Client running the previous ZenML version is no longer compatible with a ZenML Server running the new version and vice-versa. To address this, simply ensure that all your ZenML clients use the same version as the server(s) they connect to.

🥳 Community Contributions 🥳

We'd like to give a special thanks to @christianversloot for two PRs he
contributed to this release. One of them fixes a bug that prevented ZenML from
running on Windows and the other one adds a new materializer for the Polars library.

Also many thanks to @sean-hickey-wf for his contribution of an improvement to
the Slack Alerter stack component
which allows you to define custom blocks for the Slack message.

What's Changed

• Completing the hydration story with the remaining models by @bcdurak in #2151
• Remove secrets manager flavors from DB by @stefannica in #2182
• Prepare 0.53.1 release by @stefannica in #2183
• Update package name for nightly build by @strickvl in #2172
• Remove space saver action + upgrade other actions by @strickvl in #2174
• mutable names in Model and MV by @avishniakov in #2185
• Fix image building for nightly container builds by @strickvl in #2189
• Test that artifacts not get linked to model version not from context by @avishniakov in #2188
• Warn if Model(Version) config fluctuates from DB state by @avishniakov in #2144
• Add blocks field to SlackAlerterParameters for custom slack blocks by @sean-hickey-wf in #2153
• Model control plane technical documentation by @strickvl in #2111
• Alembic branching issue fix by @avishniakov in #2197
• Bump github/codeql-action from 2 to 3 by @dependabot in #2201
• Bump google-github-actions/get-gke-credentials from 0 to 2 by @dependabot in #2202
• Bump google-github-actions/auth from 1 to 2 by @dependabot in #2203
• Bump aws-actions/amazon-ecr-login from 1 to 2 by @dependabot in #2200
• Bump crate-ci/typos from 1.16.25 to 1.16.26 by @dependabot in #2207
• Fix unreliable test behaviour when using hypothesis by @strickvl in #2208
• Added more pod spec properties for k8s orchestrator by @htahir1 in #2097
• Fix API docs environment setup by @strickvl in #2190
• Use placeholder runs to show pipeline runs in the dashboard without delay by @schustmi in #2048
• Update README and CONTRIBUTING.md docs with links to good first issues for contribution by @strickvl in #2220
• Bump supported mlstacks version to 0.8.0 by @strickvl in #2196
• Misc cleanup by @schustmi in #2126
• Refactor pipeline run updates by @schustmi in #2117
• Rename log_model_version_metadata to log_model_metadata by @htahir1 in #2215
• Update starter and create new production guide by @htahir1 in #2143
• Fix typo by @strickvl in #2223
• Consolidate Custom Filter Logic by @fa9r in #2116
• Force forward slashes when saving artifacts by @christianversloot in #2195
• Temporarily disable two MLflow tests for MacOS with Python 3.9 and 3.10 by @strickvl in #2186
• Disable template updates for forked repositories by @strickvl in #2222
• Remove Label Studio text annotation example by @strickvl in #2225
• Add scarf checker script and CI workflow by @strickvl in #2227
• Add mlstacks installation instructions to docs by @strickvl in #2228
• Adding the hydrate flag to the client methods by @bcdurak in #2120
• Fixing the remaining docs pages for run_metadata by @bcdurak in #2230
• Fix CI check to disallow template testing on forked repositories by @strickvl in #2231
• Fix fork check syntax by @strickvl in #2237
• Add missing annotations section to zenml service account by @wjayesh in #2234
• Allow filtering artifacts with/without custom names by @schustmi in #2226
• Adjust migration settings based on database engine by @strickvl in #2236
• Added one more chapter to starter guide by @htahir1 in #2238
• Add Polars materializer by @christianversloot in #2229

New Contributors

• @sean-hickey-wf made their first contribution in #2153
• @dependabot 🤖 made their first contribution in #2201

Full Changelog: 0.53.1...0.54.0

0.47.0

This release fixes a bug that was introduced in 0.46.1 where the default user
was made inaccessible and was inadvertently duplicated. This release rescues
the original user and renames the duplicate.

UPDATE: a breaking update to one of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1'

What's Changed

• Create tags table by @avishniakov in #2036
• Bring dashboard back to the release by @avishniakov in #2046
• Fix duplicate default user by @stefannica in #2045

Full Changelog: 0.46.1...0.47.0

0.44.4

This patch release backports some important fixes that have been introduced in more recent versions
of ZenML to the 0.44.x release line.

UPDATE: a breaking update to one of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1'

What's Changed

• [backport/0.44] Backport important patches by @stefannica in #2042

Full Changelog: 0.44.3...0.44.4

0.46.1 [YANKED]

THIS RELEASE HAS BEEN YANKED

• Due to an unforeseen alembic migration issue and missing dashboard, this release has been yanked
• 0.47.0 is out to fix the bugs that have been uncovered

0.46.1

The 0.46.1 release introduces support for Service Accounts and API Keys that
can be used to authenticate with the ZenML server from environments that do not
support the web login flow, such as CI/CD environments, for example.

Also included in this release are some documentation updates and bug fixes,
notably moving the database migration logic deployed with the Helm chart out of
the init containers and into a Kubernetes Job, which makes it possible to scale
out the ZenML server deployments without the risk of running into database
migration conflicts.

What's Changed

• Small improvements to Hub docs page by @strickvl in #2015
• Pin OpenAI integration to <1.0.0 by @strickvl in #2027
• Make error message nicer for when two artifacts that share a prefix are found by @strickvl in #2023
• Move db-migration to job instead of init-container to allow replicas by @safoinme in #2021
• Fix stuck/broken CI by @strickvl in #2032
• Increase step.source_code Cut-Off Limit by @fa9r in #2025
• Improve artifact linkage logging in MCP by @avishniakov in #2016
• Upgrade feast so apidocs don't fail no mo by @AlexejPenner in #2028
• Remove NumPy Visualizations for 2D Arrays by @fa9r in #2033
• Fix user activation bug by @stefannica in #2037
• Remove create_new_model_version arg of ModelConfig by @avishniakov in #2030
• Extend the wait period in between PyPi package publication and Docker image building for releases by @strickvl in #2029
• Make zenml up prefill username when launching dashboard by @strickvl in #2024
• Add warning when artifact store cannot be loaded by @strickvl in #2011
• Add extra config to Kaniko docs by @safoinme in #2019
• ZenML API Keys and Service Accounts by @stefannica in #1840

Full Changelog: 0.46.0...0.46.1

0.46.0

UPDATE: a breaking update to one of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1'

This release brings some upgrades, documentation updates and bug fixes. Notably,
our langchain integration now supports more modern versions and has been
upgraded to a new version at the lower edge of supported packages on account of
a security vulnerability.

Other fixes related to the Model Control Plane which was updated to support the
deletion of model versions via the CLI, for example.

Breaking Change

We removed the llama_index integration in this release. This related to
unsolvable dependency clashes that relate to sqlmodel and our database. We
expect these clashes to be resolved in the future and then we will add our
integration back in. If you were using the llama_index materializer that was
part of the integration, you will have to use a custom materializer in the
meanwhile. We apologize for the inconvenience.

What's Changed

• MCP-driven E2E template by @avishniakov in #2004
• Model scoped endpoints by @avishniakov in #2003
• Delete model version in cli by @avishniakov in #2006
• Add latest version to model list response by @avishniakov in #2007
• Fix gcs bucket docs error message by @safoinme in #2018
• Fix Skypilot docs configuration by @safoinme in #2017
• Bump langchain, disable llama_index, and fix Vector Store materializer by @strickvl in #2013
• Fix Build Options of GCPImageBuilder by @fa9r in #1992
• Fix the stack component describe CLI output by @stefannica in #2001

Full Changelog: 0.45.6...0.46.0

0.45.6

UPDATE: a breaking update to one of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1'

This release brings an array of enhancements and refinements. Notable improvements include
allowing for disconnecting service connectors from stack components, adding connector support to the
sagemaker step operator, turning synchronous mode on by default for all orchestrators, and enabling
server-side component config validation.

What's Changed

• Updating README.md and update images by @znegrin in #1986
• Always set the active workspace to be the default workspace server side by @stefannica in #1989
• Update outdated CLI docs by @strickvl in #1990
• Turn synchronous mode on by default for all orchestrators by @stefannica in #1991
• Use docker credentials in the skypilot orchestrator by @stefannica in #1983
• Add missing space to @step warning message by @strickvl in #1994
• Fix sagemaker orchestrator and step operator env vars and other minor bugs by @stefannica in #1993
• fix: BasePyTorchMaterliazer -> Materializer by @cameronraysmith in #1969
• allow calling old base pytorch materilizzer by @safoinme in #1997
• Add connector support to sagemaker step operator. by @stefannica in #1996
• Server-Side Component Config Validation by @fa9r in #1988
• Allow disconnecting service-connector from stack component by @safoinme in #1864

New Contributors

• @znegrin made their first contribution in #1986

Full Changelog: 0.45.5...0.45.6

0.45.5

UPDATE: a breaking update to one of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1'

This minor release contains bugfixes and documentation improvements. Notably,
our sqlmodel dependency has been pinned to 0.0.8 which fixes installation
errors following the release of 0.0.9.

What's Changed

• Add a 'how do I...' section into docs by @strickvl in #1953
• Bump mypy, ruff and black by @strickvl in #1963
• Fix double slashes in weblogin by @schustmi in #1972
• SQLModel docs backport fixes by @strickvl in #1975
• Updated quickstart command in cloud quickstart by @AlexejPenner in #1977
• Make sure vertex job id is only lower case letter, number or dash by @AlexejPenner in #1978
• Fix DB initialization when using external authentication by @schustmi in #1965
• Pin SQLModel dependency to 0.0.8 by @strickvl in #1973
• Source code repository type not included in resolve source by @safoinme in #1982
• Prepare release 0.45.5 by @strickvl in #1981

Full Changelog: 0.45.4...0.45.5

0.45.4

UPDATE: a breaking update to two of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1' 'sqlmodel==0.0.8'

This minor update fixes a database migration bug that you could potentially encounter while upgrading your ZenML version and relates to use of the ExternalArtifact object.
If you are upgrading from <0.45.x version, this is the recommended release.

UPDATE (Oct 24, 2023): A new version of sqlmodel (0.0.9) might cause issues with your ZenML installation. If you encounter this error, please do a pip install 'sqlmodel==0.0.8' to fix the problem. This will be permanently fixed in higher versions of ZenML.

PROBLEMS?: If you upgraded to ZenML v0.45.2 or v0.45.3 and are experiencing issues with your database, please consider upgrading to v0.45.4 instead.

What's Changed

• Increase reuse of ModelConfig by @avishniakov in #1954
• resolve alembic branches by @avishniakov in #1964
• Fix corrupted migration for old dbs by @avishniakov in #1966

Full Changelog: 0.45.3...0.45.4

0.45.3

UPDATE: a breaking update to two of our dependencies means that you should run the following update after installing this version of zenml:

pip install 'bcrypt==4.0.1' 'sqlmodel==0.0.8'

This minor update fixes a database migration bug that you could potentially encounter while upgrading your ZenML version and relates to use of the ExternalArtifact object.

PROBLEMS?: If you upgraded to ZenML v0.45.2 and are experiencing issues with your database, please either reach out to us on Slack directly or feel free to use this migration script that will manually fix the issue. (Please do backup your database before using the migration script so as to prevent any data loss!)

This release also includes a bugfix from @cameronraysmith relating to the
resolution of our Helm chart OCI location. Thank you!

What's Changed

• fix: match chart name in docs to publish workflow by @cameronraysmith in #1942
• Evaluate YAML based config early + OSS-2511 by @avishniakov in #1876
• Fixing nullable parameter to avoid extra migrations by @bcdurak in #1955
• Pin Helm version to avoid 400 Bad Request error by @wjayesh in #1958
• external_input_artifact backward compatibility with alembic by @avishniakov in #1957
• Fix nlp template link by @htahir1 in #1960
• handle NULLs in migration by @avishniakov in #1961
• Prepare release 0.45.3 by @strickvl in #1959

New Contributors

• @cameronraysmith made their first contribution in #1942

Full Changelog: 0.45.2...0.45.3