-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow jobs to be manually unpaused in development mode #885
Allow jobs to be manually unpaused in development mode #885
Conversation
Partly mitigates databricks#859
03048e4
to
2984ef8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The two acceptable values for this field are PAUSED
and UNPAUSED
.
What is the behavior is the field is not specified? That is the only case that will be overridden in development mode after this case. Otherwise, if they specify UNPAUSED
it will be left alone, and if they specify PAUSED
, it is already paused, making the override a nop.
If this only overrides the case where the pause status is not specified, could you add a comment stating that explicitly.
Added a comment. This sets all jobs to paused, except if they were explicitly set to unpaused. Unpaused is also the behavior when nothing was specified. |
This release includes the new `databricks labs` command to install, manage, and run Databricks Labs projects. CLI: * Add `--debug` as shortcut for `--log-level debug` ([#964](#964)). * Improved usability of `databricks auth login ... --configure-cluster` ([#956](#956)). * Make `databricks configure` save only explicit fields ([#973](#973)). * Add `databricks labs` command group ([#914](#914)). * Tolerate missing .databrickscfg file during `databricks auth login` ([#1003](#1003)). * Add `--configure-cluster` flag to configure command ([#1005](#1005)). * Fix bug where the account or workspace client could be `nil` ([#1020](#1020)). Bundles: * Do not allow empty descriptions for bundle template inputs ([#967](#967)). * Added support for top-level permissions ([#928](#928)). * Allow jobs to be manually unpaused in development mode ([#885](#885)). * Fix template initialization from current working directory ([#976](#976)). * Add `--tag` and `--branch` options to bundle init command ([#975](#975)). * Work around DLT issue with `$PYTHONPATH` not being set correctly ([#999](#999)). * Enable `spark_jar_task` with local JAR libraries ([#993](#993)). * Pass `USERPROFILE` environment variable to Terraform ([#1001](#1001)). * Improve error message when path is not a bundle template ([#985](#985)). * Correctly overwrite local state if remote state is newer ([#1008](#1008)). * Add mlops-stacks to the default `databricks bundle init` prompt ([#988](#988)). * Do not add wheel content hash in uploaded Python wheel path ([#1015](#1015)). * Do not replace pipeline libraries if there are no matches for pattern ([#1021](#1021)). Internal: * Update CLI version in the VS Code extension during release ([#1014](#1014)). API Changes: * Changed `databricks functions create` command . New request type is . * Changed `databricks metastores create` command with new required argument order. * Removed `databricks metastores enable-optimization` command. * Removed `databricks account o-auth-enrollment` command group. * Removed `databricks apps delete` command. * Removed `databricks apps get` command. * Added `databricks apps delete-app` command. * Added `databricks apps get-app` command. * Added `databricks apps get-app-deployment-status` command. * Added `databricks apps get-apps` command. * Added `databricks apps get-events` command. * Added `databricks account network-connectivity` command group. OpenAPI commit 22f09783eb8a84d52026f856be3b2068f9498db3 (2023-11-23) Dependency updates: * Bump golang.org/x/term from 0.13.0 to 0.14.0 ([#981](#981)). * Bump github.com/hashicorp/terraform-json from 0.17.1 to 0.18.0 ([#979](#979)). * Bump golang.org/x/oauth2 from 0.13.0 to 0.14.0 ([#982](#982)). * Bump github.com/databricks/databricks-sdk-go from 0.24.0 to 0.25.0 ([#980](#980)). * Bump github.com/databricks/databricks-sdk-go from 0.25.0 to 0.26.0 ([#1019](#1019)).
This release includes the new `databricks labs` command to install, manage, and run Databricks Labs projects. CLI: * Add `--debug` as shortcut for `--log-level debug` ([#964](#964)). * Improved usability of `databricks auth login ... --configure-cluster` ([#956](#956)). * Make `databricks configure` save only explicit fields ([#973](#973)). * Add `databricks labs` command group ([#914](#914)). * Tolerate missing .databrickscfg file during `databricks auth login` ([#1003](#1003)). * Add `--configure-cluster` flag to configure command ([#1005](#1005)). * Fix bug where the account or workspace client could be `nil` ([#1020](#1020)). Bundles: * Do not allow empty descriptions for bundle template inputs ([#967](#967)). * Added support for top-level permissions ([#928](#928)). * Allow jobs to be manually unpaused in development mode ([#885](#885)). * Fix template initialization from current working directory ([#976](#976)). * Add `--tag` and `--branch` options to bundle init command ([#975](#975)). * Work around DLT issue with `$PYTHONPATH` not being set correctly ([#999](#999)). * Enable `spark_jar_task` with local JAR libraries ([#993](#993)). * Pass `USERPROFILE` environment variable to Terraform ([#1001](#1001)). * Improve error message when path is not a bundle template ([#985](#985)). * Correctly overwrite local state if remote state is newer ([#1008](#1008)). * Add mlops-stacks to the default `databricks bundle init` prompt ([#988](#988)). * Do not add wheel content hash in uploaded Python wheel path ([#1015](#1015)). * Do not replace pipeline libraries if there are no matches for pattern ([#1021](#1021)). Internal: * Update CLI version in the VS Code extension during release ([#1014](#1014)). API Changes: * Changed `databricks functions create` command. * Changed `databricks metastores create` command with new required argument order. * Removed `databricks metastores enable-optimization` command. * Removed `databricks account o-auth-enrollment` command group. * Removed `databricks apps delete` command. * Removed `databricks apps get` command. * Added `databricks apps delete-app` command. * Added `databricks apps get-app` command. * Added `databricks apps get-app-deployment-status` command. * Added `databricks apps get-apps` command. * Added `databricks apps get-events` command. * Added `databricks account network-connectivity` command group. OpenAPI commit 22f09783eb8a84d52026f856be3b2068f9498db3 (2023-11-23) Dependency updates: * Bump golang.org/x/term from 0.13.0 to 0.14.0 ([#981](#981)). * Bump github.com/hashicorp/terraform-json from 0.17.1 to 0.18.0 ([#979](#979)). * Bump golang.org/x/oauth2 from 0.13.0 to 0.14.0 ([#982](#982)). * Bump github.com/databricks/databricks-sdk-go from 0.24.0 to 0.25.0 ([#980](#980)). * Bump github.com/databricks/databricks-sdk-go from 0.25.0 to 0.26.0 ([#1019](#1019)).
Partly mitigates #859. It's still not clear to me if there is an actual use case or if users are trying to use "development" mode jobs for production, but making this overridable is reasonable.
Beyond this fix I think we could do something in the Jobs schedule UI, but it would help to better understand the use case (or actual reason of confusion). I expect we should hint customers to move away from dev mode rather than unpause.