Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function #54391

Conversation

cjcenizal
Copy link
Contributor

@cjcenizal cjcenizal commented Jan 9, 2020

Release notes

A regression was introduced into 7.5.0 which caused a particular configuration of Threshold Watches to fail and/or erroneously trigger if they were created or edited in 7.5. If you've created or edited a Threshold Watch with a "GROUPED OVER" condition set to "top" with Kibana 7.5.0, you'll need to upgrade to a version of Kibana that contains this fix and recreate these watches. The easiest way to do this will be to go to the edit screen of the Threshold Watch in the UI and simply click the "Save" button. This will recreate the watch with the proper configuration. No other changes to the watch will be necessary on your part.

image

Description

Fixes #53974

This fixes a regression introduced in 7.5.0 via #43232, in which these arguments were not being provided. This resulted in two bugs, both of which would occur when when GROUPED OVER is set to top, which sets a termField and sets hasTermsAgg to true.

Bug 1: Missing termOrder causes watch to fail

Under this condition, build_input.js#L50 and build_input.js#L88 will consume termOrder in the presence of a termField. If termOrder is undefined, then order will be assigned an empty object, due to the way JSON.stringify strips out properties with undefined values.

Here's what this bug looks like in the request flyout:

image

Here's what the request flyout looks like now that the bug is fixed:

image

Bug 2: Missing hasTermsAgg results in incorrect condition script

build_transform.js and build_condition.js both depend upon hasTermsAgg to specify the correct scripts. For example, in the screenshot below the condition script should be this and the transform script should be this.

image

Here's what it looks like post-fix:

image

Due diligence

I untangled the logic on the server-side and found that the execute API consumes the watchJson getter that exists on all Watch models. The server-side ThresholdWatch model consumes serializeThresholdWatch within its version of this getter. However, the execute API is only called by the UI when the user is viewing JSON watches. I.e. the way the API is currently consumed precludes this getter from ever being called. So I couldn't really test the effects my changes have upon this logic.

Unrelated changes

In 27fcefa, I removed some unused model methods and verified this had no impact on creating, editing, simulating, or deleting watches.

Next steps

I plan to perform a tightly-scoped TS conversion to ensure that serializeThresholdWatch always receives the arguments it expects. This would have prevented this regression from originally being introduced. I'm going to do this in a separate PR because of the high level of effort required.

@cjcenizal cjcenizal added bug Fixes for quality problems that affect the customer experience Feature:Watcher v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.6.0 v7.5.1 labels Jan 9, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

Copy link
Contributor

@sebelga sebelga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM! I haven't tested locally.

Copy link
Contributor

@jloleysens jloleysens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested locally and seems to work!

Well done for implementing this fix. Regarding this comment in the PR description:

I.e. the way the API is currently consumed precludes this getter from ever being called. So I
couldn't really test the effects my changes have upon this logic.

Is there some part of watcher that is not testable? Not sure I entirely follow.

@cjcenizal
Copy link
Contributor Author

@jloleysens Sorry about that! I'll try to clarify. There are three model classes on the server, representing different types of watches: ThresholdWatch, JsonWatch, and MonitoringWatch.

The handler for the /api/watcher/watch/execute endpoint is designed to accept any type of watch, and to consume the watchJson getter of the watch provided to it. If it were to be provided a ThresholdWatch, the watchJson getter in its implementation would end up calling serializeThresholdWatch, which is the function this PR is concerned with.

In our UI, the /api/watcher/watch/execute endpoint is consumed by the "Create/edit JSON watch" form, but not by the "Create/edit threshold watch" form (see screenshot below). This means that, though this endpoint is designed to accept threshold watches, our UI never actually sends threshold watches to it. Our UI only ever sends JSON watches to it. So there's really nothing for me to test.

I assume that if we were to add simulate functionality to the threshold watch form, we would test that the endpoint works as expected when we send threshold watch data to it.

image

@cjcenizal cjcenizal merged commit 2e3ce5c into elastic:master Jan 10, 2020
@cjcenizal cjcenizal deleted the bug/watcher-threshold-serialization-regression branch January 10, 2020 17:53
@cjcenizal cjcenizal added v7.5.2 and removed v7.5.1 labels Jan 10, 2020
cjcenizal added a commit to cjcenizal/kibana that referenced this pull request Jan 10, 2020
…function (elastic#54391)

* Fix Watcher regression in which a threshold watch's termOrder and hasTermsAgg properties weren't being passed to the serializeThresholdWatch function.
* Remove unused upstreamJson getter method from server models.
cjcenizal added a commit that referenced this pull request Jan 10, 2020
…function (#54391) (#54487)

* Fix Watcher regression in which a threshold watch's termOrder and hasTermsAgg properties weren't being passed to the serializeThresholdWatch function.
* Remove unused upstreamJson getter method from server models.
cjcenizal added a commit that referenced this pull request Jan 10, 2020
…function (#54391) (#54484)

* Fix Watcher regression in which a threshold watch's termOrder and hasTermsAgg properties weren't being passed to the serializeThresholdWatch function.
* Remove unused upstreamJson getter method from server models.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
thomasneirynck pushed a commit to thomasneirynck/kibana that referenced this pull request Jan 12, 2020
…function (elastic#54391)

* Fix Watcher regression in which a threshold watch's termOrder and hasTermsAgg properties weren't being passed to the serializeThresholdWatch function.
* Remove unused upstreamJson getter method from server models.
@jloleysens
Copy link
Contributor

@cjcenizal Thanks for the explanation! That makes sense!

gmmorris added a commit to gmmorris/kibana that referenced this pull request Jan 13, 2020
* master: (69 commits)
  [Graph] Fix various a11y issues (elastic#54097)
  Add ApplicationService app status management (elastic#50223)
  logs in one time (elastic#54447)
  Deprecate using `elasticsearch.ssl.certificate` without `elasticsearch.ssl.key` and vice versa (elastic#54392)
  [Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. (elastic#54457)
  Security - Role Mappings UI (elastic#53620)
  [SIEM] [Detection engine] Permission II (elastic#54292)
  Allow User to Cleanup Repository from UI  (elastic#53047)
  [Detection engine] Some UX for rule creation (elastic#54471)
  share specific instances of some ui packages (elastic#54079)
  [ML] APM modules configs for RUM Javascript and NodeJS (elastic#53792)
  [APM] Delay rendering invalid license notification (elastic#53924)
  [Graph] Improve error message on graph requests (elastic#54230)
  [ILM] Kibana should allow a min_age setting of 0ms in ILM policy phases (elastic#53719)
  Unit Tests for common/lib (elastic#53736)
  [Graph] Only show explorable fields (elastic#54101)
  remove linting rule exception for markdown (elastic#54232)
  [Monitoring] Fetch shard data more efficiently (elastic#54028)
  [Maps] Add hiddenLayers option to embeddable map input (elastic#54355)
  Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function (elastic#54391)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jan 13, 2020
* master: (69 commits)
  [Graph] Fix various a11y issues (elastic#54097)
  Add ApplicationService app status management (elastic#50223)
  logs in one time (elastic#54447)
  Deprecate using `elasticsearch.ssl.certificate` without `elasticsearch.ssl.key` and vice versa (elastic#54392)
  [Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. (elastic#54457)
  Security - Role Mappings UI (elastic#53620)
  [SIEM] [Detection engine] Permission II (elastic#54292)
  Allow User to Cleanup Repository from UI  (elastic#53047)
  [Detection engine] Some UX for rule creation (elastic#54471)
  share specific instances of some ui packages (elastic#54079)
  [ML] APM modules configs for RUM Javascript and NodeJS (elastic#53792)
  [APM] Delay rendering invalid license notification (elastic#53924)
  [Graph] Improve error message on graph requests (elastic#54230)
  [ILM] Kibana should allow a min_age setting of 0ms in ILM policy phases (elastic#53719)
  Unit Tests for common/lib (elastic#53736)
  [Graph] Only show explorable fields (elastic#54101)
  remove linting rule exception for markdown (elastic#54232)
  [Monitoring] Fetch shard data more efficiently (elastic#54028)
  [Maps] Add hiddenLayers option to embeddable map input (elastic#54355)
  Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function (elastic#54391)
  ...
@kibanamachine
Copy link
Contributor

💔 Build Failed

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Watcher regression release_note:fix Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.5.2 v7.6.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Watcher Threshold Alert creates empty order in terms agg, throws errors
5 participants