Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make action retries configurable #147876

Merged
merged 8 commits into from
Dec 23, 2022
Merged

Make action retries configurable #147876

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3654e28
Make action retries configurable
ersin-erdal Dec 20, 2022
1e736bd
Update Docs
ersin-erdal Dec 20, 2022
971c936
Change max configurable maxAttempts
ersin-erdal Dec 20, 2022
cb1970a
Fix failing test
ersin-erdal Dec 20, 2022
89e43dd
Set min maxAttempts limit
ersin-erdal Dec 21, 2022
eb1988f
Merge branch 'main' into 146222-configurable-retry
ersin-erdal Dec 21, 2022
4295f9f
replace retry with attempt
ersin-erdal Dec 21, 2022
aab15d6
Merge remote-tracking branch 'origin/146222-configurable-retry' into …
ersin-erdal Dec 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/settings/alert-action-settings.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,22 @@ Specifies the time allowed for requests to external resources. Requests that tak
+
For example, `20m`, `24h`, `7d`, `1w`. Default: `60s`.

`xpack.actions.run.maxAttempts` {ess-icon}::
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says that maxAttempts indicates the number of retries, but when verifying it seems that it means the number of tries. When I set maxAttempts to 1, I only see the action run once and then not retried, when I would expect it to see it retried 1 time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I set maxAttempts: 1 and force an email error. After the first action run, I see this log:

[2022-12-21T13:22:25.464-05:00][ERROR][plugins.actions.email] Error: email action error
    at Object.executor (/Users/ying/Code/kibana/x-pack/plugins/stack_connectors/server/connector_types/stack/email/index.ts:230:9)
    at Object.wrapper [as executor] (/Users/ying/Code/kibana/node_modules/lodash/lodash.js:5255:19)
    at /Users/ying/Code/kibana/x-pack/plugins/actions/server/lib/action_executor.ts:154:38
[2022-12-21T13:22:25.464-05:00][WARN ][plugins.actions.email] action execution failure: .email:gmail: email: my gmail: an error occurred while running the action: email action error; retry: true

which says retry: true. Since maxAttempts is meant to be max retries, I would expect this to be retried once and the logs indicate it will be retried. However, the action task is deleted after this and the action is never retried.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, @ymao1! We should fix our docs to mention "attempt" instead of "retries". Basically this value confirms how many times an action will run (attempt) before aborting it.

Copy link
Contributor

@mikecote mikecote Dec 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that the getRetry / retry: true stuff can be confusing 🙈 +1 to a follow up if we didn't break it in this PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I don't think this is caused in this PR. A followup issue is fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced retry with attempt :)
And checked the retry: true in the response, action executor returns it when there is an error other than a validation error. I think it means retryable not "the last execution was a retry".

Specifies the maximum number of times an action can be attempted to run. Can be minimum 1 and maximum 10.

`xpack.actions.run.connectorTypeOverrides` {ess-icon}::
Overrides the configs under `xpack.actions.run` for the connector type with the given ID. List the connector type identifier and its settings in an array of objects.
+
For example:
[source,yaml]
--
xpack.actions.run:
maxAttempts: 1
connectorTypeOverrides:
- id: '.server-log'
maxAttempts: 5
--

[float]
[[alert-settings]]
==== Alerting settings
Expand Down
Loading