[Response Ops][Alerting] Install framework alerts-as-data resources on startup #145100
Comments
ymao1
added
Team:ResponseOps
|
Pinging @elastic/response-ops (Team:ResponseOps) |
|
This issue can be started but complete implementation is blocked on final approval of the standard FAAD schema (TTL Nov 18). |
|
With the recent change in direction for alerts-as-data, we will want to install multiple indices on startup, one for each existing rule registry registration context and a new one for stack rules. Each registration context will be able to register custom index mappings on top of the default framework mappings and a concrete write index will be created for the default space. I will create a followup issue for space-specific indices. Current registration contexts: |
|
Have we discussed if O11y can consolidate? This granular rule registry grouping is legacy as far as I understand it. |
|
Yes, combining registration contexts was discussed and since we are pivoting to having solutions manage their own mappings again, they wanted to keep the separation that registration contexts provide |
|
Please let's ensure the existing indices or data streams are not modified and everything is handled purely by index templates to avoid situations like #139969. |
…s-as-data (#145581) Resolves #145100 ## Summary * Adds alerting config for `enableFrameworkAlerts` * When `xpack.alerting.enableFrameworkAlerts=true`, `AlertsService` is initialized during the alerting plugin setup phase * Adds optional `alerts` definition to the `RuleType` definition which allows a rule type to specify a context and a field mapping for that context * `AlertsService.initialize()` * installs an ILM policy that will be used by all alerts-as-data indices - `alerts-default-ilm-policy` * installs a component template that will be used by all alerts-as-data indices - `alerts-common-component-template`, including all the mappings for fields needed by the framework * Rule type registration - when a rule type with an `alerts` definition is registered, the context and field map are registered with the `AlertService`. When `AlertsService` initialization is completed successfully, context specific resources are installed. * Context specific resources: * installs context specific component template that reflects the registered field map - `alerts-${context}-component-template` * installs context specific index template for the default namespace - `.alerts-${context}-default-template` * creates context specific concrete write index for the default namespace - `.alerts-${context}-default-000001` * Resource installation retries for transient ES errors and short-circuits when a timeout value is reached ## Notes * We explore the possibility of creating a single index template per context and re-using it for space-aware concrete indices but a rollover alias (which must be space-aware) must be attached to an index template so it is not feasible to only have a single index template * There will be a followup issue for create space-specific index templates/indices as needed to support detection rules desire to separate alerts by space. ## To Verify * set `xpack.alerting.enableFrameworkAlerts: true` in your kibana.yml * modify `x-pack/plugins/stack_alerts/server/rule_types/es_query/rule_type.ts` to define a custom field mapping for the `stack` context ``` --- a/x-pack/plugins/stack_alerts/server/rule_types/es_query/rule_type.ts +++ b/x-pack/plugins/stack_alerts/server/rule_types/es_query/rule_type.ts @@ -187,5 +187,14 @@ export function getRuleType( }, producer: STACK_ALERTS_FEATURE_ID, doesSetRecoveryContext: true, + alerts: { + context: 'stack', + fieldMap: { + 'kibana.alert.threshold': { + required: false, + type: 'long', + }, + }, + }, }; } ``` * start up ES and Kibana and verify that the correct resources are installed. The following should be created: * ILM policy - `alerts-default-ilm-policy` * Common component template - `alerts-common-component-template` * Stack component template - `alerts-stack-component-template` * Stack index template for default space - `.alerts-stack-default-template` * Stack write index for default space - `.alerts-stack-default-000001` * verify that the index template uses both component templates and that the expected mappings are applied to the index * try making non-destructive, additive only changes to the common component template or the stack context field mapping and verify that the concrete index mappings are updated. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
As part of Phase 2 of framework alerts-as-data, we need to install alerts as data index resources on startup. Using the best practices outlined here, we should install an ILM policy, component template, index template and concrete write index. The FAAD schema should be a superset of all ECS field and the initial default set of framework fields.
Implementation details to iron out as part of this issue:
POC for possible implementation here. This issue would cover the
AlertsServiceportion of the POC but does not contain any of the best practices linked in the above issue.The text was updated successfully, but these errors were encountered: