Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Response Ops][Alerting] Install resources needed for framework alerts-as-data #145581

Merged
merged 72 commits into from
Jan 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
e8581cf
wip
ymao1 Nov 16, 2022
017676e
wip
ymao1 Nov 17, 2022
c241bbd
Got ECS and alert field maps ready. Generating and installing compone…
ymao1 Nov 17, 2022
e2c4d3c
Fixing tsconfig
ymao1 Nov 17, 2022
df02fb4
Fixing checks
ymao1 Nov 17, 2022
a061175
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Nov 17, 2022
6b54cff
Fixing checks
ymao1 Nov 17, 2022
7e02900
wip
ymao1 Nov 21, 2022
c902075
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Nov 22, 2022
044e765
Adding desired schema
ymao1 Nov 28, 2022
6b40f3b
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Nov 28, 2022
bdbbc3b
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 5, 2022
128a575
Got schema generator working
ymao1 Dec 5, 2022
e708b19
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 6, 2022
64faa67
Cleaning up schema generation script and adding check to buildkite
ymao1 Dec 6, 2022
9bde6b3
Trying to reduce bundle size
ymao1 Dec 6, 2022
5b95dd6
Installing all the resources
ymao1 Dec 6, 2022
2d8b3dc
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 12, 2022
5e5f4b7
wip
ymao1 Dec 12, 2022
8ec92b5
Adding unit tests for alerts service
ymao1 Dec 13, 2022
c3deb7f
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 13, 2022
c48f2b1
Merge branch 'main' into alerting/faad-resources
kibanamachine Dec 13, 2022
60a750c
Fixing types
ymao1 Dec 13, 2022
063fdda
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 15, 2022
32da96a
Adding retries for transient ES errors and tests for installation tim…
ymao1 Dec 15, 2022
453cd7b
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 15, 2022
cd0d3c2
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 16, 2022
1d2e9d4
Fixing types
ymao1 Dec 16, 2022
29c4402
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Dec 20, 2022
463d428
Removing ECS fields from component templates
ymao1 Dec 21, 2022
8a3176c
Installing resources for different registration contexts
ymao1 Dec 22, 2022
5cdc21f
Fixing checks
ymao1 Dec 22, 2022
6a05a0a
wip fixing timing of installations
ymao1 Dec 22, 2022
d876cf1
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 3, 2023
447dd13
Registering all the things
ymao1 Jan 4, 2023
25972fe
Merging in main
ymao1 Jan 4, 2023
4d70369
[CI] Auto-commit changed files from 'node scripts/ts_project_linter -…
kibanamachine Jan 4, 2023
4b42f69
Merging in main
ymao1 Jan 4, 2023
ecf0ec4
Merge branch 'alerting/faad-resources' of https://github.com/ymao1/ki…
ymao1 Jan 4, 2023
02e8067
Fixing types
ymao1 Jan 4, 2023
8b66f8c
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 4, 2023
52b41c0
Fixing unit test
ymao1 Jan 4, 2023
9626e0c
Helper function for resource installation
ymao1 Jan 4, 2023
77afe72
Returning promise instead of waiting
ymao1 Jan 5, 2023
17c8324
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 5, 2023
88af3a3
Adding functional test
ymao1 Jan 5, 2023
d9596a7
Space aware index template
ymao1 Jan 5, 2023
05c69e7
Fixing test and updating log messages
ymao1 Jan 5, 2023
42a6730
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 5, 2023
3a81c84
Removing schema generation as not currently needed
ymao1 Jan 5, 2023
3edfdcb
Removing solution specific context
ymao1 Jan 5, 2023
87488e2
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Jan 5, 2023
ea470cf
Allowing empty fieldMap to just use common component template
ymao1 Jan 5, 2023
2f03942
Cleanup
ymao1 Jan 5, 2023
b3544fa
Merge branch 'alerting/faad-resources' of https://github.com/ymao1/ki…
ymao1 Jan 5, 2023
3167b13
Fixing unit test
ymao1 Jan 5, 2023
67e06be
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 9, 2023
49ca188
Reverting change to es query rule
ymao1 Jan 9, 2023
ae6a784
Adding comments to alerts as data fields
ymao1 Jan 9, 2023
2776474
Fixing types
ymao1 Jan 9, 2023
a8b5918
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 10, 2023
75d8004
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 11, 2023
4356f63
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 13, 2023
b842564
Merge branch 'main' into alerting/faad-resources
kibanamachine Jan 13, 2023
9176eec
Merge branch 'main' into alerting/faad-resources
kibanamachine Jan 17, 2023
4080def
Merge branch 'main' of https://github.com/elastic/kibana into alertin…
ymao1 Jan 17, 2023
3ff7e81
Merging in main
ymao1 Jan 18, 2023
0360c36
Merge branch 'main' into alerting/faad-resources
kibanamachine Jan 23, 2023
9d48b9f
PR feedback
ymao1 Jan 23, 2023
6b74e18
Throwing error regardless
ymao1 Jan 23, 2023
19b8818
Merging in main
ymao1 Jan 23, 2023
3d54034
Merge branch 'main' into alerting/faad-resources
kibanamachine Jan 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/kbn-rule-data-utils/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
* Side Public License, v 1.
*/

export * from './src/default_alerts_as_data';
export * from './src/technical_field_names';
export * from './src/alerts_as_data_rbac';
export * from './src/alerts_as_data_severity';
Expand Down
141 changes: 141 additions & 0 deletions packages/kbn-rule-data-utils/src/default_alerts_as_data.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import { ValuesType } from 'utility-types';

const KIBANA_NAMESPACE = 'kibana' as const;
const ALERT_NAMESPACE = `${KIBANA_NAMESPACE}.alert` as const;
const ALERT_RULE_NAMESPACE = `${ALERT_NAMESPACE}.rule` as const;

// kibana.space_ids - space ID(s) of the rule that created this alert
const SPACE_IDS = `${KIBANA_NAMESPACE}.space_ids` as const;

// kibana.version - Kibana version that this alert was created
const VERSION = `${KIBANA_NAMESPACE}.version` as const;

// kibana.alert.action_group - framework action group ID for this alert
const ALERT_ACTION_GROUP = `${ALERT_NAMESPACE}.action_group` as const;

// kibana.alert.duration.us - alert duration in nanoseconds - updated each execution
// that the alert is active
const ALERT_DURATION = `${ALERT_NAMESPACE}.duration.us` as const;

// kibana.alert.end - timestamp when the alert is auto-recovered by the framework
const ALERT_END = `${ALERT_NAMESPACE}.end` as const;

// kibana.alert.flapping - whether the alert is currently in a flapping state
const ALERT_FLAPPING = `${ALERT_NAMESPACE}.flapping` as const;

// kibana.alert.id - alert ID, also known as alert instance ID
const ALERT_ID = `${ALERT_NAMESPACE}.id` as const;

// kibana.alert.reason - human readable reason that this alert is active
const ALERT_REASON = `${ALERT_NAMESPACE}.reason` as const;

// kibana.alert.start - timestamp when the alert is first active
const ALERT_START = `${ALERT_NAMESPACE}.start` as const;

// kibana.alert.status - active/recovered status of alert
const ALERT_STATUS = `${ALERT_NAMESPACE}.status` as const;

// kibana.alert.time_range - time range of alert from kibana.alert.start to now
const ALERT_TIME_RANGE = `${ALERT_NAMESPACE}.time_range` as const;

// kibana.alert.uuid - unique ID for the active span of this alert
const ALERT_UUID = `${ALERT_NAMESPACE}.uuid` as const;

// kibana.alert.workflow_status - open/closed status of alert
const ALERT_WORKFLOW_STATUS = `${ALERT_NAMESPACE}.workflow_status` as const;

// kibana.alert.rule.category - rule type name for rule that generated this alert
pmuellr marked this conversation as resolved.
Show resolved Hide resolved
const ALERT_RULE_CATEGORY = `${ALERT_RULE_NAMESPACE}.category` as const;

// kibana.alert.rule.consumer - consumer for rule that generated this alert
const ALERT_RULE_CONSUMER = `${ALERT_RULE_NAMESPACE}.consumer` as const;

// kibana.alert.rule.execution.uuid - unique ID for the rule execution that generated this alert
const ALERT_RULE_EXECUTION_UUID = `${ALERT_RULE_NAMESPACE}.execution.uuid` as const;

// kibana.alert.rule.name - rule name for rule that generated this alert
const ALERT_RULE_NAME = `${ALERT_RULE_NAMESPACE}.name` as const;

// kibana.alert.rule.parameters - rule parameters for rule that generated this alert
const ALERT_RULE_PARAMETERS = `${ALERT_RULE_NAMESPACE}.parameters` as const;

// kibana.alert.rule.producer - rule type producer for rule that generated this alert
const ALERT_RULE_PRODUCER = `${ALERT_RULE_NAMESPACE}.producer` as const;

// kibana.alert.rule.tags - rule tags for rule that generated this alert
const ALERT_RULE_TAGS = `${ALERT_RULE_NAMESPACE}.tags` as const;

// kibana.alert.rule_type_id - rule type id for rule that generated this alert
const ALERT_RULE_TYPE_ID = `${ALERT_RULE_NAMESPACE}.rule_type_id` as const;

// kibana.alert.rule.uuid - rule ID for rule that generated this alert
const ALERT_RULE_UUID = `${ALERT_RULE_NAMESPACE}.uuid` as const;

const namespaces = {
KIBANA_NAMESPACE,
ALERT_NAMESPACE,
ALERT_RULE_NAMESPACE,
};

const fields = {
ALERT_ACTION_GROUP,
ALERT_DURATION,
ALERT_END,
ALERT_FLAPPING,
ALERT_ID,
ALERT_REASON,
ALERT_RULE_CATEGORY,
ALERT_RULE_CONSUMER,
ALERT_RULE_EXECUTION_UUID,
ALERT_RULE_NAME,
ALERT_RULE_PARAMETERS,
ALERT_RULE_PRODUCER,
ALERT_RULE_TAGS,
ALERT_RULE_TYPE_ID,
ALERT_RULE_UUID,
ALERT_START,
ALERT_STATUS,
ALERT_TIME_RANGE,
ALERT_UUID,
ALERT_WORKFLOW_STATUS,
SPACE_IDS,
VERSION,
};

export {
ALERT_ACTION_GROUP,
ALERT_DURATION,
ALERT_END,
ALERT_FLAPPING,
ALERT_ID,
ALERT_REASON,
ALERT_RULE_CATEGORY,
ALERT_RULE_CONSUMER,
ALERT_RULE_EXECUTION_UUID,
ALERT_RULE_NAME,
ALERT_RULE_PARAMETERS,
ALERT_RULE_PRODUCER,
ALERT_RULE_TAGS,
ALERT_RULE_TYPE_ID,
ALERT_RULE_UUID,
ALERT_START,
ALERT_STATUS,
ALERT_TIME_RANGE,
ALERT_UUID,
ALERT_WORKFLOW_STATUS,
SPACE_IDS,
VERSION,
ALERT_NAMESPACE,
ALERT_RULE_NAMESPACE,
KIBANA_NAMESPACE,
};

export type DefaultAlertFieldName = ValuesType<typeof fields & typeof namespaces>;
82 changes: 26 additions & 56 deletions packages/kbn-rule-data-utils/src/technical_field_names.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,41 +7,51 @@
*/

import { ValuesType } from 'utility-types';
import {
KIBANA_NAMESPACE,
ALERT_ACTION_GROUP,
ALERT_DURATION,
ALERT_END,
ALERT_FLAPPING,
ALERT_REASON,
ALERT_RULE_CATEGORY,
ALERT_RULE_CONSUMER,
ALERT_RULE_EXECUTION_UUID,
ALERT_RULE_NAME,
ALERT_RULE_PARAMETERS,
ALERT_RULE_PRODUCER,
ALERT_RULE_TAGS,
ALERT_RULE_TYPE_ID,
ALERT_RULE_UUID,
ALERT_START,
ALERT_STATUS,
ALERT_TIME_RANGE,
ALERT_UUID,
ALERT_WORKFLOW_STATUS,
SPACE_IDS,
VERSION,
ALERT_NAMESPACE,
ALERT_RULE_NAMESPACE,
} from './default_alerts_as_data';

const KIBANA_NAMESPACE = 'kibana' as const;

const ALERT_NAMESPACE = `${KIBANA_NAMESPACE}.alert` as const;
const ALERT_RULE_NAMESPACE = `${ALERT_NAMESPACE}.rule` as const;
const ALERT_RULE_THREAT_NAMESPACE = `${ALERT_RULE_NAMESPACE}.threat` as const;

const ECS_VERSION = 'ecs.version' as const;
const EVENT_ACTION = 'event.action' as const;
const EVENT_KIND = 'event.kind' as const;
const EVENT_MODULE = 'event.module' as const;
const SPACE_IDS = `${KIBANA_NAMESPACE}.space_ids` as const;
const TAGS = 'tags' as const;
const TIMESTAMP = '@timestamp' as const;
const VERSION = `${KIBANA_NAMESPACE}.version` as const;

// Fields pertaining to the alert
const ALERT_ACTION_GROUP = `${ALERT_NAMESPACE}.action_group` as const;
const ALERT_BUILDING_BLOCK_TYPE = `${ALERT_NAMESPACE}.building_block_type` as const;
const ALERT_DURATION = `${ALERT_NAMESPACE}.duration.us` as const;
const ALERT_END = `${ALERT_NAMESPACE}.end` as const;
const ALERT_EVALUATION_THRESHOLD = `${ALERT_NAMESPACE}.evaluation.threshold` as const;
const ALERT_EVALUATION_VALUE = `${ALERT_NAMESPACE}.evaluation.value` as const;
const ALERT_FLAPPING = `${ALERT_NAMESPACE}.flapping` as const;
const ALERT_INSTANCE_ID = `${ALERT_NAMESPACE}.instance.id` as const;
const ALERT_REASON = `${ALERT_NAMESPACE}.reason` as const;
const ALERT_RISK_SCORE = `${ALERT_NAMESPACE}.risk_score` as const;
const ALERT_SEVERITY = `${ALERT_NAMESPACE}.severity` as const;
const ALERT_START = `${ALERT_NAMESPACE}.start` as const;
const ALERT_TIME_RANGE = `${ALERT_NAMESPACE}.time_range` as const;
const ALERT_STATUS = `${ALERT_NAMESPACE}.status` as const;
const ALERT_SYSTEM_STATUS = `${ALERT_NAMESPACE}.system_status` as const;
const ALERT_UUID = `${ALERT_NAMESPACE}.uuid` as const;
const ALERT_WORKFLOW_REASON = `${ALERT_NAMESPACE}.workflow_reason` as const;
const ALERT_WORKFLOW_STATUS = `${ALERT_NAMESPACE}.workflow_status` as const;
const ALERT_WORKFLOW_USER = `${ALERT_NAMESPACE}.workflow_user` as const;
const ALERT_SUPPRESSION_META = `${ALERT_NAMESPACE}.suppression` as const;
const ALERT_SUPPRESSION_TERMS = `${ALERT_SUPPRESSION_META}.terms` as const;
Expand All @@ -61,22 +71,16 @@ const ALERT_RULE_CREATED_BY = `${ALERT_RULE_NAMESPACE}.created_by` as const;
const ALERT_RULE_DESCRIPTION = `${ALERT_RULE_NAMESPACE}.description` as const;
const ALERT_RULE_ENABLED = `${ALERT_RULE_NAMESPACE}.enabled` as const;
const ALERT_RULE_EXCEPTIONS_LIST = `${ALERT_RULE_NAMESPACE}.exceptions_list` as const;
const ALERT_RULE_EXECUTION_UUID = `${ALERT_RULE_NAMESPACE}.execution.uuid` as const;
const ALERT_RULE_FROM = `${ALERT_RULE_NAMESPACE}.from` as const;
const ALERT_RULE_INTERVAL = `${ALERT_RULE_NAMESPACE}.interval` as const;
const ALERT_RULE_LICENSE = `${ALERT_RULE_NAMESPACE}.license` as const;
const ALERT_RULE_CATEGORY = `${ALERT_RULE_NAMESPACE}.category` as const;
const ALERT_RULE_NAME = `${ALERT_RULE_NAMESPACE}.name` as const;
const ALERT_RULE_NAMESPACE_FIELD = `${ALERT_RULE_NAMESPACE}.namespace` as const;
const ALERT_RULE_NOTE = `${ALERT_RULE_NAMESPACE}.note` as const;
const ALERT_RULE_PARAMETERS = `${ALERT_RULE_NAMESPACE}.parameters` as const;
const ALERT_RULE_REFERENCES = `${ALERT_RULE_NAMESPACE}.references` as const;
const ALERT_RULE_RULE_ID = `${ALERT_RULE_NAMESPACE}.rule_id` as const;
const ALERT_RULE_RULE_NAME_OVERRIDE = `${ALERT_RULE_NAMESPACE}.rule_name_override` as const;
const ALERT_RULE_TAGS = `${ALERT_RULE_NAMESPACE}.tags` as const;
const ALERT_RULE_TO = `${ALERT_RULE_NAMESPACE}.to` as const;
const ALERT_RULE_TYPE = `${ALERT_RULE_NAMESPACE}.type` as const;
const ALERT_RULE_TYPE_ID = `${ALERT_RULE_NAMESPACE}.rule_type_id` as const;
const ALERT_RULE_UPDATED_AT = `${ALERT_RULE_NAMESPACE}.updated_at` as const;
const ALERT_RULE_UPDATED_BY = `${ALERT_RULE_NAMESPACE}.updated_by` as const;
const ALERT_RULE_VERSION = `${ALERT_RULE_NAMESPACE}.version` as const;
Expand All @@ -97,16 +101,6 @@ const ALERT_THREAT_TECHNIQUE_SUBTECHNIQUE_NAME =
const ALERT_THREAT_TECHNIQUE_SUBTECHNIQUE_REFERENCE =
`${ALERT_RULE_THREAT_NAMESPACE}.technique.subtechnique.reference` as const;

// the feature instantiating a rule type.
// Rule created in stack --> alerts
// Rule created in siem --> siem
const ALERT_RULE_CONSUMER = `${ALERT_RULE_NAMESPACE}.consumer` as const;
// the plugin that registered the rule type.
// Rule type apm.error_rate --> apm
// Rule type siem.signals --> siem
const ALERT_RULE_PRODUCER = `${ALERT_RULE_NAMESPACE}.producer` as const;
const ALERT_RULE_UUID = `${ALERT_RULE_NAMESPACE}.uuid` as const;

const namespaces = {
KIBANA_NAMESPACE,
ALERT_NAMESPACE,
Expand Down Expand Up @@ -189,23 +183,12 @@ const fields = {
};

export {
ALERT_ACTION_GROUP,
ALERT_BUILDING_BLOCK_TYPE,
ALERT_DURATION,
ALERT_END,
ALERT_EVALUATION_THRESHOLD,
ALERT_EVALUATION_VALUE,
ALERT_FLAPPING,
ALERT_INSTANCE_ID,
ALERT_NAMESPACE,
ALERT_RULE_NAMESPACE,
ALERT_RULE_CONSUMER,
ALERT_RULE_PRODUCER,
ALERT_REASON,
ALERT_RISK_SCORE,
ALERT_STATUS,
ALERT_WORKFLOW_REASON,
ALERT_WORKFLOW_STATUS,
ALERT_WORKFLOW_USER,
ALERT_CASE_IDS,
ALERT_RULE_AUTHOR,
Expand All @@ -214,36 +197,25 @@ export {
ALERT_RULE_DESCRIPTION,
ALERT_RULE_ENABLED,
ALERT_RULE_EXCEPTIONS_LIST,
ALERT_RULE_EXECUTION_UUID,
ALERT_RULE_FROM,
ALERT_RULE_INTERVAL,
ALERT_RULE_LICENSE,
ALERT_RULE_NAME,
ALERT_RULE_NAMESPACE_FIELD,
ALERT_RULE_NOTE,
ALERT_RULE_PARAMETERS,
ALERT_RULE_REFERENCES,
ALERT_RULE_RULE_ID,
ALERT_RULE_RULE_NAME_OVERRIDE,
ALERT_RULE_TAGS,
ALERT_RULE_TO,
ALERT_RULE_TYPE,
ALERT_RULE_TYPE_ID,
ALERT_RULE_UPDATED_AT,
ALERT_RULE_UPDATED_BY,
ALERT_RULE_VERSION,
ALERT_SEVERITY,
ALERT_START,
ALERT_TIME_RANGE,
ALERT_SYSTEM_STATUS,
ALERT_UUID,
ECS_VERSION,
EVENT_ACTION,
EVENT_KIND,
EVENT_MODULE,
KIBANA_NAMESPACE,
ALERT_RULE_UUID,
ALERT_RULE_CATEGORY,
ALERT_THREAT_FRAMEWORK,
ALERT_THREAT_TACTIC_ID,
ALERT_THREAT_TACTIC_NAME,
Expand All @@ -262,8 +234,6 @@ export {
ALERT_SUPPRESSION_DOCS_COUNT,
TAGS,
TIMESTAMP,
SPACE_IDS,
VERSION,
};

export type TechnicalRuleDataFieldName = ValuesType<typeof fields & typeof namespaces>;
Loading