Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing ES Promotion: FTR Configs #22 / detection engine api security and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point #154277

Closed
Tracked by #161531
mistic opened this issue Apr 3, 2023 · 6 comments
Assignees
Labels
failed-es-promotion Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@mistic
Copy link
Member

mistic commented Apr 3, 2023

X-Pack Detection Engine API Integration Tests
x-pack/test/detection_engine_api_integration/security_and_spaces/rule_execution_logic/non_ecs_fields.ts

detection engine api security and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point

This failure is preventing the promotion of the current Elasticsearch nightly snapshot.

For more information on the Elasticsearch snapshot promotion process including how to reproduce using the unverified nightly ES build: https://www.elastic.co/guide/en/kibana/master/development-es-snapshots.html

JestAssertionError: expect(received).toContain(expected) // indexOf

Expected value: "Bulk Indexing of signals failed: failed to parse field [client.geo.location] of type [geo_point]"
Received array: ["Bulk Indexing of signals failed: [1:1201] failed to parse field [client.geo.location] of type [geo_point]"]
    at Context.<anonymous> (non_ecs_fields.ts:273:22)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (wrap_function.js:73:16) {
  matcherResult: {
    message: '\x1B[2mexpect(\x1B[22m\x1B[31mreceived\x1B[39m\x1B[2m).\x1B[22mtoContain\x1B[2m(\x1B[22m\x1B[32mexpected\x1B[39m\x1B[2m) // indexOf\x1B[22m\n' +
      '\n' +
      'Expected value: \x1B[32m"Bulk Indexing of signals failed: failed to parse field [client.geo.location] of type [geo_point]"\x1B[39m\n' +
      'Received array: \x1B[31m["Bulk Indexing of signals failed: [1:1201] failed to parse field [client.geo.location] of type [geo_point]"]\x1B[39m',
    pass: false
  }
}
@mistic mistic added skipped-test failed-es-promotion Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Alerts Security Detection Alerts Area Team v8.8.0 labels Apr 3, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@mistic
Copy link
Member Author

mistic commented Apr 3, 2023

Skipped.

main: 06f5a01

@MadameSheema MadameSheema added the Team:Detections and Resp Security Detection Response Team label Apr 3, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@WafaaNasr
Copy link
Contributor

WafaaNasr commented Jul 13, 2023

Running through https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2626#01894ed1-0606-4462-bf82-697455429fdd.

The first test fails because of

  1. detection engine api security and spaces enabled - rule execution logic

Non ECS fields in alert document source
  should remove source array of keywords field from alert if ECS field mapping is nested:
 
  Error: expect(received).toHaveProperty(path, value)
  
  Expected path: "threat.indicator.port"
  Received path: "threat"
   
  Expected value: 443
  Received value: {"enrichments": [], "indicator.port": 443}
  at Context. (non_ecs_fields.ts:234:27)
  at runMicrotasks ()
  at processTicksAndRejections (node:internal/process/task_queues:96:5)
  at Object.apply (wrap_function.js:73:16)

 

@WafaaNasr
Copy link
Contributor

PR used to skip the test #161846

@WafaaNasr WafaaNasr removed their assignment Jul 17, 2023
@e40pud e40pud self-assigned this Jul 19, 2023
e40pud added a commit to e40pud/kibana that referenced this issue Jul 19, 2023
…ecurity and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point (elastic#154277)
e40pud added a commit that referenced this issue Jul 20, 2023
… and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point (#162247)

## Summary

Failing tests ticket: #154277

This PR fixes the non ECS fields in alert document source failing tests.

There are two failing tests:
1. `should remove source array of keywords field from alert if ECS field
mapping is nested` was filing due to wrong key path format passed to
jest's `toHaveProperty`. When the field name has dot notation we should
be using array format as a key path. See discussion
[here](jestjs/jest#5653) and usage examples
[here](https://github.com/jestjs/jest/blob/main/docs/ExpectAPI.md#tohavepropertykeypath-value).
2. `should fail creating alert when ECS field mapping is geo_point` was
failing due to changed error message format.
@e40pud
Copy link
Contributor

e40pud commented Jul 20, 2023

Un-skipped in 8.10.0

dgieselaar pushed a commit to dgieselaar/kibana that referenced this issue Jul 23, 2023
… and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point (elastic#162247)

## Summary

Failing tests ticket: elastic#154277

This PR fixes the non ECS fields in alert document source failing tests.

There are two failing tests:
1. `should remove source array of keywords field from alert if ECS field
mapping is nested` was filing due to wrong key path format passed to
jest's `toHaveProperty`. When the field name has dot notation we should
be using array format as a key path. See discussion
[here](jestjs/jest#5653) and usage examples
[here](https://github.com/jestjs/jest/blob/main/docs/ExpectAPI.md#tohavepropertykeypath-value).
2. `should fail creating alert when ECS field mapping is geo_point` was
failing due to changed error message format.
ThomThomson pushed a commit to ThomThomson/kibana that referenced this issue Aug 1, 2023
… and spaces enabled - rule execution logic Non ECS fields in alert document source should fail creating alert when ECS field mapping is geo_point (elastic#162247)

## Summary

Failing tests ticket: elastic#154277

This PR fixes the non ECS fields in alert document source failing tests.

There are two failing tests:
1. `should remove source array of keywords field from alert if ECS field
mapping is nested` was filing due to wrong key path format passed to
jest's `toHaveProperty`. When the field name has dot notation we should
be using array format as a key path. See discussion
[here](jestjs/jest#5653) and usage examples
[here](https://github.com/jestjs/jest/blob/main/docs/ExpectAPI.md#tohavepropertykeypath-value).
2. `should fail creating alert when ECS field mapping is geo_point` was
failing due to changed error message format.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-es-promotion Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
Development

No branches or pull requests

8 participants