Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.10][Telemetry] Update data collection panel to show example security event. #78262

Closed
wants to merge 10,000 commits into from
Closed

[7.10][Telemetry] Update data collection panel to show example security event. #78262

wants to merge 10,000 commits into from

Conversation

pjhampton
Copy link
Contributor

@pjhampton pjhampton commented Sep 23, 2020

Summary

Resolves #68942
Related to #77200
Related to elastic/protections-team#172 (Data Engineering)

This PR adds a fly-out with an example of a security event collected from Elastic Endpoint when enabled.

Screenshot 2020-09-23 at 10 43 52

Screenshot 2020-09-23 at 10 44 17

Checklist

Delete any items that are not applicable to this PR.

For maintainers

gmmorris and others added 27 commits September 22, 2020 16:01
…78134)

Adds a tab in the _Edit Alert_ flyout which allows the user to _test_ their connector by executing it using an example action. The execution relies on the connector being updated, so is only enabled when there are no saved changes in the Connector form itself.
…78159)

This PR optimizes the calculation of instances which should be executed, by optimizing the way the muted instances are removed from the collection of triggered instances.
…ecting to Enterprise Search (#78091) (#78164)

* Display error connecting prompt on Overview page instead of blank page

* Fix App Search and Workplace Search to not crash during error connecting
- due to obj type errors
…#78078)

* [Security Solution][Resolver] Show all event counts
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Adding bulk upgrade api

* Addressing comments

* Removing todo

* Changing body field

* Adding helper for getting the bulk install route

* Adding request spec

* Pulling in Johns changes

* Removing test for same package upgraded multiple times

* Pulling in John's error handling changes

* Fixing type error
This is a re-do of PR #74063, which was merged to the wrong branch

Co-authored-by: Nimex94 <34445912+Nimex94@users.noreply.github.com>
…issing dataset (#76775) (#78184)

Backports the following commits to 7.x:
 - [Logs UI] Correctly filter for log rate anomaly examples with missing dataset (#76775)
…prefer directly mounting (#78167) (#78204)

* Remove HttpProvider in favor of mounting HttpLogic directly w/ props

- removes need for initializeHttp call
- ensures http value is loaded into HttpLogic as soon as possible / should never load in as null, reducing # of rerenders/checks

see: https://kea.js.org/docs/guide/advanced#mounting-and-unmounting

* Update simplest components using http for sendTelemetry

* Update simplest tests for components using HttpLogic + default Kea mocks

- Kea mock import should now contain mock default values which can be overridden

* Update moderately complex tests using HttpLogic

send_telemetry:
- refactor to use shallow (w/ useEffect mocked) vs mount
- check mockHttpValues directly

engine_table:
- refactor to use mount w/ an I18nProvider rather than mountWithContext helper (which we'll likely need to overhaul in the future)
- assert mockHttpValues directly

* Update EngineOverview to HttpLogic + refactors

EngineOverview:
- Change use of FormattedMessage to i18n.translate (simpler, no provider required)

Tests:
- Create mock values/actions for FlashMessages, since EngineOverview calls it
- Create combined mockAllValues obj for easier overriding
- Create setMockValues helper for easier test overriding (credit to @scottybollinger for the idea!)
- Update engine_overview tests to setMockValues instead of passing context to mountWithAsyncContext
- Fix mountWithAsyncContext to accept an undefined obj

* Remove http from KibanaContext

- it should now only live in HttpLogic 🔥

* Remove FlashMessagesProvider in favor of mounting logic directly w/ props

- send history as prop
- refactor out now-unnecessary listenToHistory (we can just do it directly in afterMount without worrying about duplicate react rerenders)
- add mount helper

Tests:
- refactor history.listen mock to mockHistory (so that set_message_helpers can use it as well)
- use mountFlashMessagesLogic + create an even shorter mount() helper (credit to @JasonStoltz for the idea!)
- refactor out DEFAULT_VALUES since we're not really using it anywhere else in the file, and it's not super applicable to this store
- update history listener tests to account for logic occurring immediately on mount
…r Calendars and Filter Lists (#77566) (#78198)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Reporting] Clean Up TypeScript Definitions (#76566)

* [Reporting] Simplify Export Type Definitions, use defaults for generics, refactor

* ReportApiJSON interface for common

* rename JobSummary to JobStatusBucket for clarity

* revert unneeded create mock changes

* clean up the diff

* revert changes to worker.js

* rewrite comment

* rename type to jobtype in JobStatusBucket

* allow type inference

* JobSummarySet

* remove odd comment

* Reflect that browser timezone may be undefined in the BaseParams

* comment about optional browserTimezone

* revert unecessary es archive change

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* fix ts

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…78197)

* Rollback the logger & try/catch changes from #77640 (#77806)

* Doing a try/catch and re-throwing doesn't gain us anything. We already catching the error in the route handler
 * We have logging for the issue in the existing handler. We also don't pass a logging context to functions

* Add missing import
…78240)

# Conflicts:
#	src/dev/build/tasks/os_packages/docker_generator/resources/bin/kibana-docker
* Remove [key: string]: any; from IIndexPattern (#77968)
* Bump react-beautiful-dnd

* fix types

* fix types
* remove ALL the things.

* adapt some types and tests

* restore ensureValidConfiguration

* fix legacy service tests

* adapt uiRender mixin

* remove legacy types

* update generated doc

* restore legacy plugin schema

* update generated doc

* remove remaining code of x-pack/legacy

* adapt imports due to merge

* cleanup CODEOWNERS

* cleanup gitignore & i18nrc

* cleanup tsconfig.json

* remove unused i18n keys

* add back `"legacy/plugins/**/*",` to tsconfig until legacy space plugin is deleted

* fix create_jest_config

* remove references from eslintrc

* more eslint cleanup

* remove `x-pack/index.js`

* fix xpack gulp scripts

* fix bug with default + named imports from boom

* remove rules from eslintrc

* remove LegacyInternals

* review comments

* update generated doc

* cleanup legacy metadatas

* revert changes to eslintrc

* update generated doc
# Conflicts:
#	.github/CODEOWNERS
#	x-pack/.i18nrc.json
#	x-pack/dev-tools/jest/create_jest_config.js
#	x-pack/legacy/plugins/xpack_main/index.js
#	x-pack/legacy/server/lib/constants/index.ts
#	x-pack/legacy/server/lib/key_case_converter.js
#	x-pack/legacy/server/lib/watch_status_and_license_to_initialize.js
simianhacker and others added 15 commits September 29, 2020 14:38
…8859)

* [Metrics UI] Fix EC2 Query to only include aws.ec2 nodes

* Making the filter more generic so we can apply it easily to any inventory model
…8102) (#78857)

* wip

* advanced settings

* removed unused vars

* unused vars

* fix label issue and ignore duplicate id issue

* fixing advanced management

* fixing jest tests

* i18n translations

* i18n translations

* fixed jest tests

* addressed review comments

* addressed review comments

* addressed review comments

* reverted the snapshot

Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Make Field a React.lazy export

* Fix broken types

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
… main app + share its store (#78691) (#78866)

* [Setup] Refactor renderApp tests

- This is anticipation of upcoming renderHeaderActions changes which will require these helpers (particularly the unmount portion, for some reason renderHeaderAction tests fail if apps are not unmounted properly)

- Add mount/unmount helper (but leave first test the same so that devs can see the normal/expected usage)
- Add missing EnterpriseSearch app test
- Add mockContainer var for brevity, pull out MockApp in anticipation of future usage

* Store renderHeaderActions in KibanaLogic

- a light wrapper/helper around params.setHeaderActionMenu

+ update renderHeaderActions - move to main renderApp block to reflect that its relationship with renderApp ('child'/should be called within renderApp/shares dependencies).

* Update WorkplaceSearch to render its header actions from app, not plugin.ts

* Update WorkplaceSearch tests (+ bonus refactor)

- We need to update WS tests to setMockValues so that it doesn't override the renderHeaderActions mock
+ bonus - add setMockActions as well because since we're already here
+ bonus - update App Search index tests as well to match
+ ?? - for some reason tests were failing beceause react router wasn't mocked properly - requireActual seems to fix that

* 🔥 Remove temporary externalUrl workaround

- set in 9d993d8

- WorkplaceSearchHeaderActions should now still have the correct URL - and also now be able to access all Kea logic set up by the main app :)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Add `wrap` to the flex group for the APM header and remove the `break-all` CSS.

Makes it so the date picker doesn't get cut off on narrow screens and the header text does not wrap unnecessarily.
Add OpenTelemetry agent names and and icon. OpenTelemetry services will have the same icon as the corresponding APM services.

Services that report "otlp" as their agent name use the OpenTelemetry icon.

The OpenTelemetry agents are now collected in the data telemetry along with the other agent names.

Change "ratio" in the telemetry to a float.
# Conflicts:
#	src/plugins/data/server/server.api.md
…ion/classification views (#78702) (#78883)

* add training quickfilter buttons in results view

* explorationQueryBar- add ability to use quickfilters

* update type for button group
Move indexPattern.popularizeField into discover
@pjhampton
Copy link
Contributor Author

@elasticmachine merge upstream

elasticmachine and others added 2 commits September 30, 2020 03:56
…emetry_management_section.tsx

Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
@pjhampton
Copy link
Contributor Author

@TinaHeiligers RE

The usage data is the only section that doesn't have the title there. So for consistency, it would be better to also list there.

I spent an hour looking into this. It is because the Telemetry Management Section registers itself as a footer to the Advanced Settings plugin:

export class TelemetryManagementSectionPlugin implements Plugin {
public setup(
core: CoreSetup,
{ advancedSettings, telemetry: { telemetryService } }: TelemetryManagementSectionPluginDepsSetup
) {
advancedSettings.component.register(
advancedSettings.component.componentType.PAGE_FOOTER_COMPONENT,
telemetryManagementSectionWrapper(telemetryService),
true
);
}
(See L47)

I had my findings confirmed by a Kibana Arch team member. They described the change as a non-negligible amount of coding. Because of not wanting to hold this change up it can be addressed in another PR - either by me or you. Regardless, this was a great find by the design team!

@pjhampton pjhampton changed the base branch from 7.x to master September 30, 2020 10:23
@pjhampton pjhampton requested review from a team as code owners September 30, 2020 10:23
@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 30, 2020

💔 Build Failed

Failed CI Steps


Test Failures

Jest Tests.src/plugins/telemetry_management_section/public/components.OptInDetailsComponent security flyout renders as expected

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches


Stack Trace

Error: expect(received).toMatchSnapshot()

Snapshot name: `OptInDetailsComponent security flyout renders as expected 1`

- Snapshot  - 2
+ Received  + 2

@@ -77,16 +77,16 @@
      "agent": {
        "id": "b2e88aea-2671-402a-828a-957526bac315"
      }
    },
    "file": {
-     "path": "C:\\Users\\User\\Downloads\\mimikatz_trunk\\x64\\mimikatz.exe",
+     "path": "C:\\Windows\\Temp\\mimikatz.exe",
      "size": 1263880,
      "created": "2020-05-19T07:50:06.0Z",
      "accessed": "2020-09-22T14:29:19.93531400Z",
      "mtime": "2020-09-22T14:29:03.6040000Z",
-     "directory": "C:\\Users\\User\\Downloads\\mimikatz_trunk\\x64",
+     "directory": "C:\\Windows\\Temp",
      "hash": {
        "sha1": "c9fb7f8a4c6b7b12b493a99a8dc6901d17867388",
        "sha256": "cb1553a3c88817e4cc774a5a93f9158f6785bd3815447d04b6c3f4c2c4b21ed7",
        "md5": "465d5d850f54d9cde767bda90743df30"
      },
    at Object.it (/dev/shm/workspace/parallel/6/kibana/src/plugins/telemetry_management_section/public/components/opt_in_security_example_flyout.test.tsx:25:81)
    at Object.asyncJestTest (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)
    at resolve (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:45:12)
    at new Promise (<anonymous>)
    at mapper (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:28:19)
    at promise.then (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:75:41)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Metrics [docs]

‼️ ERROR: no builds found for mergeBase sha [012b6fe]

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@afharo
Copy link
Member

afharo commented Sep 30, 2020

I think this branch was originally created from 7.x as a base and it's messing up quite a lot when now trying to merge it into master. @pjhampton can you rebase your branch to master instead (or maybe it'll be easier to create a new branch from master and apply the changes again)

@pjhampton
Copy link
Contributor Author

Yeah, agreed @afharo. This branch is toast. I will re-open a fresh PR 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow additional solutions to take advantage of the telemetry opt-in