Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLEET][SECURITY_SOLUTION] Fix .fleet-artifacs index property names and add endpoint package policy migrations #94977

Merged

Conversation

paul-tavares
Copy link
Contributor

Summary

  1. Fix the schema for the .fleet-artifacts index so that property names are snake_cased instead of camleCased (change rippled through a few types, but contained internally to the artifacts code)
  2. Add Saved Object migration for endpoint package policies
Click here to see output of updated index definition
// GET .fleet-artifacs
{
  ".fleet-artifacts_1" : {
    "aliases" : {
      ".fleet-artifacts" : { }
    },
    "mappings" : {
      "dynamic" : "false",
      "_meta" : {
        "migrationHash" : "57418e5dc86c4a3245f0532fab4a6a911b3b4123"
      },
      "properties" : {
        "body" : {
          "type" : "binary"
        },
        "compression_algorithm" : {
          "type" : "keyword",
          "index" : false
        },
        "created" : {
          "type" : "date"
        },
        "decoded_sha256" : {
          "type" : "keyword"
        },
        "decoded_size" : {
          "type" : "long",
          "index" : false
        },
        "encoded_sha256" : {
          "type" : "keyword"
        },
        "encoded_size" : {
          "type" : "long",
          "index" : false
        },
        "encryption_algorithm" : {
          "type" : "keyword",
          "index" : false
        },
        "identifier" : {
          "type" : "keyword"
        },
        "package_name" : {
          "type" : "keyword"
        },
        "relative_url" : {
          "type" : "keyword"
        },
        "type" : {
          "type" : "keyword"
        }
      }
    },
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "number_of_shards" : "1",
        "provided_name" : ".fleet-artifacts_1",
        "creation_date" : "1616094927090",
        "number_of_replicas" : "1",
        "uuid" : "qPRyTEHCTTygMlWnA1ezHA",
        "version" : {
          "created" : "8000099"
        }
      }
    }
  }
}

Checklist

@paul-tavares paul-tavares added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.13.0 labels Mar 18, 2021
@paul-tavares paul-tavares self-assigned this Mar 18, 2021
…13-policy-migration

# Conflicts:
#	x-pack/plugins/fleet/server/saved_objects/migrations/to_v7_13_0.ts
migrationContext
) => {
// Endpoint specific migrations
// FIXME:PT remove `-OFF` from below once ready to be released
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jfsiii , @nchaulet for now, I just turned this off below. This needs to run only when fleet-server is in the picture. Should I bring back the xpack.fleet.agents.fleetServerEnabled flag?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the only place in which needs to fork based on that? Are there other options for detecting/inferring Fleet Server?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay to bring back the flag dependings on the timing of Fleet server for 7.13 we may have to hide some UI too, so as a temporary workaround it's probably our best solution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jfsiii I don't know the answers there.
For endpoint, I think this is the only branching we need in fleet - we already have security_solution specific flag in our code to handle one other branch.

@nchaulet thanks. I will add back in with another PR. I want this one to merge since it corrects the index property definition.

@paul-tavares paul-tavares marked this pull request as ready for review March 18, 2021 20:01
@paul-tavares paul-tavares requested review from a team as code owners March 18, 2021 20:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

Copy link
Contributor

@jfsiii jfsiii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only able to read on mobile at the moment. Left two small comments but otherwise 👍🏻

Will review more deeply on my computer later

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / Closes and opens alerts.Closing alerts Closes and opens alerts

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: expected '<span.euiBadge.euiBadge--hollow.euiBadge--iconLeft>' to have text '58', but the text was '297'
    at Context.eval (http://localhost:61191/__cypress/tests?p=cypress/integration/detection_alerts/closing.spec.ts:20209:43)

Kibana Pipeline / general / "before all" hook for "should open a modal".Open timeline Open timeline modal "before all" hook for "should open a modal"

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: Expected to find element: `[data-test-subj="title-8fd0c970-8b33-11eb-a570-113fe9d80a7c"]`, but never found it.

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Open timeline`

Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail
    at Object.openTimelineById (http://localhost:61191/__cypress/tests?p=cypress/integration/timelines/open_timeline.spec.ts:16091:15)
    at Context.eval (http://localhost:61191/__cypress/tests?p=cypress/integration/timelines/open_timeline.spec.ts:15046:28)

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @paul-tavares

Copy link
Member

@nchaulet nchaulet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Copy link
Member

@pzl pzl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@paul-tavares paul-tavares merged commit 7ebffc3 into elastic:master Mar 22, 2021
@paul-tavares paul-tavares deleted the task/olm-90513-7_13-policy-migration branch March 22, 2021 19:26
paul-tavares added a commit that referenced this pull request Mar 22, 2021
…nd add endpoint package policy migrations (#94977) (#95097)

* migration of Endpoint Integration policies to adjust artifact relative urs
* Fix ``.fleet-artifacts` property name to be snake_cased
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:Fleet Team label for Observability Data Collection Fleet team v7.13.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants