Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: package aliases for snapshots #21960

Merged
merged 9 commits into from
Oct 21, 2020

Conversation

mdelapenya
Copy link
Contributor

@mdelapenya mdelapenya commented Oct 19, 2020

What does this PR do?

It modifies the packaging job on CI to include the aliases (7.9-SNAPSHOT, 7.10-SNAPSHOT, 8.0-SNAPSHOT, etc) when packaging Beats in the Docker image format, resulting in two images: one for the version from libbeat and another for the alias. This new behaviour will only appear for snapshots.

We extracted the common logic for tagging and pushing a docker image to a method, so that it's reusable.

Why is it important?

With this code the CI will be able to push the alias images for Beats to the observability-ci namespace.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

Follow-ups

I think this should be backported to the following maintenance branches:

  • 7.x
  • 7.9
  • 7.10

@mdelapenya mdelapenya self-assigned this Oct 19, 2020
@mdelapenya mdelapenya added the Team:Automation Label for the Observability productivity team label Oct 19, 2020
@mdelapenya mdelapenya requested a review from a team October 19, 2020 11:26
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 19, 2020
@mdelapenya mdelapenya changed the title feat: package aliases ffor snapshots feat: package aliases for snapshots Oct 19, 2020
.ci/packaging.groovy Outdated Show resolved Hide resolved
@@ -182,13 +182,26 @@ func (b *dockerBuilder) expandDockerfile(templatesDir string, data map[string]in

func (b *dockerBuilder) dockerBuild() (string, error) {
tag := fmt.Sprintf("%s:%s", b.imageName, b.Version)
aliasTag := ""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsoriano I was not able to easily execute this locally (create the elastic-agent docker image) with one single command. Do you know the build steps?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mage package in the elastic-agent directory should generate all the packages, including the docker images. There is no command to generate only the docker images.
You may want to select the platforms with the PLATFORM environment variable, if not it will generate all the packages for all the platforms, what will probably take a while.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 19, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21960 updated]

  • Start Time: 2020-10-19T16:22:47.194+0000

  • Duration: 28 min 26 sec

Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this code the CI will be able to push the alias images for Beats to the observability-ci namespace.

Just curious, why is a new alias needed for this?

@@ -182,13 +182,26 @@ func (b *dockerBuilder) expandDockerfile(templatesDir string, data map[string]in

func (b *dockerBuilder) dockerBuild() (string, error) {
tag := fmt.Sprintf("%s:%s", b.imageName, b.Version)
aliasTag := ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mage package in the elastic-agent directory should generate all the packages, including the docker images. There is no command to generate only the docker images.
You may want to select the platforms with the PLATFORM environment variable, if not it will generate all the packages for all the platforms, what will probably take a while.

dev-tools/mage/dockerbuilder.go Outdated Show resolved Hide resolved
@jsoriano
Copy link
Member

/package

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
@mdelapenya
Copy link
Contributor Author

Just curious, why is a new alias needed for this?

We'd like to have a match between the Git branch, the docker image and the binaries, to be consumed in downstream projects, such as the E2E tests (see elastic/e2e-testing#374):

Git branch Docker image  Docker alias GCP Storage
7.9 7.9.3-SNAPSHOT 7.9-SNAPSHOT 7.9.3-SNAPSHOT
7.10 7.10.0-SNAPSHOT 7.10-SNAPSHOT 7.10.0-SNAPSHOT
master 8.0.0-SNAPSHOT 8.0-SNAPSHOT 8.0.0-SNAPSHOT

@mdelapenya
Copy link
Contributor Author

/package

1 similar comment
@mdelapenya
Copy link
Contributor Author

/package

@jsoriano
Copy link
Member

We'd like to have a match between the Git branch, the docker image and the binaries, to be consumed in downstream projects, such as the E2E tests

Snapshots are built from release branches that could change, could this be a source of flakiness? Or images in observability-ci namespace are retagged to have controlled versions?

@mdelapenya
Copy link
Contributor Author

mdelapenya commented Oct 19, 2020

We'd like to have a match between the Git branch, the docker image and the binaries, to be consumed in downstream projects, such as the E2E tests

Snapshots are built from release branches that could change, could this be a source of flakiness? Or images in observability-ci namespace are retagged to have controlled versions?

Yes, we retag for merges and PRs

We could tag the observability-ci images only on CI, as we do in the packaging.groovy file in this PR, skipping the Go code. I added it for the reproducibility sake, so it's possible to perform it locally too

@@ -191,10 +191,14 @@ def pushCIDockerImages(){
}
}

def tagAndPush(name){
def tagAndPush(beatName){
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rename of this variable is a cosmetic change for readability sake

def oldName = "${DOCKER_REGISTRY}/beats/${name}${variant}:${libbetaVer}"
def newName = "${DOCKER_REGISTRY}/observability-ci/${name}${variant}:${tagName}"
def commitName = "${DOCKER_REGISTRY}/observability-ci/${name}${variant}:${env.GIT_BASE_COMMIT}"
doTagAndPush(beatName, variant, libbetaVer, tagName)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm moving the generation of the docker image name to the method, hiding the implementation

* @param sourceTag tag to be used as source for the docker tag command, usually under the 'beats' namespace
* @param targetTag tag to be used as target for the docker tag command, usually under the 'observability-ci' namespace
*/
def doTagAndPush(beatName, variant, sourceTag, targetTag) {
Copy link
Contributor Author

@mdelapenya mdelapenya Oct 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @v1v for the suggestion! this refactor extracting the common logic to a method helps with potential debug if/when tag/push fails, as each retagged image is decoupled.

@mdelapenya mdelapenya marked this pull request as ready for review October 19, 2020 15:42
@mdelapenya mdelapenya added ci needs_backport PR is waiting to be backported to other branches. labels Oct 19, 2020
@mdelapenya
Copy link
Contributor Author

/package

docker push ${targetName}
""", returnStatus: true)

if ( status > 0 && iterations < 3) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do all beats has OSS and UBI8 variant? IIRC not all have OSS, so it will throw 3 errors on those cases, it is a little weird fail by default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are logging the message on errors. We could build a data structure with the combinations, but we thought that this was simpler

@mdelapenya mdelapenya merged commit 65df4e1 into elastic:master Oct 21, 2020
v1v added a commit to v1v/beats that referenced this pull request Oct 21, 2020
…ter-commit

* upstream/master:
  feat: package aliases for snapshots (elastic#21960)
  [DOC] Add firewall as possible troubleshooting issue (elastic#21743)
  [Filebeat] Add max_number_of_messages config parameter for S3 input (elastic#21993)
  [Elastic Agent] Fix missing elastic_agent event data  (elastic#21994)
  Document auditbeat system process module config (elastic#21766)
  Update links (elastic#22012)
v1v added a commit to v1v/beats that referenced this pull request Oct 21, 2020
* upstream/master:
  feat: package aliases for snapshots (elastic#21960)
  [DOC] Add firewall as possible troubleshooting issue (elastic#21743)
  [Filebeat] Add max_number_of_messages config parameter for S3 input (elastic#21993)
  [Elastic Agent] Fix missing elastic_agent event data  (elastic#21994)
  Document auditbeat system process module config (elastic#21766)
  Update links (elastic#22012)
  dynamically find librpm (elastic#21936)
  Fix Istio docs (elastic#22019)
  [beats-tester][packaging] store packages in another location (elastic#21903)
  [Kubernetes] Remove redundant dockersock volume mount (elastic#22009)
  [Ingest Manager] Always try snapshot repo for agent upgrade (elastic#21951)
  Azure storage metricset values not inside the metricset name (elastic#21845)
  fix diskio and memory bugs under windows (elastic#21992)
  Fix TestDockerStart flaky test (elastic#21681)
  filebeat: add SSL options to checkpoint module (elastic#19560)
  Stop storing stateless kubernetes keystores (elastic#21880)
  [Elastic Agent] Fix named pipe communication on Windows 7 (elastic#21931)
  [Elastic Agent] Fix index for Agent monitoring to to elastic_agent. (elastic#21932)
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Oct 22, 2020
* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Oct 22, 2020
* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
mdelapenya added a commit that referenced this pull request Oct 22, 2020
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100)

* feat: add a new step to run the e2e tests for certain parts of Beats

We are going to trigger the tests for those parts affected by the
elastic-agent, filebeat, or metricbeat, because those are the ones we
verify  in the e2e-testing suite

* chore: do not include heartbeat

* feat: trigger the e2e tests

* fix: use relative path

* chore: use proper target branch name for PRs

* chore: use different tag

* fix: use proper env variable

* chore: pass github checks context to downstream job

* chore: revert shared lib version

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: add BASE_DIR env variable

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: remove duplicated env

* ffix: add param comma separator

* fix: wrong copy&paste

* chore: move e2e GH check out of the release context

* chore: simplify conditional logic

* chore: refine execution of test suites

* fix: use proper parameter name

* chore: set metricbeat version

* chore: remove slack notifications on PRs

* chore: update parameter

* chore: run multiple test suites per beat type

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [E2E Tests] fix: set versions ony for PRs (#21608)

* fix: set versions ony for PRs

We want to use default versions per branch when running after a merge

* fix: add trailing comma

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [CI: Packaging] fix: push ubi8 images too (#21621)

* fix: push ubi8 images too

* chore: enhance retries

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: use variables in log

* chore: add "-oss" images

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* fix: remove extra curly brace in script (#21692)

* fix: remove extra curly brace

* chore: proper indent

* fix: update fleet test suite name (#21738)

* chore: create CI artifacts for DEV usage (#21645)

It will create the artifacts with some requirements related to integrity

* chore: simplify triggering the E2E tests for Beats (#21790)

* chore: pass beat as a method argument (no side-effects)

* chore: run tests in a separate stage

* fix: use parenthesis

* chore: update comment

* chore: do not trigger E2E tests if no suite was added

* fix: use missing curly brackets

* fix: wrong closure wrapping

* fix: condition was not set

* chore: delegate variant pushes to the right method (#21861)

* fix: delegate pushes to variants

* chore: group conditions for x-pack

* chore: simplify with endsWith

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* feat: package aliases for snapshots (#21960)

* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* chore: Use third number as x

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
mdelapenya added a commit that referenced this pull request Oct 22, 2020
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100)

* feat: add a new step to run the e2e tests for certain parts of Beats

We are going to trigger the tests for those parts affected by the
elastic-agent, filebeat, or metricbeat, because those are the ones we
verify  in the e2e-testing suite

* chore: do not include heartbeat

* feat: trigger the e2e tests

* fix: use relative path

* chore: use proper target branch name for PRs

* chore: use different tag

* fix: use proper env variable

* chore: pass github checks context to downstream job

* chore: revert shared lib version

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: add BASE_DIR env variable

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: remove duplicated env

* ffix: add param comma separator

* fix: wrong copy&paste

* chore: move e2e GH check out of the release context

* chore: simplify conditional logic

* chore: refine execution of test suites

* fix: use proper parameter name

* chore: set metricbeat version

* chore: remove slack notifications on PRs

* chore: update parameter

* chore: run multiple test suites per beat type

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [E2E Tests] fix: set versions ony for PRs (#21608)

* fix: set versions ony for PRs

We want to use default versions per branch when running after a merge

* fix: add trailing comma

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [CI: Packaging] fix: push ubi8 images too (#21621)

* fix: push ubi8 images too

* chore: enhance retries

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: use variables in log

* chore: add "-oss" images

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* fix: remove extra curly brace in script (#21692)

* fix: remove extra curly brace

* chore: proper indent

* fix: update fleet test suite name (#21738)

* chore: create CI artifacts for DEV usage (#21645)

It will create the artifacts with some requirements related to integrity

* chore: simplify triggering the E2E tests for Beats (#21790)

* chore: pass beat as a method argument (no side-effects)

* chore: run tests in a separate stage

* fix: use parenthesis

* chore: update comment

* chore: do not trigger E2E tests if no suite was added

* fix: use missing curly brackets

* fix: wrong closure wrapping

* fix: condition was not set

* chore: delegate variant pushes to the right method (#21861)

* fix: delegate pushes to variants

* chore: group conditions for x-pack

* chore: simplify with endsWith

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
# Conflicts:
#	.ci/packaging.groovy

* feat: package aliases for snapshots (#21960)

* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* chore: Use third number as x

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
v1v added a commit to v1v/beats that referenced this pull request Oct 22, 2020
* upstream/master: (27 commits)
  chore: use ubuntu 18 as linux agent (elastic#22084)
  docs: Prepare Changelog for 7.9.3 (elastic#22073) (elastic#22075)
  docs: Prepare Changelog for 6.8.13 (elastic#22072) (elastic#22079)
  [build][packaging] Add resilience when docker build (elastic#22050)
  Fix the url of reviewdog (elastic#21981)
  revert WSS process reporting for windows (elastic#22055)
  Fix typo (elastic#19585) (elastic#22061)
  [Ingest Manager] Use ML_SYSTEM to detect if agent is running as a service (elastic#21884)
  Prevent log input from sending duplicate messages due to file renaming (elastic#21911)
  feat: package aliases for snapshots (elastic#21960)
  [DOC] Add firewall as possible troubleshooting issue (elastic#21743)
  [Filebeat] Add max_number_of_messages config parameter for S3 input (elastic#21993)
  [Elastic Agent] Fix missing elastic_agent event data  (elastic#21994)
  Document auditbeat system process module config (elastic#21766)
  Update links (elastic#22012)
  dynamically find librpm (elastic#21936)
  Fix Istio docs (elastic#22019)
  [beats-tester][packaging] store packages in another location (elastic#21903)
  [Kubernetes] Remove redundant dockersock volume mount (elastic#22009)
  [Ingest Manager] Always try snapshot repo for agent upgrade (elastic#21951)
  ...
jsoriano pushed a commit to jsoriano/beats that referenced this pull request Dec 15, 2020
* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100)

* feat: add a new step to run the e2e tests for certain parts of Beats

We are going to trigger the tests for those parts affected by the
elastic-agent, filebeat, or metricbeat, because those are the ones we
verify  in the e2e-testing suite

* chore: do not include heartbeat

* feat: trigger the e2e tests

* fix: use relative path

* chore: use proper target branch name for PRs

* chore: use different tag

* fix: use proper env variable

* chore: pass github checks context to downstream job

* chore: revert shared lib version

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: add BASE_DIR env variable

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: remove duplicated env

* ffix: add param comma separator

* fix: wrong copy&paste

* chore: move e2e GH check out of the release context

* chore: simplify conditional logic

* chore: refine execution of test suites

* fix: use proper parameter name

* chore: set metricbeat version

* chore: remove slack notifications on PRs

* chore: update parameter

* chore: run multiple test suites per beat type

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [E2E Tests] fix: set versions ony for PRs (elastic#21608)

* fix: set versions ony for PRs

We want to use default versions per branch when running after a merge

* fix: add trailing comma

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [CI: Packaging] fix: push ubi8 images too (elastic#21621)

* fix: push ubi8 images too

* chore: enhance retries

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: use variables in log

* chore: add "-oss" images

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* fix: remove extra curly brace in script (elastic#21692)

* fix: remove extra curly brace

* chore: proper indent

* fix: update fleet test suite name (elastic#21738)

* chore: create CI artifacts for DEV usage (elastic#21645)

It will create the artifacts with some requirements related to integrity

* chore: simplify triggering the E2E tests for Beats (elastic#21790)

* chore: pass beat as a method argument (no side-effects)

* chore: run tests in a separate stage

* fix: use parenthesis

* chore: update comment

* chore: do not trigger E2E tests if no suite was added

* fix: use missing curly brackets

* fix: wrong closure wrapping

* fix: condition was not set

* chore: delegate variant pushes to the right method (elastic#21861)

* fix: delegate pushes to variants

* chore: group conditions for x-pack

* chore: simplify with endsWith

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* feat: package aliases for snapshots (elastic#21960)

* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* chore: Use third number as x

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
(cherry picked from commit 624c459)
jsoriano added a commit that referenced this pull request Jan 7, 2021
…23142)

Backports the following commits to 7.x:
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100)
* [E2E Tests] fix: set versions ony for PRs (#21608)
* [CI: Packaging] fix: push ubi8 images too (#21621)
* fix: remove extra curly brace in script (#21692)
* fix: update fleet test suite name (#21738)
* chore: create CI artifacts for DEV usage (#21645)
* chore: simplify triggering the E2E tests for Beats (#21790)
* chore: delegate variant pushes to the right method (#21861)
* feat: package aliases for snapshots (#21960)
* fix: use proper param name for e2e tests (#22836)

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
jsoriano added a commit to jsoriano/beats that referenced this pull request Jan 7, 2021
…ports (elastic#23142)

Backports the following commits to 7.x:
* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100)
* [E2E Tests] fix: set versions ony for PRs (elastic#21608)
* [CI: Packaging] fix: push ubi8 images too (elastic#21621)
* fix: remove extra curly brace in script (elastic#21692)
* fix: update fleet test suite name (elastic#21738)
* chore: create CI artifacts for DEV usage (elastic#21645)
* chore: simplify triggering the E2E tests for Beats (elastic#21790)
* chore: delegate variant pushes to the right method (elastic#21861)
* feat: package aliases for snapshots (elastic#21960)
* fix: use proper param name for e2e tests (elastic#22836)

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
(cherry picked from commit 1006bd9)
jsoriano added a commit that referenced this pull request Jan 15, 2021
…23142) (#23396)

Backports the following commits to 7.x:
* feat: add a new step to run the e2e tests for certain parts of Beats (#21100)
* [E2E Tests] fix: set versions ony for PRs (#21608)
* [CI: Packaging] fix: push ubi8 images too (#21621)
* fix: remove extra curly brace in script (#21692)
* fix: update fleet test suite name (#21738)
* chore: create CI artifacts for DEV usage (#21645)
* chore: simplify triggering the E2E tests for Beats (#21790)
* chore: delegate variant pushes to the right method (#21861)
* feat: package aliases for snapshots (#21960)
* fix: use proper param name for e2e tests (#22836)

(cherry picked from commit 1006bd9)

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100)

* feat: add a new step to run the e2e tests for certain parts of Beats

We are going to trigger the tests for those parts affected by the
elastic-agent, filebeat, or metricbeat, because those are the ones we
verify  in the e2e-testing suite

* chore: do not include heartbeat

* feat: trigger the e2e tests

* fix: use relative path

* chore: use proper target branch name for PRs

* chore: use different tag

* fix: use proper env variable

* chore: pass github checks context to downstream job

* chore: revert shared lib version

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: add BASE_DIR env variable

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: remove duplicated env

* ffix: add param comma separator

* fix: wrong copy&paste

* chore: move e2e GH check out of the release context

* chore: simplify conditional logic

* chore: refine execution of test suites

* fix: use proper parameter name

* chore: set metricbeat version

* chore: remove slack notifications on PRs

* chore: update parameter

* chore: run multiple test suites per beat type

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [E2E Tests] fix: set versions ony for PRs (elastic#21608)

* fix: set versions ony for PRs

We want to use default versions per branch when running after a merge

* fix: add trailing comma

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* [CI: Packaging] fix: push ubi8 images too (elastic#21621)

* fix: push ubi8 images too

* chore: enhance retries

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* chore: use variables in log

* chore: add "-oss" images

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

* fix: remove extra curly brace in script (elastic#21692)

* fix: remove extra curly brace

* chore: proper indent

* fix: update fleet test suite name (elastic#21738)

* chore: create CI artifacts for DEV usage (elastic#21645)

It will create the artifacts with some requirements related to integrity

* chore: simplify triggering the E2E tests for Beats (elastic#21790)

* chore: pass beat as a method argument (no side-effects)

* chore: run tests in a separate stage

* fix: use parenthesis

* chore: update comment

* chore: do not trigger E2E tests if no suite was added

* fix: use missing curly brackets

* fix: wrong closure wrapping

* fix: condition was not set

* chore: delegate variant pushes to the right method (elastic#21861)

* fix: delegate pushes to variants

* chore: group conditions for x-pack

* chore: simplify with endsWith

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
# Conflicts:
#	.ci/packaging.groovy

* feat: package aliases for snapshots (elastic#21960)

* feat: push aliases for docker images

* feat: build alias for snapshots

* fix: only update alias on snapshots

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* fix: wrong image name for alias

* fix: reuse variable as groovy does not hide variables by scope

* chore: extract common logic to a method

* Revert "fix: only update alias on snapshots"

This reverts commit cff2cef.

* Revert "feat: build alias for snapshots"

This reverts commit 707e0d7.

* chore: do not push aliases for PRs

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* chore: Use third number as x

Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci needs_backport PR is waiting to be backported to other branches. Team:Automation Label for the Observability productivity team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants