2.6.1

Elastic Cloud on Kubernetes 2.6.1

• Release highlights
• Quickstart guide

Bug fixes

• Update minimum version to use Elasticsearch file-based settings feature #6305 (issue: #6303)

2.6.0

Elastic Cloud on Kubernetes v2.6.0

• Release highlights
• Quickstart guide

New features

• New CRD StackConfigPolicy to declaratively configure multiple Elasticsearch clusters. #6148
• ECK resources Helm Chart - Beats. #5899 (issue: #5505)

Enhancements

• Expose Kubernetes client QPS as a flag. #6157
• Extend existing reattach-pv tool to allow using existing PVs to create newly named cluster. #6118
• Add container-suffix operator flag to allow users to specify a container suffix to be applied across all Elastic stack container images. #6086 (issue: #6064)
• Elasticsearch observer improvements to avoid blocking between workers. #6084 (issue: #6078)
• Improve user password hash comparison performance by utilizing an LRU cache. #6080 (issue: #6076)
• Add default securityContext to the manager container in Operator Helm Chart. #6047
• Allow Fleet Server to be run without TLS. #6020 (issue: #6000)

Bug fixes

• Fix potential panic in Elasticsearch client equal function. #6128
• Increment ECK-stack Helm chart version to support addition of Agent/Fleet Server. #6179
• Try to reconcile license even in absence of known health status #6278 (issue: #6274)

Documentation improvements

• Add experimental label to the StackConfigPolicy doc. #6247
• Document Elastic Stack configuration policies. #6215
• Update eck-diagnostics documentation for filters. #6191
• Add additional Helm documentation for Fleet Server, and Agent. #6154
• Update the list of Kibana keys managed by the operator. #6119
• Document limitation on Minikube without CNI. #6075
• Add latest APM fleet package in Kibana examples when using standalone APM server. #6063 (issue: #5059)

Misc

• Update module github.com/hashicorp/golang-lru to v0.6.0 #6172
• Update module github.com/google/go-containerregistry to v0.12.1 #6168
• Update k8s to v0.25.4 #6167
• Update module helm.sh/helm/v3 to v3.10.2 #6166
• Update module golang.org/x/oauth2 to v0.2.0 #6159
• Update module golang.org/x/crypto to v0.2.0 #6158
• Update module golang.org/x/net to v0.2.0 #6155
• Update module github.com/prometheus/client_golang to v1.14.0 #6150
• Update module github.com/spf13/viper to v1.14.0 #6145
• Update module sigs.k8s.io/controller-runtime to v0.13.1 #6141
• Update module github.com/prometheus/client_golang to v1.13.1 #6136
• Update docker.io/library/golang Docker tag to v1.19.3 #6135
• Update module go.elastic.co/apm/module/apmzap/v2 to v2.2.0 #6131
• Update module go.elastic.co/apm/module/apmelasticsearch/v2 to v2.2.0 #6129
• Update module github.com/hashicorp/vault/api to v1.8.2 #6127
• Update module github.com/spf13/cobra to v1.6.1 #6110
• Update module golang.org/x/text to v0.4.0 #6100

2.5.0

Elastic Cloud on Kubernetes 2.5.0

• Release highlights
• Quickstart guide

New features

• Autoscaling Elasticsearch: Introduce a dedicated custom resource #5978 (issue: #5997)
• ECK resources Helm chart - Elastic Agent & Elastic Fleet Server Agent #5889 (issue: #5505)
• Enable Beats stack monitoring configuration #5878 (issue: #5563)

Enhancement

• Surface Kubernetes client rate limiter metrics #6007
• Add Elasticsearch observation interval as configurable value to Helm Chart #5989 (issue: #5988)
• Don’t log non-standard ES JSON error responses as errors #5971 (issue: #5473)
• Report incorrect license type in logs and events #5966 (issue: #5963)
• Inherit all environment variables from ES container in initContainers #5962 (issue: #5577)
• Elasticsearch: always set discovery.seed_hosts to empty array #5950 (issue: #5834)
• [Autoscaling] Add CPU recommender #5924 (issue: #5823)
• Log correlation for operator APM traces #5883

Bug fixes

• Increment desired nodes version on each call #6037 (issue: #5979)
• Ignore unmanaged namespaces in webhook validation for all resources. #6013 (issue: #5814)
• Fix helm chart rendering issues associated with indentation #6004
• Stack monitoring: trust custom cert. in output configuration #5945 (issue: #5917)

Documentation improvements

• Add License files for Helm Charts and Updating Chart README #6008 (issue: #6005)
• Rewrite snapshot documentation and add CSP specific setups #5969 (issues: #5230, #5652)
• Restructure secure settings docs and minor additions #5965 (issue: #5425)
• Update documentation to clarify ES node.processors section. #5941 (issue: #5940)
• Fix typo in "manage compute resources" doc #5929

Misc

• Update module go to 1.19 #6040
• Update k8s to v0.25.2 #6032
• Update module sigs.k8s.io/controller-tools to v0.10.0 #6031
• Update module helm.sh/helm/v3 to v3.10.0 #6030
• Update module github.com/hashicorp/vault/api to v1.8.0 #6022
• Update k8s to v0.25.1 #6018
• Update registry.access.redhat.com/ubi8/ubi-minimal Docker tag to v8.6-941 #6012
• Update module k8s.io/klog/v2 to v2.80.1 #6009
• Update module github.com/google/go-cmp to v0.5.9 #6006
• Update docker.io/library/golang Docker tag to v1.19.1 #6003
• Update module github.com/spf13/viper to v1.13.0 #6001
• Update module github.com/gobuffalo/flect to v0.3.0 #5996
• Update module sigs.k8s.io/controller-runtime to v0.13.0 #5995
• Update module go.uber.org/zap to v1.23.0 #5972
• Update golang to 1.19 #5939
• Update module github.com/prometheus/client_golang to v1.13.0 #5930
• Update module sigs.k8s.io/kustomize/kyaml to v0.13.9 #5918
• Update dependency registry.access.redhat.com/ubi8/ubi-minimal to v8.6-902 #5914
• Update module github.com/stretchr/testify to v1.8.0 #5912
• Update module github.com/prometheus/common to v0.37.0 #5911
• Update module github.com/google/go-containerregistry to v0.11.0 #5910

2.4.0

Elastic Cloud on Kubernetes 2.4.0

• Release highlights
• Quickstart guide

Breaking changes

• Configure Elastic Agent host path volume to point to correct path #5890 (issue: #4428)

  Fleet-managed Elastic Agents now default to use a hostPath volume for storing their state. This will prevent more than one Pod from the same Elastic Agent Deployment to be deployed on the same Kubernetes node. For cases where this is desired, the volume type can be changed to an emptyDir volume. Check the docs to learn more.

New features

• Introduce ECK-managed resources Helm Charts #5781 (issue: #5505)

Enhancements

• Add new operator flag to control Elasticsearch health observation intervals #5861 (issue: #5839)
• Make xpack.security.http.ssl.client_authentication an unsupported setting #5852 (issue: #5817)
• Use static transaction names for APM #5850 (issue: #5840)
• Create Elastic Agent enrolment tokens in the operator #5846 (issue: #5779)
• Support RevisionHistoryLimit for all ECK-managed resources #5818 (issue: #5789)
• Stricter notion of esReacheable: require health response #5796 (issue: #5776)
• Increase default Beats guaranteed memory to 300Mi #5793 (issue: #5036)

Bug fixes

• Move first ES cluster state observation out of go routine #5783 (issue: #5812)
• Check shard activity before removing a node #5758 (issues: #3070, #5713)

Documentation improvements

• Remove experimental label from Elastic Agent docs #5894
• Improve "Operator crashes on startup with OOMKilled" docs section #5836
• Expose recipes in ECK product documentation #5763 (issue: #5012)
• Fix minimum Helm supported version 3.2.0 in README #5753

Misc

• Update dependency docker.io/library/golang to v1.18.5 #5907
• Update k8s to v0.24.3 #5904
• Update module sigs.k8s.io/kustomize/kyaml to v0.13.8 #5900
• Update module helm.sh/helm/v3 to v3.9.2 #5876
• Update dependency golang to v1.18.4 #5873
• Update dependency registry.access.redhat.com/ubi8/ubi-minimal to v8.6-854 #5855
• Update module sigs.k8s.io/controller-tools to v0.9.1 #5842
• Update module github.com/elastic/go-ucfg to v0.8.6 #5841
• Update module sigs.k8s.io/controller-runtime to v0.12.2 #5828
• Update module github.com/google/go-containerregistry to v0.10.0 #5821
• Update module k8s.io/klog/v2 to v2.70.0 #5819
• Update module github.com/spf13/cobra to v1.5.0 #5811
• Update module github.com/prometheus/common to v0.35.0 #5808
• Update module github.com/stretchr/testify to v1.7.3 #5807
• Update module github.com/hashicorp/vault/api to v1.7.2 #5761

2.3.0

Elastic Cloud on Kubernetes 2.3.0

• Release highlights
• Quickstart guide

New features

• Allow providing cleartext passwords for creating Elasticsearch users #5613 (issue: #3056)
• Support a globally shared CA #5539

Enhancements

• Set status.ObservedGeneration from metadata.Generation:
  • Elastic Maps #5536 (issue: #3392)
  • EnterpriseSearch #5526 (issue: #3392)
  • Elastic APM Server #5470 (issue: #3392)
• Upgrade PodDisruptionBudget from v1beta1 to v1 #5709
• Support disable-downgrade-validation for all relevant apps #5680 (issue: #5531)
• Allow non-IPs in service spec to avoid noop updates #5663 (issue: #5657)
• Add APM tracing for client-go requests to the Kubernetes API #5651
• Add support for the desired nodes API #5650
• Base ECK docker image on distroless instead of UBI by default #5580 (issue: #4561)
• Added priority class and leader election to operator Helm chart #5538
• Log info for service not found error when reconciling associations #5533

Bug fixes

• Do not use service accounts until Elasticsearch nodes have been upgraded #5830 (issue: #5684)
• Ensure CA is always updated in HTTP Secret #5622 (issue: #5621)
• Fix resources limits conversion in ToInt64() used for logging #5596
• Fix non-closed http responses #5755

Documentation improvements

• a11y Fix "below" occurrences #5714 (issue: #5306)
• a11y Fix "above" occurrences #5672 (issue: #5306)
• Change references to the master branch to main in the CONTRIBUTING guide. #5741
• Fix Helm command examples in docs #5737
• Update list of ECK versions that triggers a rolling restart #5715 (issue: #5648)
• Update documentation pages that use the repository-gcs plugin #5700 (issue: #5457)
• CronJob batch/v1beta1 no longer served in 1.25 #5685
• Update documentation to customize pods #5660
• Wrong indentation of the kibana config #5595
• Update recommended reading Kubebuilder links #5593 (issue: #5584)
• Add APM Server Deprecation Message #5575 (issue: #5419)
• Update license usage data example #5569

Misc

• Update dependency golang to v1.18.3 #5722
• Update k8s to v0.24.1 #5703
• Update module sigs.k8s.io/controller-runtime to v0.12.1 #5697
• Update module sigs.k8s.io/controller-tools to v0.9.0 #5688
• Update module sigs.k8s.io/kustomize/kyaml to v0.13.7 #5682
• Update module github.com/elastic/go-ucfg to v0.8.5 #5661
• Update module github.com/google/go-cmp to v0.5.8 #5619
• Update module github.com/google/go-containerregistry to v0.9.0 #5675
• Update module github.com/hashicorp/vault/api to v1.6.0 #5702
• Update module github.com/imdario/mergo to v0.3.13 #5701
• Update module github.com/jonboulle/clockwork to v0.3.0 #5606
• Update module github.com/prometheus/client_golang to v1.12.2 #5667
• Update module github.com/prometheus/common to v0.34.0 #5594
• Update module github.com/spf13/viper to v1.12.0 #5704
• Update module go.elastic.co/apm/module/apmelasticsearch/v2 to v2.1.0 #5693
• Update module go.uber.org/automaxprocs to v1.5.1 #5560
• Update module gopkg.in/yaml.v3 to v3.0.1 #5706
• Update module helm.sh/helm/v3 to v3.9.0 #5678
• Update dependency registry.access.redhat.com/ubi8/ubi-minimal to v8.6 #5654

2.2.0

Elastic Cloud on Kubernetes 2.2.0

• Release highlights
• Quickstart guide

Enhancements

• Report total managed memory in GiB and raw bytes #5527 (issue: #5465)
• Use service accounts for Kibana and Fleet Server #5468 (issue: #5244)
• Support custom Secret for associating external Elastic resources not managed by ECK #5240 (issue: #5078)

Bug fixes

• Update operator Pod to speed up secret propagation #5519 (issue: #3321)
• Reset phase on each reconciliation in Elasticsearch status #5507 (issue: #5506)
• Make nodes field in status optional #5496 (issue: #5493)
• Make sure to read association configuration again from annotations if it was cleared #5489 (issue: #4709)
• Avoid unnecessary DELETE for non-existent secrets #5488 (issue: #5450)
• Avoid unnecessary DELETE calls to manage legacy transport secret #5461 (issue: #5450)
• Avoid unnecessary UPDATE calls when reconciling PVC owner refs #5459 (issue: #5451)
• Do not upgrade all Elasticsearch nodes of a given tier at once #5452 (issue: #1753)
• Operatorhub: Ensure local YAML files have a proper "end of directives" marker #5447

Documentation improvements

• Update license usage data example #5569
• Fix YAML example in custom HTTP certificate doc #5529
• Update the license documentation #5509 (issue: #5475)
• Remove ECK 1.7 from the list of versions that cause a restart on upgrade #5503
• Add known issue for Red Hat certified operator upgrades #5492
• Update example with init container to increase vm.max_map_count #5469 (issue: #5410)
• Fix latinisms occurrences #5456 (issue: #5306)
• Add explicit assumptions before the installation steps #5455 (issue: #5275)

Misc

• Update module go.uber.org/automaxprocs to v1.5.0 #5552
• Update module github.com/gobuffalo/flect to v0.2.5 #5550
• Update module sigs.k8s.io/controller-runtime to v0.11.2 #5541
• Update module github.com/prometheus/common to v0.33.0 #5530
• Update module github.com/hashicorp/vault/api to v1.5.0 #5522
• Update module sigs.k8s.io/kustomize/kyaml to v0.13.6 #5521
• Update module github.com/prometheus/client_golang to v1.12.1 #5517
• Update module github.com/jonboulle/clockwork to v0.2.3 #5516
• Update module k8s.io/klog/v2 to v2.60.1 #5502
• Update module github.com/go-logr/logr to v1.2.3 #5494
• Update k8s to v0.23.5 #5491
• Update module github.com/stretchr/testify to v1.7.1 #5476
• Update module github.com/spf13/cobra to v1.4.0 #5463
• Update module helm.sh/helm/v3 to v3.8.1 #5454

2.1.0

Elastic Cloud on Kubernetes 2.1.0

• Release highlights
• Quickstart guide

New features

• Allow predicates to be disabled on a case-by-case basis through annotation. #5284 (issue: #2092)

Enhancements

• Elasticsearch: Set status.ObservedGeneration from metadata.Generation #5331 (issue: #3392)
• Kibana: Set status.ObservedGeneration from metadata.Generation #5409 (issue: #3392)
• Extend full upgrade to any version upgrade of non-HA Elasticsearch #5408
• Handle resource conflict while updating status in association reconciler #5337
• Improve Elasticsearch status sub-resource #5328
• Use new node.roles notation in all example manifests #5289 (issue: #4130)
• Handle data tiers during rolling upgrades #5248 (issue: #5228)
• Isolate operator from HTTP service misconfiguration - Use internal service #5211 (issue: #4394)
• Improve handling of managed namespaces - resolving 'unknown namespace for the cache' errors #5187

Bug fixes

• Avoid reporting outdated Elasticsearch health on reconciliation error that prevents getting the real one #5349 (issue: #5330)
• Only configure Stack Monitoring if association reconciled #5339
• Do not attempt rolling upgrades for non-HA Elasticsearch clusters #5327 (issue: #5321)
• Use precondition when deleting secrets #5273 (issue: #5249)
• Support new Agent base image as of 7.17 #5268
• Fix webhook match policy for OLM based installations #5437 (issue: #5423)
• Fix Agent trust CA commands for all image variants #5438 (issue: #5434)

Documentation improvements

• Add a sentence explaining the upgrade strategy restriction for non-HA Elasticsearch clusters #5400
• Add example code to the Quickstart #5378 (issue: #5322)
• Fix links to Elasticsearch upgrade docs #5347
• Adjust Fleet recipes for default policy change #5281 (issue: #5262)

Misc

• Update golang version to v1.17.8 #5440
• Update module sigs.k8s.io/controller-runtime to v0.11.1 #5384
• Update module sigs.k8s.io/kustomize/kyaml to v0.13.3 #5380
• Update module github.com/google/go-cmp to v0.5.7 #5264

2.0.0

Elastic Cloud on Kubernetes 2.0.0

• Release highlights
• Quickstart guide

Breaking changes

---

IMPORTANT
Operator Lifecycle Manager (OLM) and OpenShift OperatorHub users that run with automatic upgrades enabled, are advised to set the set-default-security-context operator flag explicitly before upgrading to ECK 2.0. If not set ECK can fail to auto-detect the correct security context configuration and Elasticsearch Pods may not be allowed to run.

---

• Remove legacy v1beta1 CRDs #5029 (issue: #4885)

New features

• Allow K8S node labels to be propagated as Pod annotations #5054 (issue: #3933)

Enhancements

• Allow users to disable version validation to downgrade clusters #5272 (issue: #5259)
• Add support for transport TLS certificate other/common name suffix #5189 (issue: #5148)
• Bump the operator memory limit to 1Gi for larger deployments #5185 (issue: #5177)
• Move config hash stored in Pod labels to Pod annotations #5175 (issue: #5071)
• Elastic Maps Server: no Elasticsearch connection required since 7.14 #5172
• Add autodetect option to security context configuration #5150 (issue: #5061)
• Stop using deprecated host field in Beat configs #5113 (issue: #4954)
• Stop using deprecated xpack.fleet.agents.elasticsearch.host Kibana config #5112 (issue: #5091)
• Set Enterprise Search config kibana.host defaults in advance of 8.0 #5109 (issue: #4952)
• Support trial licenses inside enterprise_trial orchestration licenses #5097
• Include license expiry date in licensing config map #5013 (issue: #5008)
• Reuse existing private key for CA rotation #4947 (issue: #507)
• Use node shutdown API for rolling upgrades and downscales #4597

Bug fixes

• Fix Stack Monitoring with custom certificate without CA #5310 (issue: #5309)
• Enterprise Search: avoid generating invalid config in the presence of user overrides #5298 (issue: #5290)
• Support new Agent base image as of 7.17 #5268
• Change upgrade path validation for 8.0 to only allow 7.17 #5261 (issue: #5258)
• Adjust Agent startup command to Ubuntu base image #5253 (issue: #5250)
• Do not delete last master-eligible node if other nodes are not up-to-date #5242 (issue: #5241)
• Add missing nodes resource RBAC permission in recipes #5178
• Main go routine does not exit if there's no error #5106

Documentation improvements

• Document node labels propagation feature #5254 (issue: #5098)
• Update Kubernetes/OCP compatibility statements for ECK 2.0 #5239

Misc

• Update golang Docker tag to v1.17.6 #5234
• Update module sigs.k8s.io/controller-tools to v0.8.0 #5233
• Update module go.uber.org/zap to v1.20.0 #5224
• Update module github.com/go-logr/logr to v1.2.2 #5220
• Update k8s to v0.23.1 #5219
• Update module github.com/hashicorp/vault/api to v1.3.1 #5208
• Update module k8s.io/klog/v2 to v2.40.1 #5184
• Update module github.com/spf13/viper to v1.10.1 #5171
• Update module github.com/spf13/cobra to v1.3.0 #5163
• Update module sigs.k8s.io/controller-runtime to v0.11.0 #5161
• Update module github.com/elastic/go-ucfg to v0.8.4 #5037
• Update registry.access.redhat.com/ubi8/ubi-minimal Docker tag to v8.5 #5034
• Update golang Docker tag to v1.17.3 #5018
• Update module sigs.k8s.io/kustomize/kyaml to v0.12.0 #5004
• Update module go.elastic.co/apm/module/apmelasticsearch to v1.14.0 #5002
• Update module go.elastic.co/apm to v1.14.0 #5001

Elastic Cloud on Kubernetes 1.9.1

Elastic Cloud on Kubernetes 1.9.1

• Release highlights
• Quickstart guide

Enhancements

• Add log4j2.formatMsgNoLookups logging property to ES < 7.2 #5157

This change will mitigate the Log4Shell vulnerability (CVE-2021-44228) in susceptible Elasticsearch clusters (below version 7.2). ECK will prepend the -Dlog4j2.formatMsgNoLookups=true JVM parameter to the environment variable ES_JAVA_OPTS if it is not yet defined by the user. This triggers a rolling restart of all Pods of the affected Elasticsearch clusters to apply these changes.

• Add internal product header to requests #5129

Bug fixes

• Resource aggregator should handle missing memory settings #5158

Misc

• Update golang Docker tag to v1.17.5 #5149

Elastic Cloud On Kubernetes 1.9.0

Elastic Cloud on Kubernetes 1.9.0

• Release highlights
• Quickstart guide

Breaking changes

---

IMPORTANT

For Operator Lifecycle Manager (OLM) and OpenShift OperatorHub users that run with automatic upgrades enabled, it is advised to set set-default-security-context operator flag explicitly before next ECK release. Next version of ECK will introduce a change that could result in setting an invalid Pod Security Context to Elasticsearch Pods.

---

• Inherit resource requirements in init containers #4963

All init containers will inherit resource requirements from the main containers by default.

• Move to env vars based Fleet mode configuration #4889

ECK operator will configure Elastic Agent in Fleet mode using environment variables instead of a configuration file. This method allows you to easily override the default settings. If the override is configured with any other method, an additional step is required before the operator upgrade.

1. Update your Agent resources such that environment variables are set for settings you'd like to override, for example:

...
spec:
  deployment:
    podTemplate:
      spec:
        containers:
        - name: agent
          env:
          - name: KIBANA_FLEET_HOST
            value: "https://kibana.example.com:443"
...

2. Upgrade ECK to version 1.9.x.
3. Remove any modification and references to fleet-setup.yml. As ECK 1.9.x uses environment variables to set the default config, the environment variables you've set in step 1. override the default values.

New features

• Allow users to suspend Elasticsearch Pods for debugging purposes #4946 (issue: #4546)

Enhancements

• Add logging for call to ES #4958 (issue: #4935)
• Shorten the reconciliation loop duration if Elasticsearch is down #4938 (issues: #2939, #3496)
• Ignore ClusterIPs, and IPFamilyPolicy in services, and timestamp in license during reconciliation #4929
• Add operator license key check #4925
• Skip remote clusters reconciliation earlier if remote clusters are not configured #4924
• Refactor Elasticsearch HTTP error handling #4922 (issue: #4918)
• Verify supported Elasticsearch distribution during license reconciliation #4920
• Remove permissions not used by the operator #4823

Bug fixes

• Avoid updating the association status when no association #4986 (issue: #4985)
• Forcibly recreate Beat keystore #4942 (issues: #4527, #4926)
• Init. fs container: make cp more resilient #4888 (issue: #4877)
• Allow the upgrade of a single node cluster with yellow status #4787 (issue: #4625)

Misc

• Update module github.com/go-test/deep to v1.0.8 #4948
• Update golang Docker tag to v1.17.2 #4934
• Update module github.com/spf13/viper to v1.9.0 #4847