Allow users to specify custom repo when specifying CONTAINER_REPOSITORY for Elastic Operator

[renzedj]

Use Case

As my company's Elastic Administrator, I need to be able to specify a custom respository within my local CONTAINER_REGISTRY.

Details

Elastic Operator version: 2.6.1

I am required to use my company's local registry to pull images from in our environment. Because of conventions, I am also required to use a specific naming schema for images within the registry:

registry.example.com/team/elastic-agent:8.7.0

When deploying the Elastic Operator, I am able to specify a default container registry, e.g.:

CONTAINER_REGISTRY=registry.example.com/team

The issue is that Elastic Operator appears to have the image naming schema hardcoded. E.g., when I deploy Elastic Agent with my custom registry and repository, Elastic Operator attempts to pull:

registry.example.com/team/beats/elastic-agent:8.7.0

As a workaround, I can specify the image in my deployment:

image: 'registry.example.com/team/elastic-agent:8.7.0'

However, this partially defeats the purpose of using the Elastic Operator, which is to allow Elastic Operator to manage the image source.

Solution

Either require the CONTAINER_REGISTRY to contain both registry and repository (registry.example.com/team in this example) or add another configuration option CONTAINER_REPOSITORY, which would default to the current value (beats in this example), which would mirror current behavior when pulling an image from the default registry (docker.elastic.co).

Since requiring CONTAINER_REGISTRY to contain the full registry and repository name would likely be a breaking change for users who pull their images directly from Elastic, the latter option would be better.

[thbkrkr]

Could you use registry.example.com/team/beats/elastic-agent:8.7.0 instead?

So, you would start the operator with CONTAINER_REGISTRY=registry.example.com/team and move the Elastic images:

• from docker.elastic.co/<elastic::repository>/<elastic::name>
• to registry.example.com/team/<elastic::repository>/<elastic::name>

Note: here are the image names to follow:

cloud-on-k8s/pkg/controller/common/container/container.go

Lines 31 to 42 in 4f9fa13

	APMServerImage Image = "apm/apm-server"
	ElasticsearchImage Image = "elasticsearch/elasticsearch"
	KibanaImage Image = "kibana/kibana"
	EnterpriseSearchImage Image = "enterprise-search/enterprise-search"
	FilebeatImage Image = "beats/filebeat"
	MetricbeatImage Image = "beats/metricbeat"
	HeartbeatImage Image = "beats/heartbeat"
	AuditbeatImage Image = "beats/auditbeat"
	JournalbeatImage Image = "beats/journalbeat"
	PacketbeatImage Image = "beats/packetbeat"
	AgentImage Image = "beats/elastic-agent"
	MapsImage Image = "elastic-maps-service/elastic-maps-server-ubi8"

[renzedj]

Could you use registry.example.com/team/beats/elastic-agent:8.7.0 instead?

That was my first thought as well, but I'm not able to do that with the way our registry is configured.

I can also see the use case where a user may be restricted to pulling their images through Docker Hub, which has all Elastic's images in the elastic repo (e.g., elastic/elastic-agent, elastic/elasticsearch, etc.).

[carinadigital]

I can also see the use case where a user may be restricted to pulling their images through Docker Hub, which has all Elastic's images in the elastic repo (e.g., elastic/elastic-agent, elastic/elasticsearch, etc.).

I have this exact problem. We only have dockerhub mirrored and the paths to the container images are different.
elasticsearch/elasticsearch vs elastic/elasticsearch

Without the ability to change the image name, we can't use the operator.

[renzedj]

Without the ability to change the image name, we can't use the operator.

See my original post. There is a workaround for this so that you can use the operator anyway. It works, but it's not a good solution because you end up having to define the agent version in multiple locations within your deployment, so you risk not upgrading it when you intend to.

[carinadigital]

Custom images are documented on the website, but not anywhere in the repo READMEs or example charts . I have linked it here for reference.
https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-custom-images.html

I this means no fix is needed as there is a way to override the images in the spec of the ES resource.

[renzedj]

I this means no fix is needed as there is a way to override the images in the spec of the ES resource.

I disagree, as including the image is an antipattern:

• Elastic Operator manages the image source: As an admin if I need to change my image source (e.g., from the Elastic repo to a Docker repo to an on-prem repo), I just need to change that in the operator and redeploy, rather than in multiple charts or Kubernetes manifests; the operator then manages the image pull.
• Increased risk: As an admin, if I specify the image in my chart/manifest, I have to validate that the image source and/or version has been updated in multiple locations, which can lead to a version conflict or an image that can't be pulled.

I can certainly manage this using the workaround for as long as I need to, but this is something that Elastic Operator should handle. I've looked at the code and it doesn't appear that it should be difficult to specify the repo, but I'm not sufficiently proficient with Go to fork the repo and make the update myself.

[thbkrkr]

We discussed and we are going to accept this enhancement and add a new flag --container-repository.

[renzedj]

We discussed and we are going to accept this enhancement and add a new flag --container-repository.

Thank you!