Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix readiness probe #2272

Merged
merged 7 commits into from
Dec 17, 2019
Merged

Fix readiness probe #2272

merged 7 commits into from
Dec 17, 2019

Conversation

barkbay
Copy link
Contributor

@barkbay barkbay commented Dec 16, 2019

Fix #2248

This PR updates the readiness probe by:

  • Calling the root of the web server /
  • Expecting a 200 response code unless Elasticsearch version is 6.x in which case 503 errors are tolerated

@barkbay barkbay added >bug Something isn't working v1.0.0 labels Dec 16, 2019
Copy link
Collaborator

@pebrc pebrc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I am 👍 One question about the probe failure reporting but nothing major. We are already mixing different log formats on stdout which whatever is ingesting these has to deal with.

pkg/controller/common/volume/downward_api.go Outdated Show resolved Hide resolved
# fail should be called as a last resort to help the user to understand why the probe failed
function fail {
timestamp=$(date --iso-8601=seconds)
echo "{\"timestamp\": \"${timestamp}\", \"message\": \"readiness probe failed\", "$1"}" | tee /proc/1/fd/1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's clever :-) But should it be /fd/2 for stderr?

Comment on lines 12 to 32
var downwardApiVolume = corev1.Volume{
Name: volume.DownwardApiVolumeName,
VolumeSource: corev1.VolumeSource{
DownwardAPI: &corev1.DownwardAPIVolumeSource{
Items: []corev1.DownwardAPIVolumeFile{
{
Path: volume.LabelsFile,
FieldRef: &corev1.ObjectFieldSelector{
FieldPath: "metadata.labels",
},
},
},
},
},
}

var downwardApiVolumeMount = corev1.VolumeMount{
Name: volume.DownwardApiVolumeName,
MountPath: volume.DownwardApiMountPath,
ReadOnly: true,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I feel like these don't need to be global variables (even though they're only package-private). Maybe they can be directly returned from the method calls of the DownwardApi object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here is to avoid to recreate what must be a constant everytime a Pod spec is created. I understand that it may sound like a very tiny optimization though 😐

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I don't feel strongly about it. Happy to leave it as is.

pkg/controller/common/volume/downward_api.go Outdated Show resolved Hide resolved
Copy link
Contributor

@charith-elastic charith-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

fi

# get Elasticsearch version from the downward API
version=$(grep "elasticsearch.k8s.elastic.co/version" ${labels} | cut -d '=' -f 2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could be safer to use the VersionLabelName constant:

VersionLabelName = "elasticsearch.k8s.elastic.co/version"

@barkbay barkbay merged commit 0c8ded9 into elastic:master Dec 17, 2019
mjmbischoff pushed a commit to mjmbischoff/cloud-on-k8s that referenced this pull request Jan 13, 2020
* Mount annotations in the Pod

* Redirect errors to stderr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug Something isn't working v1.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Elasticsearch readiness probe might fail if a single node is stuck
4 participants