etcdhttp/metrics.go: exclude alarms from health check conditionally with ?exclude=NOSPACE

[chaochn47]

Overview

Exclude alarms from health check with ?exclude=NOSPACE

Similar to kubernetes excludes specific checks
https://kubernetes.io/docs/reference/using-api/health-checks/#api-endpoints-for-health

curl -k 'https://localhost:6443/readyz?verbose&exclude=etcd'

The motivation is to exclude NOSPACE alarms in etcd supervisor or peer to peer health checks.

Testing

Added unit test against serverV2 to verify the logic due to v3 *etcdserver.EtcdServer is hard to mock out.

Unit test output

go test -v -run TestHealthHandler -mod=readonly
=== RUN   TestHealthHandler
{"level":"debug","msg":"serving /health true"}
{"level":"debug","msg":"/health OK","status-code":200}
{"level":"warn","msg":"serving /health false due to an alarm","alarm":"alarm:NOSPACE "}
{"level":"warn","msg":"/health error","output":"{\"health\":\"false\",\"reason\":\"ALARM NOSPACE\"}","status-code":503}
{"level":"debug","msg":"/health excluded alarm","alarm":"NOSPACE"}
{"level":"debug","msg":"serving /health true"}
{"level":"debug","msg":"/health OK","status-code":200}
{"level":"warn","msg":"fail exclude alarms from health check","exclude alarms":"map[NOSPACE:{}]"}
{"level":"debug","msg":"serving /health true"}
{"level":"debug","msg":"/health OK","status-code":200}
{"level":"debug","msg":"/health excluded alarm","alarm":"NOSPACE"}
{"level":"warn","msg":"serving /health false due to an alarm","alarm":"memberID:1 alarm:CORRUPT "}
{"level":"warn","msg":"/health error","output":"{\"health\":\"false\",\"reason\":\"ALARM CORRUPT\"}","status-code":503}
{"level":"debug","msg":"/health excluded alarm","alarm":"NOSPACE"}
{"level":"debug","msg":"/health excluded alarm","alarm":"CORRUPT"}
{"level":"debug","msg":"serving /health true"}
{"level":"debug","msg":"/health OK","status-code":200}
--- PASS: TestHealthHandler (0.01s)
PASS
ok      go.etcd.io/etcd/server/v3/etcdserver/api/etcdhttp       0.120s

[chaochn47]

@gyuho PTAL, thanks!!

[gyuho]

lgtm