[blog] Security vulnerability in confighttp (CVE-2024-36129)

content/en/blog/2024/cve-2024-36129/index.md

@@ -0,0 +1,126 @@
+---
+title: Security vulnerability in confighttp (CVE-2024-36129)
+linkTitle: Security vulnerability in confighttp
+date: 2024-06-05
+author:
+  '[Juraci Paixão Kröhling](https://github.com/jpkrohling) (OpenTelemetry,
+  Grafana Labs), [Pablo Baeyens](https://github.com/mx-psi) (OpenTelemetry,
+  Datadog)'
+cSpell:ignore: Baeyens configgrpc confighttp Miroslav OSTIF Stampar zstd
+---
+
+On our path toward graduation, the OpenTelemetry project is currently undergoing
+a security audit sponsored by the [CNCF](https://www.cncf.io/), facilitated by
+[OSTIF](https://ostif.org/), and performed by
+[7ASecurity](https://7asecurity.com/). During this process, we have received a
+few ideas about things that we could do better, like using specific compiler
+flags when preparing our
+[OpenTelemetry Collector binaries](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).
+On 31 May 2024, we received
+[a more serious report](https://github.com/open-telemetry/opentelemetry-collector/security/advisories/GHSA-c74f-6mfw-mm4v):
+a malicious user could cause a denial of service (DoS) when using a specially
+crafted HTTP or gRPC request. The advisory was assigned the following CVE
+identifier: CVE-2024-36129.
+
+When sending an HTTP request with a compressed payload, the Collector would
+verify only whether the compressed payload is beyond a certain limit, but not
+its uncompressed version. A malicious payload could then send a
+“[compressed bomb](https://en.wikipedia.org/wiki/Zip_bomb)”, causing the
+Collector to crash.
+
+Similarly, when sending a gRPC request using zstd compression, the decompression
+mechanism would not respect the limits imposed by gRPC, also causing the
+Collector to crash while decompressing the malicious payload.
+
+A few business hours after the report, on 03 June 2024, Collector developers
+were able to reproduce the report related to HTTP and worked together on a fix
+that
+[got merged the same day](https://github.com/open-telemetry/opentelemetry-collector/pull/10289).
+Given the high score of this issue, we’ve decided to hold the release that would
+have happened on the same day,
+[completing it](https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.0)
+on 04 June 2024 instead.
+
+After the release, we got a confirmation that gRPC with zstd was also affected.
+Within a few business hours of the confirmation, we worked on a fix that also
+[got merged the same day](https://github.com/open-telemetry/opentelemetry-collector/pull/10323).
+[We released v0.102.1](https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.1)
+right after that.
+
+## Am I affected?
+
+You are affected by this vulnerability if you have an OpenTelemetry Collector
+with one or more HTTP or gRPC receivers on a public port, such as the OTLP
+Receiver with the “HTTP” or “gRPC” protocol enabled (typically on ports 4318 and
+4317, respectively) AND the receiver has version 0.102.0 or below. The
+vulnerability is exploitable only by attackers who can send payloads to your
+HTTP/gRPC endpoint(s).
+
+Note that if you require authentication, an attacker would need to have valid
+credentials in order to exploit the vulnerability using the HTTP protocol. For
+gRPC, the exploitable code is executed before authentication.
+
+## What does it mean to me?
+
+If you manage a Collector that has an interface to the public internet, you
+should upgrade it as soon as feasible, and consider setting the parameter
+`max_request_body_size` on HTTP receivers, such as the OTLP receiver, to a value
+that makes sense to your workload. Up to v0.101.0, this setting applied only to
+the payload size sent by the client, which could often be compressed.
+
+Starting from v0.102.0, this setting applies to uncompressed, compressed, and
+decompressed payload sizes and we are establishing a default value of 20 MiB for
+this. This new default characterizes a breaking change, as clients sending
+payloads bigger than 20 MiB will start seeing an error. While we believe most
+authentic requests will be way within this limit, it’s still wise to monitor
+your Collector for increased error rates after this update. Here’s an example of
+a configuration setting a different limit to this field:
+
+```yaml
+receivers:
+  otlp:
+    protocols:
+      http:
+        endpoint: localhost:4318
+        max_request_body_size: 10485760 # 10 MiB
+```
+
+For gRPC receivers, it’s sufficient to upgrade to v0.102.1, as there’s a default
+value being applied to the message size already.
+
+If your Collector instances are receiving data only from trusted clients, like
+your own applications, you are still encouraged to upgrade to the latest
+Collector version but you can do it at your regular pace.
+
+If you are using a custom distribution and building it with the
+[OpenTelemetry Collector Builder (ocb)](https://opentelemetry.io/docs/collector/custom-collector/),
+you can add a
+[“replaces”](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder#configuration)
+entry pointing to the latest version of the
+[confighttp](http://go.opentelemetry.io/collector/config/confighttp) and
+[configgrpc](http://go.opentelemetry.io/collector/config/configgrpc) Go module.
+If your base Collector version is at v0.96.0 or higher, we do not expect any
+compatibility issues by just bumping to the latest version.
+
+## Lessons learned
+
+During this process, we found a couple of gaps in the telemetry for the
+Collector, as well as to the options we provide to Collector admins. Concretely,
+we noticed that we do not have a good way to verify what’s the distribution of
+request sizes received by the Collector, which would have been useful to
+determine whether the change would break clients for a given Collector. We also
+noticed that we don’t provide a way for admins to completely disable
+compression, which would be a good way to mitigate an attack without having to
+upgrade the Collector. We are working to fill those gaps over the next releases.
+
+We are also working on stabilizing the component.UseLocalHostAsDefaultHost
+feature gate to reduce exposure of all Collector endpoints by default. This
+feature gate was motivated by a previous,
+[similar vulnerability on Go's standard library](https://github.com/advisories/GHSA-4374-p667-p6c8)
+and has been in alpha for several months. You can follow the discussion
+surrounding stabilization at
+[issue 8510](https://github.com/open-telemetry/opentelemetry-collector/issues/8510).
+
+This issue was identified by [Miroslav Stampar](https://github.com/stamparm/),
+from 7ASecurity. We’d like to thank 7ASecurity for the responsible reporting of
+this vulnerability.

static/refcache.json

@@ -19,6 +19,14 @@
     "StatusCode": 206,
     "LastSeen": "2024-01-18T08:54:18.004628-05:00"
   },
+  "http://go.opentelemetry.io/collector/config/configgrpc": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-05T14:38:00.458726875+02:00"
+  },
+  "http://go.opentelemetry.io/collector/config/confighttp": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:53.957107724+02:00"
+  },
   "http://mypy-lang.org/": {
     "StatusCode": 206,
     "LastSeen": "2024-01-18T19:02:04.404963-05:00"
@@ -39,6 +47,10 @@
     "StatusCode": 206,
     "LastSeen": "2024-01-30T16:15:09.743434-05:00"
   },
+  "https://7asecurity.com/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:49.77219755+02:00"
+  },
   "https://access.redhat.com/products/ansible-tower-red-hat": {
     "StatusCode": 200,
     "LastSeen": "2024-01-18T08:05:55.59597-05:00"
@@ -1943,6 +1955,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-18T19:01:58.402439-05:00"
   },
+  "https://en.wikipedia.org/wiki/Zip_bomb": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:51.894512776+02:00"
+  },
   "https://equalitymi.org/": {
     "StatusCode": 200,
     "LastSeen": "2024-01-18T19:02:20.222589-05:00"
@@ -2323,6 +2339,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-30T06:01:30.362583-05:00"
   },
+  "https://github.com/advisories/GHSA-4374-p667-p6c8": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:54.745634368+02:00"
+  },
   "https://github.com/aishyandapalli": {
     "StatusCode": 200,
     "LastSeen": "2024-01-18T20:06:24.492346-05:00"
@@ -2923,6 +2943,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-18T20:05:04.146005-05:00"
   },
+  "https://github.com/mx-psi": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:48.499621359+02:00"
+  },
   "https://github.com/nemoshlag": {
     "StatusCode": 200,
     "LastSeen": "2024-01-18T20:05:15.066411-05:00"
@@ -3203,6 +3227,14 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-18T19:37:01.186504-05:00"
   },
+  "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.0": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:53.270047268+02:00"
+  },
+  "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.1": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-05T18:14:20.086786151+02:00"
+  },
   "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.74.0": {
     "StatusCode": 200,
     "LastSeen": "2024-01-30T05:18:23.898255-05:00"
@@ -3231,10 +3263,22 @@
     "StatusCode": 200,
     "LastSeen": "2024-04-04T11:07:15.276911438-07:00"
   },
+  "https://github.com/open-telemetry/opentelemetry-collector/issues/8510": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:55.674365003+02:00"
+  },
   "https://github.com/open-telemetry/opentelemetry-collector/issues/9375": {
     "StatusCode": 200,
     "LastSeen": "2024-05-03T08:34:29.3339-07:00"
   },
+  "https://github.com/open-telemetry/opentelemetry-collector/pull/10289": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:52.747224454+02:00"
+  },
+  "https://github.com/open-telemetry/opentelemetry-collector/pull/10323": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-05T14:37:59.891183931+02:00"
+  },
   "https://github.com/open-telemetry/opentelemetry-collector/pull/6140": {
     "StatusCode": 200,
     "LastSeen": "2024-01-30T05:18:24.402543-05:00"
@@ -3299,6 +3343,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-30T15:25:16.610744-05:00"
   },
+  "https://github.com/open-telemetry/opentelemetry-collector/security/advisories/GHSA-c74f-6mfw-mm4v": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-05T18:14:19.254754319+02:00"
+  },
   "https://github.com/open-telemetry/opentelemetry-configuration": {
     "StatusCode": 200,
     "LastSeen": "2024-01-30T06:01:13.097575-05:00"
@@ -4499,6 +4547,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-30T16:15:09.784881-05:00"
   },
+  "https://github.com/stamparm/": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:57.257378271+02:00"
+  },
   "https://github.com/statsd/statsd": {
     "StatusCode": 200,
     "LastSeen": "2024-01-18T19:37:16.308411-05:00"
@@ -5839,6 +5891,10 @@
     "StatusCode": 200,
     "LastSeen": "2024-01-22T14:34:52.350494+01:00"
   },
+  "https://nvd.nist.gov/vuln/detail/CVE-2024-36129": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:51.244907508+02:00"
+  },
   "https://o11y.news": {
     "StatusCode": 206,
     "LastSeen": "2024-01-18T19:10:29.899332-05:00"
@@ -6043,6 +6099,10 @@
     "StatusCode": 206,
     "LastSeen": "2024-03-19T10:16:59.755536357Z"
   },
+  "https://opentelemetry.io/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:59.050384645+02:00"
+  },
   "https://opentelemetry.io/blog/2022/jaeger-native-otlp/": {
     "StatusCode": 206,
     "LastSeen": "2024-04-27T13:39:35.171666-04:00"
@@ -6123,6 +6183,14 @@
     "StatusCode": 206,
     "LastSeen": "2024-04-30T09:31:38.379400427Z"
   },
+  "https://opentelemetry.io/docs/migration/opentracing/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:58.687939269+02:00"
+  },
+  "https://opentelemetry.io/docs/migration/opentracing/#language-version-support": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:58.393305846+02:00"
+  },
   "https://opentelemetry.io/docs/specs/otel/glossary/#instrumentation-library": {
     "StatusCode": 206,
     "LastSeen": "2024-04-30T09:31:37.929550219Z"
@@ -6131,6 +6199,34 @@
     "StatusCode": 206,
     "LastSeen": "2024-02-24T14:33:05.630341-08:00"
   },
+  "https://opentelemetry.io/docs/specs/otel/versioning-and-stability/#semantic-conventions-stability": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:30:02.895017798+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/otlp/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:30:00.93746657+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/semconv/exceptions/exceptions-spans/#recording-an-exception": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:30:00.355601942+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/semconv/general/attribute-naming/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:59.520526055+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:56.977663774+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/semconv/general/metric-requirement-level/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:59.855130203+02:00"
+  },
+  "https://opentelemetry.io/docs/specs/semconv/resource/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:30:01.040485926+02:00"
+  },
   "https://opentelemetry.io/docs/specs/semconv/resource/k8s/": {
     "StatusCode": 206,
     "LastSeen": "2024-05-15T19:23:47.920456821+03:00"
@@ -6167,6 +6263,10 @@
     "StatusCode": 206,
     "LastSeen": "2024-01-19T13:15:33.481201+01:00"
   },
+  "https://ostif.org/": {
+    "StatusCode": 200,
+    "LastSeen": "2024-06-04T17:29:49.319661335+02:00"
+  },
   "https://otterize.com/": {
     "StatusCode": 200,
     "LastSeen": "2024-01-30T16:15:04.230377-05:00"
@@ -8431,6 +8531,10 @@
     "StatusCode": 206,
     "LastSeen": "2024-01-18T19:10:25.924884-05:00"
   },
+  "https://www.cncf.io/": {
+    "StatusCode": 206,
+    "LastSeen": "2024-06-04T17:29:48.60678168+02:00"
+  },
   "https://www.cncf.io/blog/2019/05/21/a-brief-history-of-opentelemetry-so-far/": {
     "StatusCode": 206,
     "LastSeen": "2024-01-18T20:05:14.383107-05:00"