OpAMP Agent Extension (open-telemetry#16594)

See [design document](https://docs.google.com/document/d/1KtH5atZQUs9Achbce6LiOaJxLbksNJenvgvyKLsJrkc/edit#heading=h.ioikt02qpy5f). Depends on: - [Implement ability for extensions to be notified about effective configuration opentelemetry-collector#6596](open-telemetry/opentelemetry-collector#6596) - [Make service.instance.id and other telemetry attributes available to extensions opentelemetry-collector#6599](open-telemetry/opentelemetry-collector#6599) Closes open-telemetry#16618 --------- Signed-off-by: Sean Porter <portertech@gmail.com> Co-authored-by: Daniel Jaglowski <jaglows3@gmail.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
ClickHouse · Nov 24, 2023 · 4503d2d · 4503d2d
1 parent a4c57b6
commit 4503d2d
Show file tree

Hide file tree

Showing 21 changed files with 1,109 additions and 11 deletions.
diff --git a/.chloggen/add-opamp-agent-extension.yaml b/.chloggen/add-opamp-agent-extension.yaml
@@ -0,0 +1,16 @@
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: new_component
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: extension/opampextension
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Add a new extension that implements an OpAMP agent for reporting the collector's health and effective configuration.
+
+# One or more tracking issues related to the change
+issues: [16462]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -108,6 +108,7 @@ extension/observer/ecstaskobserver/                                     @open-te
 extension/observer/hostobserver/                                        @open-telemetry/collector-contrib-approvers @MovieStoreGuy
 extension/observer/k8sobserver/                                         @open-telemetry/collector-contrib-approvers @rmfitzpatrick @dmitryax
 extension/oidcauthextension/                                            @open-telemetry/collector-contrib-approvers @jpkrohling
+extension/opampextension/                                               @open-telemetry/collector-contrib-approvers @portertech @evan-bradley @tigrannajaryan
 extension/pprofextension/                                               @open-telemetry/collector-contrib-approvers @MovieStoreGuy
 extension/sigv4authextension/                                           @open-telemetry/collector-contrib-approvers @Aneurysm9 @erichsueh3
 extension/storage/                                                      @open-telemetry/collector-contrib-approvers @dmitryax @atoulme @djaglowski

diff --git a/.github/ISSUE_TEMPLATE/bug_report.yaml b/.github/ISSUE_TEMPLATE/bug_report.yaml
@@ -100,6 +100,7 @@ body:
       - extension/observer/hostobserver
       - extension/observer/k8sobserver
       - extension/oidcauth
+      - extension/opamp
       - extension/pprof
       - extension/sigv4auth
       - extension/storage

diff --git a/.github/ISSUE_TEMPLATE/feature_request.yaml b/.github/ISSUE_TEMPLATE/feature_request.yaml
@@ -94,6 +94,7 @@ body:
       - extension/observer/hostobserver
       - extension/observer/k8sobserver
       - extension/oidcauth
+      - extension/opamp
       - extension/pprof
       - extension/sigv4auth
       - extension/storage

diff --git a/.github/ISSUE_TEMPLATE/other.yaml b/.github/ISSUE_TEMPLATE/other.yaml
@@ -94,6 +94,7 @@ body:
       - extension/observer/hostobserver
       - extension/observer/k8sobserver
       - extension/oidcauth
+      - extension/opamp
       - extension/pprof
       - extension/sigv4auth
       - extension/storage

diff --git a/.github/dependabot.yml b/.github/dependabot.yml
@@ -8,7 +8,12 @@ updates:
       interval: "weekly"
       day: "wednesday"
   - package-ecosystem: "gomod"
-    directory: "/cmd/githubgen"
+    directory: "/cmd/otelcontribcol"
+    schedule:
+      interval: "weekly"
+      day: "wednesday"
+  - package-ecosystem: "gomod"
+    directory: "/cmd/oteltestbedcol"
     schedule:
       interval: "weekly"
       day: "wednesday"
@@ -302,11 +307,6 @@ updates:
     schedule:
       interval: "weekly"
       day: "wednesday"
-  - package-ecosystem: "gomod"
-    directory: "/extension/encoding/jaegerencodingextension"
-    schedule:
-      interval: "weekly"
-      day: "wednesday"
   - package-ecosystem: "gomod"
     directory: "/extension/headerssetterextension"
     schedule:
@@ -1012,11 +1012,6 @@ updates:
     schedule:
       interval: "weekly"
       day: "wednesday"
-  - package-ecosystem: "gomod"
-    directory: "/receiver/splunkenterprisereceiver"
-    schedule:
-      interval: "weekly"
-      day: "wednesday"
   - package-ecosystem: "gomod"
     directory: "/receiver/splunkhecreceiver"
     schedule:
@@ -1067,6 +1062,11 @@ updates:
     schedule:
       interval: "weekly"
       day: "wednesday"
+  - package-ecosystem: "gomod"
+    directory: "/receiver/webhookeventreceiver"
+    schedule:
+      interval: "weekly"
+      day: "wednesday"
   - package-ecosystem: "gomod"
     directory: "/receiver/windowseventlogreceiver"
     schedule:

diff --git a/extension/opampextension/Makefile b/extension/opampextension/Makefile
@@ -0,0 +1 @@
+include ../../Makefile.Common
diff --git a/extension/opampextension/README.md b/extension/opampextension/README.md
@@ -0,0 +1,46 @@
+# OpAMP Agent Extension
+<!-- status autogenerated section -->
+| Status        |           |
+| ------------- |-----------|
+| Stability     | [development]  |
+| Distributions | [] |
+| Issues        | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Aextension%2Fopamp%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Aextension%2Fopamp) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Aextension%2Fopamp%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Aextension%2Fopamp) |
+| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner)    | [@portertech](https://www.github.com/portertech), [@evan-bradley](https://www.github.com/evan-bradley), [@tigrannajaryan](https://www.github.com/tigrannajaryan) |
+
+[development]: https://github.com/open-telemetry/opentelemetry-collector#development
+<!-- end autogenerated section -->
+
+## Configuration
+
+The following settings are required:
+
+- `server`: The OpAMP server connection settings.
+  - `ws`: The OpAMP websocket transport settings.
+    - `endpoint` (no default): The OpAMP server websocket endpoint (URL).
+
+The following settings are optional:
+
+- `server`: The OpAMP server connection settings.
+  - `ws`: The OpAMP websocket transport settings.
+    - `tls`: TLS settings.
+    - `headers`: HTTP headers to set.
+- `instance_uid`: A ULID formatted as a 26 character string in canonical
+  representation. Auto-generated on start if missing. Setting this ensures the
+  instance UID remains constant across process restarts.
+
+### Example
+
+``` yaml
+extensions:
+  opamp:
+    server:
+      ws:
+        endpoint: wss://127.0.0.1:4320/v1/opamp
+```
+
+## Status
+
+This OpenTelemetry OpAMP agent extension is intended to support the [OpAMP
+Supervisor][supervisor].
+
+[supervisor]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/opampsupervisor
diff --git a/extension/opampextension/config.go b/extension/opampextension/config.go
@@ -0,0 +1,50 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package opampextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension"
+
+import (
+	"errors"
+
+	"github.com/oklog/ulid/v2"
+	"go.opentelemetry.io/collector/config/configopaque"
+	"go.opentelemetry.io/collector/config/configtls"
+)
+
+// Config contains the configuration for the opamp extension. Trying to mirror
+// the OpAMP supervisor config for some consistency.
+type Config struct {
+	Server *OpAMPServer `mapstructure:"server"`
+
+	// InstanceUID is a ULID formatted as a 26 character string in canonical
+	// representation. Auto-generated on start if missing.
+	InstanceUID string `mapstructure:"instance_uid"`
+}
+
+// OpAMPServer contains the OpAMP transport configuration.
+type OpAMPServer struct {
+	WS *OpAMPWebsocket `mapstructure:"ws"`
+}
+
+// OpAMPWebsocket contains the OpAMP websocket transport configuration.
+type OpAMPWebsocket struct {
+	Endpoint   string                         `mapstructure:"endpoint"`
+	TLSSetting configtls.TLSClientSetting     `mapstructure:"tls,omitempty"`
+	Headers    map[string]configopaque.String `mapstructure:"headers,omitempty"`
+}
+
+// Validate checks if the extension configuration is valid
+func (cfg *Config) Validate() error {
+	if cfg.Server.WS.Endpoint == "" {
+		return errors.New("opamp server websocket endpoint must be provided")
+	}
+
+	if cfg.InstanceUID != "" {
+		_, err := ulid.ParseStrict(cfg.InstanceUID)
+		if err != nil {
+			return errors.New("opamp instance_uid is invalid")
+		}
+	}
+
+	return nil
+}
diff --git a/extension/opampextension/config_test.go b/extension/opampextension/config_test.go
@@ -0,0 +1,57 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package opampextension
+
+import (
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/confmap"
+	"go.opentelemetry.io/collector/confmap/confmaptest"
+)
+
+func TestUnmarshalDefaultConfig(t *testing.T) {
+	factory := NewFactory()
+	cfg := factory.CreateDefaultConfig()
+	assert.NoError(t, component.UnmarshalConfig(confmap.New(), cfg))
+	assert.Equal(t, factory.CreateDefaultConfig(), cfg)
+}
+
+func TestUnmarshalConfig(t *testing.T) {
+	cm, err := confmaptest.LoadConf(filepath.Join("testdata", "config.yaml"))
+	require.NoError(t, err)
+	factory := NewFactory()
+	cfg := factory.CreateDefaultConfig()
+	assert.NoError(t, component.UnmarshalConfig(cm, cfg))
+	assert.Equal(t,
+		&Config{
+			Server: &OpAMPServer{
+				WS: &OpAMPWebsocket{
+					Endpoint: "wss://127.0.0.1:4320/v1/opamp",
+				},
+			},
+			InstanceUID: "01BX5ZZKBKACTAV9WEVGEMMVRZ",
+		}, cfg)
+}
+
+func TestConfigValidate(t *testing.T) {
+	cfg := &Config{
+		Server: &OpAMPServer{
+			WS: &OpAMPWebsocket{},
+		},
+	}
+	err := cfg.Validate()
+	assert.Equal(t, "opamp server websocket endpoint must be provided", err.Error())
+	cfg.Server.WS.Endpoint = "wss://127.0.0.1:4320/v1/opamp"
+	assert.NoError(t, cfg.Validate())
+	cfg.InstanceUID = "01BX5ZZKBKACTAV9WEVGEMMVRZFAIL"
+	err = cfg.Validate()
+	require.Error(t, err)
+	assert.Equal(t, "opamp instance_uid is invalid", err.Error())
+	cfg.InstanceUID = "01BX5ZZKBKACTAV9WEVGEMMVRZ"
+	require.NoError(t, cfg.Validate())
+}
diff --git a/extension/opampextension/doc.go b/extension/opampextension/doc.go
@@ -0,0 +1,6 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+//go:generate mdatagen metadata.yaml
+
+package opampextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension"
diff --git a/extension/opampextension/factory.go b/extension/opampextension/factory.go
@@ -0,0 +1,34 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package opampextension // import "github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension"
+
+import (
+	"context"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/extension"
+
+	"github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension/internal/metadata"
+)
+
+func NewFactory() extension.Factory {
+	return extension.NewFactory(
+		metadata.Type,
+		createDefaultConfig,
+		createExtension,
+		metadata.ExtensionStability,
+	)
+}
+
+func createDefaultConfig() component.Config {
+	return &Config{
+		Server: &OpAMPServer{
+			WS: &OpAMPWebsocket{},
+		},
+	}
+}
+
+func createExtension(_ context.Context, set extension.CreateSettings, cfg component.Config) (extension.Extension, error) {
+	return newOpampAgent(cfg.(*Config), set.Logger, set.BuildInfo, set.Resource)
+}
diff --git a/extension/opampextension/factory_test.go b/extension/opampextension/factory_test.go
@@ -0,0 +1,31 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package opampextension
+
+import (
+	"context"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/component/componenttest"
+	"go.opentelemetry.io/collector/extension/extensiontest"
+)
+
+func TestFactory_CreateDefaultConfig(t *testing.T) {
+	cfg := NewFactory().CreateDefaultConfig()
+	assert.Equal(t, createDefaultConfig().(*Config), cfg)
+
+	assert.NoError(t, componenttest.CheckConfigStruct(cfg))
+	ext, err := createExtension(context.Background(), extensiontest.NewNopCreateSettings(), cfg)
+	require.NoError(t, err)
+	require.NotNil(t, ext)
+}
+
+func TestFactory_CreateExtension(t *testing.T) {
+	cfg := NewFactory().CreateDefaultConfig()
+	ext, err := createExtension(context.Background(), extensiontest.NewNopCreateSettings(), cfg)
+	require.NoError(t, err)
+	require.NotNil(t, ext)
+}
diff --git a/extension/opampextension/go.mod b/extension/opampextension/go.mod
@@ -0,0 +1,47 @@
+module github.com/open-telemetry/opentelemetry-collector-contrib/extension/opampextension
+
+go 1.20
+
+require (
+	github.com/google/uuid v1.3.1
+	github.com/oklog/ulid/v2 v2.1.0
+	github.com/open-telemetry/opamp-go v0.9.0
+	github.com/stretchr/testify v1.8.4
+	go.opentelemetry.io/collector/component v0.88.0
+	go.opentelemetry.io/collector/config/configopaque v0.88.0
+	go.opentelemetry.io/collector/config/configtls v0.88.0
+	go.opentelemetry.io/collector/confmap v0.88.0
+	go.opentelemetry.io/collector/extension v0.88.0
+	go.opentelemetry.io/collector/pdata v1.0.0-rcv0017
+	go.opentelemetry.io/collector/semconv v0.88.0
+	go.uber.org/zap v1.26.0
+)
+
+require (
+	github.com/cenkalti/backoff/v4 v4.2.1 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/fsnotify/fsnotify v1.6.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/gorilla/websocket v1.4.2 // indirect
+	github.com/knadh/koanf v1.5.0 // indirect
+	github.com/knadh/koanf/v2 v2.0.1 // indirect
+	github.com/mitchellh/copystructure v1.2.0 // indirect
+	github.com/mitchellh/mapstructure v1.5.1-0.20220423185008-bf980b35cac4 // indirect
+	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	go.opentelemetry.io/collector/config/configtelemetry v0.88.0 // indirect
+	go.opentelemetry.io/collector/featuregate v1.0.0-rcv0017 // indirect
+	go.opentelemetry.io/otel v1.19.0 // indirect
+	go.opentelemetry.io/otel/metric v1.19.0 // indirect
+	go.opentelemetry.io/otel/trace v1.19.0 // indirect
+	go.uber.org/goleak v1.2.1 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	golang.org/x/net v0.17.0 // indirect
+	golang.org/x/sys v0.13.0 // indirect
+	golang.org/x/text v0.13.0 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
+	google.golang.org/grpc v1.59.0 // indirect
+	google.golang.org/protobuf v1.31.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+)