Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add S3 logs/traces exporter #19

Open
YANG-DB opened this issue Jun 12, 2023 · 0 comments
Open

Add S3 logs/traces exporter #19

YANG-DB opened this issue Jun 12, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@YANG-DB
Copy link
Member

YANG-DB commented Jun 12, 2023

Feature Request

Add an S3 exporter for logs/trace data as part of the OTEL-Collector pipline

Is your feature request related to a problem?

Optimize Cost/Store for Observability telemetry signals

Describe the solution you'd like:

I'd like to optimize the volume of data being stored in my DB vs in my Object Store
I'd like to have the ability to store most of the data in an Object Store and use it in conjunction with the data stored in the DB

Describe alternatives you've considered.

Which alternative solutions or features have you considered?

Additional Context

Add min-io support in opensearch Dockerfile

FROM opensearchproject/opensearch:2.8.0

ENV MINIO_ACCESS_KEY_ID minioadmin
ENV MINIO_SECRET_ACCESS_KEY minioadmin
ENV MINIO_ENDPOINT minio:9000
ENV MINIO_PROTOCOL http

RUN /usr/share/opensearch/bin/opensearch-plugin install --batch repository-s3
RUN /usr/share/opensearch/bin/opensearch-keystore create

RUN echo $MINIO_ACCESS_KEY_ID | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.access_key

RUN echo $MINIO_SECRET_ACCESS_KEY | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.secret_key

RUN echo $MINIO_ENDPOINT | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.endpoint
RUN echo $MINIO_PROTOCOL | /usr/share/opensearch/bin/opensearch-keystore add --stdin s3.client.default.protocol

Add the next min-io service to the docker-compose :

services:
  minio:
    container_name: minio
    image: quay.io/minio/minio:latest
    command: server http://minio/mnt/disk{1...4}/minio --console-address ":9001"
    volumes:
      - /home/aj/minio/disk-1:/mnt/disk1
      - /home/aj/minio/disk-2:/mnt/disk2
      - /home/aj/minio/disk-3:/mnt/disk3
      - /home/aj/minio/disk-4:/mnt/disk4
    ports:
      - 20091:9001
    hostname: minio


```

Add any other context about the feature request here.

@YANG-DB YANG-DB added the enhancement New feature or request label Jun 12, 2023
@Swiddis Swiddis removed the untriaged label Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants