Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

orchestrate: improve visibility on incoming connections #36

Open
jiping-s opened this issue Aug 10, 2022 · 0 comments
Open

orchestrate: improve visibility on incoming connections #36

jiping-s opened this issue Aug 10, 2022 · 0 comments

Comments

@jiping-s
Copy link
Collaborator

It's been impossible to tell which clients and what they are logging through slog-agent.

The existing information:

  • client addresses
  • orchestration keys

are not sufficient in situations where the keys cannot differentiate clients (e.g. multi-instance services), and full information cannot be represented in metrics due to concerns about metric label cardinality (M inputs * N pipelines * T steps * U outputs).

A workaround is to log key fields when an incoming connection requests to dispatch logs to a previously-unused pipeline (id'ed by by orchestration keys), done in major: Use Go 1.18 generics, but turned out useless since it logs only the orchestration keys.

An option is to log the metrics keys in addition to orchestration keys, and / or dump raw headers in one of the new logs.

A better solution would be to save samples of incoming logs with information on destination pipelines assigned by orchestrator, and print them on certain signals, which may be done as part of #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant