Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lambda-promtail: Allow ingesting real-time logs from Cloudfront #5978

Closed
juissi-t opened this issue Apr 21, 2022 · 0 comments · Fixed by #5977
Closed

lambda-promtail: Allow ingesting real-time logs from Cloudfront #5978

juissi-t opened this issue Apr 21, 2022 · 0 comments · Fixed by #5977

Comments

@juissi-t
Copy link
Contributor

Is your feature request related to a problem? Please describe.

We have a use case for ingesting AWS Cloudfront real-time logs to Loki, to generate metrics and alerts for some CF distributions faster than by pulling the metrics from Cloudwatch to Prometheus. I would like lambda-promtail to be able to ingest logs from Cloudfront using Kinesis data streams.

Describe the solution you'd like

I wrote a patch to lambda-promtail which does that: #5977

Additional context

I tested the solution by running a custom build of lambda-promtail which forwarded real-time logs from a CF distribution to Loki.
@juissi-t juissi-t mentioned this issue Apr 21, 2022
Merged
4 tasks
dannykopping pushed a commit that referenced this issue Aug 26, 2022
@juissi-t
lambda-promtail: Add support for Kinesis data stream events (#5977)
073c620 
**What this PR does / why we need it**:

This PR adds support for sending Kinesis data stream events to lambda-promtail. One use case would be e.g. to send CloudFront realtime logs to Loki.

**Which issue(s) this PR fixes**:

Fixes #5978 

**Special notes for your reviewer**:

n/a

**Checklist**
- [x] Documentation added
- [x] Tests updated
- [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
lxwzy pushed a commit to lxwzy/loki that referenced this issue Nov 7, 2022
@juissi-t @lxwzy
lambda-promtail: Add support for Kinesis data stream events (grafana#…
70d6234 
…5977)

**What this PR does / why we need it**:

This PR adds support for sending Kinesis data stream events to lambda-promtail. One use case would be e.g. to send CloudFront realtime logs to Loki.

**Which issue(s) this PR fixes**:

Fixes grafana#5978 

**Special notes for your reviewer**:

n/a

**Checklist**
- [x] Documentation added
- [x] Tests updated
- [x] Is this an important fix or new feature? Add an entry in the `CHANGELOG.md`.
- [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/upgrading/_index.md`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

lambda-promtail: Add support for Kinesis data stream events juissi-t/loki
1 participant
@juissi-t