Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep event when add_docker_metadata fails to extract container id #7133

Merged
merged 1 commit into from
May 22, 2018

Conversation

exekias
Copy link
Contributor

@exekias exekias commented May 18, 2018

Before this change an error extracting container id from the source path
resulted on lossing the log line.

This change ensures we report the error but still send the log lines
through, without enriching.

Fixes #7129

@ruflin
Copy link
Member

ruflin commented May 18, 2018

Can you add a changelog and will this need to be backported?

Before this change an error extracting container id from the source path
resulted on lossing the log line.

This change ensures we report the error but still send the log lines
through, without enriching.
@exekias exekias force-pushed the fix-extract-field-keep-event branch from 3f87306 to 808bce4 Compare May 18, 2018 09:08
@exekias
Copy link
Contributor Author

exekias commented May 18, 2018

I had my doubts about the backport, this should not be a really common case, but backporting should not hurt either, wdyt?

@exekias exekias added needs_backport PR is waiting to be backported to other branches. v6.3.0 v6.2.5 labels May 21, 2018
@exekias
Copy link
Contributor Author

exekias commented May 21, 2018

On a second thought, I added backport tags :)

@ruflin ruflin merged commit c7f3f45 into elastic:master May 22, 2018
@ruflin
Copy link
Member

ruflin commented May 22, 2018

Backport or not, both works for me :-)

@exekias exekias added v6.3.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 22, 2018
exekias added a commit to exekias/beats that referenced this pull request Jun 14, 2018
…lastic#7133)

Before this change an error extracting container id from the source path
resulted on lossing the log line.

This change ensures we report the error but still send the log lines
through, without enriching.

(cherry picked from commit c7f3f45)
ph pushed a commit that referenced this pull request Jun 18, 2018
…7133) (#7156)

Before this change an error extracting container id from the source path
resulted on lossing the log line.

This change ensures we report the error but still send the log lines
through, without enriching.

(cherry picked from commit c7f3f45)
@karmi
Copy link

karmi commented Jul 4, 2018

Hi, I'm still getting the lines ignored (not ingested) when there's an error with add_docker_metadata:

2018-07-04T13:20:28.220Z	INFO	[beat]	instance/beat.go:725	Build info	{"system_info": {"build": {"commit": "a04cb664d5fbd4b1aab485d1766f3979c138fd38", "libbeat": "6.3.0", "time": "2018-06-11T22:34:44.000Z", "version": "6.3.0"}}}
...
2018-07-04T13:20:55.295Z	DEBUG	[add_docker_metadata]	add_docker_metadata/add_docker_metadata.go:111	Error while extracting container ID from source path: index is out of range for field 'source'
2018-07-04T13:20:55.297Z	DEBUG	[registrar]	registrar/registrar.go:290	Registry file updated. 1 states written.
2018-07-04T13:20:55.297Z	DEBUG	[registrar]	registrar/registrar.go:232	Processing 1 events
2018-07-04T13:20:55.297Z	DEBUG	[registrar]	registrar/registrar.go:202	Registrar state updates processed. Count: 1
2018-07-04T13:20:55.297Z	DEBUG	[registrar]	registrar/registrar.go:263	Write registry file: /usr/share/filebeat/data/registry
...
2018-07-04T13:21:05.079Z	DEBUG	[input]	log/input.go:168	input states cleaned up. Before: 1, After: 1, Pending: 0
2018-07-04T13:21:10.298Z	DEBUG	[harvester]	log/log.go:85	End of file reached: /var/log/myapp/myapp.log; Backoff now.

@exekias
Copy link
Contributor Author

exekias commented Jul 4, 2018

Hi @karmi, this fix will go out in filebeat 6.3.1

@karmi
Copy link

karmi commented Jul 4, 2018

@exekias, ah right, the v6.3.0 label mislead me...

More importantly, it's an user error on my side :) I've switched from type: docker to type: log (writing from the container to a regular file), so obviously Beats can't infer anything about the Docker environment from the file itself... So apologies for a false alarm!

leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…lastic#7133) (elastic#7156)

Before this change an error extracting container id from the source path
resulted on lossing the log line.

This change ensures we report the error but still send the log lines
through, without enriching.

(cherry picked from commit b1d8542)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants