Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add google-pubsub input type to Filebeat #12746

Merged
merged 4 commits into from
Jul 2, 2019
Merged

Add google-pubsub input type to Filebeat #12746

merged 4 commits into from
Jul 2, 2019

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Jul 2, 2019 •
edited
Loading

Add new input type for consuming messages from Google Cloud Pub/Sub topic
subscriptions.

This input can, for example, be used to receive Stackdriver logs that have been exported to a Google Cloud Pub/Sub topic.

Example configuration:

filebeat.inputs:
- type: google-pubsub
  project_id: my-gcp-project-id
  topic: vpc-firewall-logs-topic
  subscription.name: filebeat-vpc-firewall-logs-sub
  credentials_file: ${path.config}/my-pubsub-subscriber-credentails.json

Most of the code is for the GCP SDK and its deps. So I recommend reviewing cadf25e, the Filebeat input code.

@andrewkroh
Add the cloud.google.com/go/pubsub to vendor (with its many deps)
89b1611 
This adds v0.40.0 of the cloud.google.com/go/pubsub SDK along with its dependencies.
@andrewkroh andrewkroh requested a review from a team as a code owner July 2, 2019 03:17
@andrewkroh andrewkroh force-pushed the feature/fb/pubsub-input-type branch 2 times, most recently from b9cea98 to fa1471b Compare July 2, 2019 03:21
@andrewkroh
Add google-pubsub input type to Filebeat
cadf25e 
Add new input type for consuming messages from Google Cloud Pub/Sub topic
subscriptions.
@andrewkroh andrewkroh force-pushed the feature/fb/pubsub-input-type branch from fa1471b to cadf25e Compare July 2, 2019 03:26
@elasticmachine
Copy link
Collaborator

Pinging @elastic/secops

@andrewkroh
Remove invalid vendor.json entries
098fec1 
I think a botched govendor run might have caused this.
adriansr
adriansr approved these changes Jul 2, 2019
View reviewed changes
Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewkroh
Copy link
Member Author

jenkins, test this

1 similar comment
@andrewkroh
Copy link
Member Author

jenkins, test this

@andrewkroh andrewkroh merged commit 82ef39b into elastic:master Jul 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@houndci-bot houndci-bot houndci-bot left review comments

@adriansr adriansr adriansr approved these changes

Assignees
No one assigned
Labels
enhancement Filebeat Filebeat review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewkroh @elasticmachine @houndci-bot @adriansr