Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New hashing algorithm for fingerprint processor #15418

Merged
merged 10 commits into from
Jan 10, 2020

Conversation

rvillablanca
Copy link
Contributor

@rvillablanca rvillablanca commented Jan 9, 2020

This adds a the xxHash hashing to the list of available methods in the fingerprint processor.

Solves #15080

@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

1 similar comment
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@rvillablanca
Copy link
Contributor Author

Ready to be reviewed @ycombinator

@ycombinator
Copy link
Contributor

CI fails like so:

>> check: Checking source code for common problems
processors/fingerprint/hash.go:28:2: cannot find package "github.com/OneOfOne/xxhash" in any of:
	/home/travis/gopath/src/github.com/elastic/beats/vendor/github.com/OneOfOne/xxhash (vendor tree)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/github.com/OneOfOne/xxhash (from $GOROOT)
	/home/travis/gopath/src/github.com/OneOfOne/xxhash (from $GOPATH)

This is because we need to vendor the github.com/OneOfOne/xxhash package as part of this PR.

@ycombinator
Copy link
Contributor

jenkins, test this

@ycombinator
Copy link
Contributor

ycombinator commented Jan 9, 2020

Could you also add a line about this enhancement in the CHANGELOG.next.asciidoc file?

@rvillablanca
Copy link
Contributor Author

rvillablanca commented Jan 9, 2020

CI fails like so:

>> check: Checking source code for common problems
processors/fingerprint/hash.go:28:2: cannot find package "github.com/OneOfOne/xxhash" in any of:
	/home/travis/gopath/src/github.com/elastic/beats/vendor/github.com/OneOfOne/xxhash (vendor tree)
	/home/travis/.gimme/versions/go1.13.5.linux.amd64/src/github.com/OneOfOne/xxhash (from $GOROOT)
	/home/travis/gopath/src/github.com/OneOfOne/xxhash (from $GOPATH)

This is because we need to vendor the github.com/OneOfOne/xxhash package as part of this PR.

@ycombinator Do you have vendoring process automated with any make/mage task or should I copy manually the dependency ? I was looking for some task but without success

@ycombinator
Copy link
Contributor

Do you have vendoring process automated with any make/mage task or should I copy manually the dependency ?

We use govendor. See https://www.elastic.co/guide/en/beats/devguide/current/beats-contributing.html#dependencies.

@rvillablanca
Copy link
Contributor Author

Ok all sugestions were solved

CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
@ycombinator
Copy link
Contributor

@rvillablanca Left a couple of minor suggestions but this is looking good otherwise!

@ycombinator
Copy link
Contributor

BTW, looks like CI is now failing on an outdated NOTICE.txt file. This should be fixed if you run make notice in the root directory.

@rvillablanca
Copy link
Contributor Author

I will check

@ycombinator
Copy link
Contributor

Also, could you add the new hash method (xxhash) to the fingerprint processor docs? See https://github.com/elastic/beats/blob/master/libbeat/processors/fingerprint/docs/fingerprint.asciidoc.

@ycombinator
Copy link
Contributor

jenkins, test this

Copy link
Contributor

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the contribution @rvillablanca and for working through the issue with the hash.Hash interface. I'll merge this once CI looks good.

@rvillablanca
Copy link
Contributor Author

I think we are done with all @ycombinator, any questions do not hesitate to ask me

@ycombinator
Copy link
Contributor

NOTICE.txt needs an update again, since the version of the dependency was updated.

@rvillablanca
Copy link
Contributor Author

Ok NOTICE.txt updated

@ycombinator
Copy link
Contributor

jenkins, test this

@ycombinator
Copy link
Contributor

Travis CI is green. Jenkins CI failures are unrelated. Merging.

@ycombinator ycombinator merged commit 7b1ac24 into elastic:master Jan 10, 2020
@rvillablanca rvillablanca deleted the ft/xxhash branch January 10, 2020 14:58
ycombinator pushed a commit to ycombinator/beats that referenced this pull request Jan 10, 2020
* XXHash configuration and test

* Removed unused option

* Inlining function

* Changelog documentation

* xxhash library vendored

* Updated changelog

* Update NOTICE.txt

* * Update version github.com/OneOfOne/xxhash@1.2.7
* Using a new constructor function xxhash.NewHash64

* Add xxhash as option for hashing algorithm in fingerprint's docs

* Update NOTICE.txt
ycombinator added a commit that referenced this pull request Jan 14, 2020
* XXHash configuration and test

* Removed unused option

* Inlining function

* Changelog documentation

* xxhash library vendored

* Updated changelog

* Update NOTICE.txt

* * Update version github.com/OneOfOne/xxhash@1.2.7
* Using a new constructor function xxhash.NewHash64

* Add xxhash as option for hashing algorithm in fingerprint's docs

* Update NOTICE.txt

Co-authored-by: Rodrigo Villablanca Vásquez <villa061004@gmail.com>
@andresrc andresrc added the Team:Integrations Label for the Integrations team label Mar 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs_backport PR is waiting to be backported to other branches. review Team:Integrations Label for the Integrations team v7.6.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants