Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Misc hashing #12687

Merged
merged 8 commits into from
Jul 31, 2024
Merged

Misc hashing #12687

merged 8 commits into from
Jul 31, 2024

Conversation

bep
Copy link
Member

@bep bep commented Jul 30, 2024

  • identity: Add BenchmarkHashString
  • identity: Upgrade to github.com/mitchellh/hashstructure/v2 v2.0.2
  • identity: Use xxHash in hashstructure (note)

@bep bep force-pushed the feat/hashstructv2-11644 branch 7 times, most recently from 97bf8bc to 52d82f8 Compare July 31, 2024 12:27
@bep bep marked this pull request as ready for review July 31, 2024 13:35
bep added 8 commits July 31, 2024 15:39
@bep
identity: Add BenchmarkHashString
44a60b5
@bep
identity: Upgrade to github.com/mitchellh/hashstructure/v2 v2.0.2
c22389e 
```
name                 old time/op    new time/op    delta
HashString/n28-10       131ns ± 7%     133ns ±19%   ~     (p=0.343 n=4+4)
HashString/n112-10      237ns ± 5%     241ns ± 3%   ~     (p=0.229 n=4+4)
HashString/n448-10      688ns ± 3%     687ns ± 1%   ~     (p=0.886 n=4+4)
HashString/n1792-10    2.51µs ± 3%    2.56µs ± 3%   ~     (p=0.200 n=4+4)
HashString/n7168-10    10.1µs ± 2%    10.0µs ± 4%   ~     (p=0.686 n=4+4)

name                 old alloc/op   new alloc/op   delta
HashString/n28-10       80.0B ± 0%     80.0B ± 0%   ~     (all equal)
HashString/n112-10       160B ± 0%      160B ± 0%   ~     (all equal)
HashString/n448-10       496B ± 0%      496B ± 0%   ~     (all equal)
HashString/n1792-10    1.84kB ± 0%    1.84kB ± 0%   ~     (all equal)
HashString/n7168-10    8.24kB ± 0%    8.24kB ± 0%   ~     (all equal)

name                 old allocs/op  new allocs/op  delta
HashString/n28-10        4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n112-10       4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n448-10       4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n1792-10      4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n7168-10      4.00 ± 0%      4.00 ± 0%   ~     (all equal)
```

Closes gohugoio#11644
@bep
identity: Use xxHash in hashstructure (note)
e7d2f3b 
```
name                 old time/op    new time/op    delta
HashString/n28-10       133ns ± 9%     107ns ±10%  -19.58%  (p=0.029 n=4+4)
HashString/n112-10      243ns ± 5%     121ns ± 4%  -50.08%  (p=0.029 n=4+4)
HashString/n448-10      698ns ± 3%     174ns ± 5%  -75.02%  (p=0.029 n=4+4)
HashString/n1792-10    2.58µs ± 4%    0.38µs ± 4%  -85.11%  (p=0.029 n=4+4)
HashString/n7168-10    10.0µs ± 3%     1.3µs ± 4%  -86.91%  (p=0.029 n=4+4)

name                 old alloc/op   new alloc/op   delta
HashString/n28-10       80.0B ± 0%     72.0B ± 0%  -10.00%  (p=0.029 n=4+4)
HashString/n112-10       160B ± 0%      152B ± 0%   -5.00%  (p=0.029 n=4+4)
HashString/n448-10       496B ± 0%      488B ± 0%   -1.61%  (p=0.029 n=4+4)
HashString/n1792-10    1.84kB ± 0%    1.83kB ± 0%   -0.43%  (p=0.029 n=4+4)
HashString/n7168-10    8.24kB ± 0%    8.23kB ± 0%   -0.06%  (p=0.029 n=4+4)

name                 old allocs/op  new allocs/op  delta
HashString/n28-10        4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n112-10       4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n448-10       4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n1792-10      4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n7168-10      4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
```
@bep
deps: Replace github.com/mitchellh/hashstructure/v2 with github.com/g…
c03675c 
…ohugoio/hashstructure

The old repo is archived and no longer maintained.

See gohugoio#11644
@bep
resources: Add BenchmarkHashImage
016942b
@bep
Replace the MD5 hashing of images with xxHash
6fee54e 
Note that we only use this for change detection.

The previous implementation invoked `MD5FromReaderFast` that created a MD5 has from 8 64 bytes chunks in the file, which is obviously very fast. The new implementation creates the hash from the entire file and ... seems to be even more effective:

```
name          old time/op    new time/op    delta
HashImage-10    9.45µs ±21%   10.89µs ± 1%     ~     (p=0.343 n=4+4)

name          old alloc/op   new alloc/op   delta
HashImage-10      144B ± 0%        8B ± 0%  -94.44%  (p=0.029 n=4+4)

name          old allocs/op  new allocs/op  delta
HashImage-10      4.00 ± 0%      1.00 ± 0%  -75.00%  (p=0.029 n=4+4)
```
@bep
Consolidate all hashing to the common/hashing package
f9156b2 
And remove now unsued hashing funcs.
@bep
Shorten processed image filenames
60e77aa 
Fixes gohugoio#12688
Fixes gohugoio#12656
@bep bep force-pushed the feat/hashstructv2-11644 branch from 52d82f8 to 60e77aa Compare July 31, 2024 13:44
@bep bep merged commit 216a69a into gohugoio:master Jul 31, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@bep