Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Misc hashing #12687

Merged
merged 8 commits into from
Jul 31, 2024
Merged

Misc hashing #12687

merged 8 commits into from
Jul 31, 2024

Commits on Jul 31, 2024

  1. identity: Add BenchmarkHashString

    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    44a60b5 View commit details
    Browse the repository at this point in the history

  2. identity: Upgrade to github.com/mitchellh/hashstructure/v2 v2.0.2 

    ```
name                 old time/op    new time/op    delta
HashString/n28-10       131ns ± 7%     133ns ±19%   ~     (p=0.343 n=4+4)
HashString/n112-10      237ns ± 5%     241ns ± 3%   ~     (p=0.229 n=4+4)
HashString/n448-10      688ns ± 3%     687ns ± 1%   ~     (p=0.886 n=4+4)
HashString/n1792-10    2.51µs ± 3%    2.56µs ± 3%   ~     (p=0.200 n=4+4)
HashString/n7168-10    10.1µs ± 2%    10.0µs ± 4%   ~     (p=0.686 n=4+4)

name                 old alloc/op   new alloc/op   delta
HashString/n28-10       80.0B ± 0%     80.0B ± 0%   ~     (all equal)
HashString/n112-10       160B ± 0%      160B ± 0%   ~     (all equal)
HashString/n448-10       496B ± 0%      496B ± 0%   ~     (all equal)
HashString/n1792-10    1.84kB ± 0%    1.84kB ± 0%   ~     (all equal)
HashString/n7168-10    8.24kB ± 0%    8.24kB ± 0%   ~     (all equal)

name                 old allocs/op  new allocs/op  delta
HashString/n28-10        4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n112-10       4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n448-10       4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n1792-10      4.00 ± 0%      4.00 ± 0%   ~     (all equal)
HashString/n7168-10      4.00 ± 0%      4.00 ± 0%   ~     (all equal)
```

Closes gohugoio#11644
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c22389e View commit details
    Browse the repository at this point in the history

  3. identity: Use xxHash in hashstructure (note) 

    ```
name                 old time/op    new time/op    delta
HashString/n28-10       133ns ± 9%     107ns ±10%  -19.58%  (p=0.029 n=4+4)
HashString/n112-10      243ns ± 5%     121ns ± 4%  -50.08%  (p=0.029 n=4+4)
HashString/n448-10      698ns ± 3%     174ns ± 5%  -75.02%  (p=0.029 n=4+4)
HashString/n1792-10    2.58µs ± 4%    0.38µs ± 4%  -85.11%  (p=0.029 n=4+4)
HashString/n7168-10    10.0µs ± 3%     1.3µs ± 4%  -86.91%  (p=0.029 n=4+4)

name                 old alloc/op   new alloc/op   delta
HashString/n28-10       80.0B ± 0%     72.0B ± 0%  -10.00%  (p=0.029 n=4+4)
HashString/n112-10       160B ± 0%      152B ± 0%   -5.00%  (p=0.029 n=4+4)
HashString/n448-10       496B ± 0%      488B ± 0%   -1.61%  (p=0.029 n=4+4)
HashString/n1792-10    1.84kB ± 0%    1.83kB ± 0%   -0.43%  (p=0.029 n=4+4)
HashString/n7168-10    8.24kB ± 0%    8.23kB ± 0%   -0.06%  (p=0.029 n=4+4)

name                 old allocs/op  new allocs/op  delta
HashString/n28-10        4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n112-10       4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n448-10       4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n1792-10      4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
HashString/n7168-10      4.00 ± 0%      3.00 ± 0%  -25.00%  (p=0.029 n=4+4)
```
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    e7d2f3b View commit details
    Browse the repository at this point in the history

  4. deps: Replace github.com/mitchellh/hashstructure/v2 with github.com/g… 

    …ohugoio/hashstructure

The old repo is archived and no longer maintained.

See gohugoio#11644
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    c03675c View commit details
    Browse the repository at this point in the history

  5. resources: Add BenchmarkHashImage

    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    016942b View commit details
    Browse the repository at this point in the history

  6. Replace the MD5 hashing of images with xxHash 

    Note that we only use this for change detection.

The previous implementation invoked `MD5FromReaderFast` that created a MD5 has from 8 64 bytes chunks in the file, which is obviously very fast. The new implementation creates the hash from the entire file and ... seems to be even more effective:

```
name          old time/op    new time/op    delta
HashImage-10    9.45µs ±21%   10.89µs ± 1%     ~     (p=0.343 n=4+4)

name          old alloc/op   new alloc/op   delta
HashImage-10      144B ± 0%        8B ± 0%  -94.44%  (p=0.029 n=4+4)

name          old allocs/op  new allocs/op  delta
HashImage-10      4.00 ± 0%      1.00 ± 0%  -75.00%  (p=0.029 n=4+4)
```
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    6fee54e View commit details
    Browse the repository at this point in the history

  7. Consolidate all hashing to the common/hashing package 

    And remove now unsued hashing funcs.
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    f9156b2 View commit details
    Browse the repository at this point in the history

  8. Shorten processed image filenames 

    Fixes gohugoio#12688
Fixes gohugoio#12656
    @bep
    bep committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    60e77aa View commit details
    Browse the repository at this point in the history