Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miscellaneous tidying up #155

Merged
merged 4 commits into from
Jan 30, 2020
Merged

Miscellaneous tidying up #155

merged 4 commits into from
Jan 30, 2020

Conversation

koushiro
Copy link
Contributor

@koushiro koushiro commented Jan 17, 2020

  • Replace chashmap with dashmap
  • Replace fnv with fxhash Replace fnv with ahash
  • Update some outdated dependencies
  • Fix some clippy warnings
  • Upgrade MSRV to 1.40.0 (required by dashmap)

Signed-off-by: koushiro koushiro.cqx@gmail.com

$ cargo tree --no-indent | sed 's/ (\*)//' | sort -u | wc -l
110 => 108

* Replace chashmap with dashmap
* Replace fnv with fxhash
* Update some outdated dependencies
* Fix some clippy warnings

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
@jonhoo
Copy link
Owner

jonhoo commented Jan 17, 2020

Updating outdated dependencies + fixing clippy issues is great, but I'm not sure I follow the reasons for moving from chashmap + fnv to dashmap + fxhash? Could you run the benchmarks and see how much of a difference it makes?

@koushiro
Copy link
Contributor Author

I run the benchmarks several times, the main difference is the sample.

fnv vs fxhash
chashmap vs dashmap (In addition, chashmap has not been actively maintained for a long time)

The following are the results of the benchmarks :

Benchmarking collapse/dtrace/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 36.0s or reduce sample count to 20
collapse/dtrace/1       time:   [7.1749 ms 7.1834 ms 7.1936 ms]                               
                        thrpt:  [183.28 MiB/s 183.54 MiB/s 183.75 MiB/s]
                 change:
                        time:   [-10.591% -10.208% -9.8182%] (p = 0.00 < 0.05)
                        thrpt:  [+10.887% +11.369% +11.846%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking collapse/dtrace/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 15.6s or reduce sample count to 30
collapse/dtrace/8       time:   [3.7003 ms 3.7200 ms 3.7415 ms]                               
                        thrpt:  [352.38 MiB/s 354.41 MiB/s 356.30 MiB/s]
                 change:
                        time:   [+3.5867% +6.3238% +8.9067%] (p = 0.00 < 0.05)
                        thrpt:  [-8.1783% -5.9477% -3.4625%]
                        Performance has regressed.
Found 19 outliers among 100 measurements (19.00%)
  12 (12.00%) low severe
  4 (4.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe

Benchmarking collapse/perf/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 75.7s or reduce sample count to 10
collapse/perf/1         time:   [14.517 ms 14.579 ms 14.656 ms]                             
                        thrpt:  [204.27 MiB/s 205.35 MiB/s 206.22 MiB/s]
                 change:
                        time:   [-4.5712% -4.0204% -3.3481%] (p = 0.00 < 0.05)
                        thrpt:  [+3.4640% +4.1888% +4.7902%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
Benchmarking collapse/perf/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 30.2s or reduce sample count to 20
collapse/perf/8         time:   [6.9734 ms 7.0537 ms 7.1317 ms]                             
                        thrpt:  [419.78 MiB/s 424.42 MiB/s 429.31 MiB/s]
                 change:
                        time:   [-0.8887% +1.1977% +3.4207%] (p = 0.29 > 0.05)
                        thrpt:  [-3.3076% -1.1835% +0.8967%]
                        No change in performance detected.

Benchmarking collapse/sample: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 31.2s or reduce sample count to 20
collapse/sample         time:   [5.9941 ms 6.0039 ms 6.0157 ms]                             
                        thrpt:  [230.73 MiB/s 231.18 MiB/s 231.56 MiB/s]
                 change:
                        time:   [-32.651% -32.275% -31.850%] (p = 0.00 < 0.05)
                        thrpt:  [+46.735% +47.657% +48.480%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

     Running target/release/deps/flamegraph-06895b51f7b48927
Benchmarking flamegraph: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 93.9s or reduce sample count to 10
flamegraph              time:   [18.137 ms 18.169 ms 18.207 ms]                        
                        thrpt:  [33.862 MiB/s 33.932 MiB/s 33.992 MiB/s]
                 change:
                        time:   [+0.0002% +0.4953% +0.9201%] (p = 0.03 < 0.05)
                        thrpt:  [-0.9117% -0.4929% -0.0002%]
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking collapse/dtrace/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 36.9s or reduce sample count to 20
collapse/dtrace/1       time:   [7.2298 ms 7.2383 ms 7.2487 ms]                               
                        thrpt:  [181.88 MiB/s 182.15 MiB/s 182.36 MiB/s]
                 change:
                        time:   [+3.3492% +3.7599% +4.2092%] (p = 0.00 < 0.05)
                        thrpt:  [-4.0392% -3.6236% -3.2407%]
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
Benchmarking collapse/dtrace/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 15.0s or reduce sample count to 40
collapse/dtrace/8       time:   [3.5561 ms 3.6172 ms 3.6662 ms]                               
                        thrpt:  [359.61 MiB/s 364.48 MiB/s 370.75 MiB/s]
                 change:
                        time:   [-13.011% -10.631% -8.3676%] (p = 0.00 < 0.05)
                        thrpt:  [+9.1317% +11.895% +14.957%]
                        Performance has improved.

Benchmarking collapse/perf/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 72.1s or reduce sample count to 10
collapse/perf/1         time:   [13.954 ms 13.966 ms 13.980 ms]                             
                        thrpt:  [214.14 MiB/s 214.36 MiB/s 214.54 MiB/s]
                 change:
                        time:   [-2.5281% -2.2371% -1.8959%] (p = 0.00 < 0.05)
                        thrpt:  [+1.9325% +2.2882% +2.5937%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking collapse/perf/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 32.5s or reduce sample count to 20
collapse/perf/8         time:   [6.9738 ms 6.9876 ms 7.0031 ms]                             
                        thrpt:  [427.49 MiB/s 428.44 MiB/s 429.29 MiB/s]
                 change:
                        time:   [+1.8551% +2.4680% +3.0320%] (p = 0.00 < 0.05)
                        thrpt:  [-2.9427% -2.4086% -1.8213%]
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  3 (3.00%) high severe

Benchmarking collapse/sample: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 30.8s or reduce sample count to 20
collapse/sample         time:   [5.9055 ms 5.9179 ms 5.9322 ms]                             
                        thrpt:  [233.98 MiB/s 234.54 MiB/s 235.04 MiB/s]
                 change:
                        time:   [-33.139% -32.558% -31.924%] (p = 0.00 < 0.05)
                        thrpt:  [+46.895% +48.275% +49.564%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

     Running target/release/deps/flamegraph-06895b51f7b48927
Benchmarking flamegraph: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 95.3s or reduce sample count to 10
flamegraph              time:   [18.249 ms 18.272 ms 18.299 ms]                        
                        thrpt:  [33.691 MiB/s 33.741 MiB/s 33.783 MiB/s]
                 change:
                        time:   [+0.4541% +0.7753% +1.1155%] (p = 0.00 < 0.05)
                        thrpt:  [-1.1031% -0.7694% -0.4521%]
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

@jonhoo
Copy link
Owner

jonhoo commented Jan 20, 2020

Thanks! I'm inclined to merge this once CI passes :)
cc @jasonrhansen

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
@koushiro
Copy link
Contributor Author

Replace fxhash with ahash.

fnv vs fxhash vs ahash

The following are the results of the benchmarks (fnv vs ahash):

Benchmarking collapse/dtrace/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 38.4s or reduce sample count to 20.
collapse/dtrace/1       time:   [7.2652 ms 7.2828 ms 7.3032 ms]                               
                        thrpt:  [180.52 MiB/s 181.03 MiB/s 181.47 MiB/s]
                 change:
                        time:   [-2.4640% -2.0866% -1.6656%] (p = 0.00 < 0.05)
                        thrpt:  [+1.6938% +2.1311% +2.5262%]
                        Performance has improved.
Benchmarking collapse/dtrace/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 15.4s or reduce sample count to 40.
collapse/dtrace/8       time:   [3.6853 ms 3.7133 ms 3.7389 ms]                               
                        thrpt:  [352.62 MiB/s 355.06 MiB/s 357.75 MiB/s]
                 change:
                        time:   [+0.7949% +3.2707% +5.6091%] (p = 0.01 < 0.05)
                        thrpt:  [-5.3112% -3.1671% -0.7886%]
                        Change within noise threshold.

Benchmarking collapse/perf/1: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 70.2s or reduce sample count to 10.
collapse/perf/1         time:   [13.347 ms 13.369 ms 13.395 ms]                             
                        thrpt:  [223.50 MiB/s 223.94 MiB/s 224.29 MiB/s]
                 change:
                        time:   [-11.943% -11.483% -11.034%] (p = 0.00 < 0.05)
                        thrpt:  [+12.403% +12.973% +13.563%]
                        Performance has improved.
Benchmarking collapse/perf/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 29.9s or reduce sample count to 20.
collapse/perf/8         time:   [6.8566 ms 6.8700 ms 6.8830 ms]                             
                        thrpt:  [434.95 MiB/s 435.77 MiB/s 436.62 MiB/s]
                 change:
                        time:   [+1.3629% +2.8959% +4.4474%] (p = 0.00 < 0.05)
                        thrpt:  [-4.2580% -2.8144% -1.3445%]
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild

Benchmarking collapse/sample: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 28.9s or reduce sample count to 20.
collapse/sample         time:   [5.6617 ms 5.6686 ms 5.6768 ms]                             
                        thrpt:  [244.51 MiB/s 244.86 MiB/s 245.15 MiB/s]
                 change:
                        time:   [-35.032% -34.876% -34.706%] (p = 0.00 < 0.05)
                        thrpt:  [+53.154% +53.553% +53.921%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe

     Running target/release/deps/flamegraph-167a004553da2145
Benchmarking flamegraph: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 90.7s or reduce sample count to 10.
flamegraph              time:   [17.696 ms 17.825 ms 17.949 ms]                        
                        thrpt:  [34.349 MiB/s 34.588 MiB/s 34.839 MiB/s]
                 change:
                        time:   [-5.4637% -4.9816% -4.4606%] (p = 0.00 < 0.05)
                        thrpt:  [+4.6688% +5.2428% +5.7795%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

@koushiro
Copy link
Contributor Author

Azure Pipelines seems to be wrong due to configuration issues. I'm not familiar with the configuration of Azure Pipelines :(
Can you fix the CI? @jonhoo

@jonhoo
Copy link
Owner

jonhoo commented Jan 30, 2020

@koushiro CI should now be fixed on master

@codecov
Copy link

codecov bot commented Jan 30, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@db49bd8). Click here to learn what that means.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #155   +/-   ##
=========================================
  Coverage          ?   88.51%           
=========================================
  Files             ?       16           
  Lines             ?     2212           
  Branches          ?        0           
=========================================
  Hits              ?     1958           
  Misses            ?      254           
  Partials          ?        0
Impacted Files Coverage Δ
src/collapse/perf.rs 97.39% <ø> (ø)
src/collapse/common.rs 61.11% <ø> (ø)
src/collapse/dtrace.rs 89.8% <ø> (ø)
src/flamegraph/color/mod.rs 73.04% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db49bd8...7d7ed65. Read the comment docs.

Signed-off-by: koushiro <koushiro.cqx@gmail.com>
@jonhoo jonhoo merged commit c813456 into jonhoo:master Jan 30, 2020
@koushiro koushiro deleted the use-dashmap branch January 30, 2020 15:51
@jonhoo
Copy link
Owner

jonhoo commented Jan 30, 2020

Great, thanks @koushiro! I forgot to ask you if you could please also update the CHANGELOG to reflect this change? Once you do I can do another release! I think this should just be a minor release?

@koushiro
Copy link
Contributor Author

@jonhoo Yes, it should just be a minor release.

How about the CHANGELOG below?

## [0.9.2] - 2020-01-30
### Changed
- Replace `chashmap` with `dashmap`
- Replace `fnv` with `ahash`
- Update some outdated dependencies
- Fix some clippy warnings
- Upgrade MSRV to 1.40.0 (required by `dashmap`)

Is there anything else to add?

@jonhoo
Copy link
Owner

jonhoo commented Jan 30, 2020

Just add it under the existing Unreleased section and I'll bump things when I do the release. You can probably remove the line about clippy warnings. It'd also be good to link to this PR! Apart from that it looks good :)

@koushiro
Copy link
Contributor Author

Okay :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants