Skip to content
This repository has been archived by the owner on Aug 2, 2021. It is now read-only.

DO NOT MERGE: Filehasher #2022

Open
wants to merge 67 commits into
base: master
Choose a base branch
from
Open

DO NOT MERGE: Filehasher #2022

wants to merge 67 commits into from

Commits on Dec 10, 2019

  1. storage: Elaborate comment on Encryption.Reset

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    a8f3fc4 View commit details
    Browse the repository at this point in the history

  2. storage: Add Reset() to interface

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    22839fb View commit details
    Browse the repository at this point in the history

  3. file: Add reference filehasher pre-cleanup state

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    df3f3b7 View commit details
    Browse the repository at this point in the history

  4. file: Reference hasher commenting and partial cleanup

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    472b6fd View commit details
    Browse the repository at this point in the history

  5. file: Replaced hash data size calc with util.go function, add comments

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    01d5af2 View commit details
    Browse the repository at this point in the history

  6. file: Add test for dangling chunk, correct last test case param

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    872e93e View commit details
    Browse the repository at this point in the history

  7. file: Add correct, non-optimized new hasher implementation

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    181e0bf View commit details
    Browse the repository at this point in the history

  8. file: Fix races

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    bc1e288 View commit details
    Browse the repository at this point in the history

  9. file: Use all test cases in vector test even if fail

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    300d373 View commit details
    Browse the repository at this point in the history

  10. file: Use local chan pointer to avoid tight loop in doneC

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    8d3e020 View commit details
    Browse the repository at this point in the history

  11. file: Fix race in target members read

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    781cb6b View commit details
    Browse the repository at this point in the history

  12. file: Cleanup commented code

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    a528496 View commit details
    Browse the repository at this point in the history

  13. file: Remove (properly) tight loop in doneC select

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    a418656 View commit details
    Browse the repository at this point in the history

  14. file: Set target doneC to nil if target set on job create

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    f76e00f View commit details
    Browse the repository at this point in the history

  15. file: Move serialdata generation in bench outside bench loop

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    4b87da3 View commit details
    Browse the repository at this point in the history

  16. file: Lock dummy writer data buffer access

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    988ff37 View commit details
    Browse the repository at this point in the history

  17. file: Remove commented code

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    243b92b View commit details
    Browse the repository at this point in the history

  18. file: Update Write and Sum to correct job object interface

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e491ee0 View commit details
    Browse the repository at this point in the history

  19. file: Add hasher benchmark

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    bec33a0 View commit details
    Browse the repository at this point in the history

  20. file: Make Hasher writers asynchronous within jobs

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    45db7d9 View commit details
    Browse the repository at this point in the history

  21. file: Add explicit proof of ref hash on chunkSize*branches+chunkSize 

    Same case fails for pyramidhasher = bug
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    21d25b7 View commit details
    Browse the repository at this point in the history

  22. file: Complete comments

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    04705b6 View commit details
    Browse the repository at this point in the history

  23. file: Delint

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    b4ba44a View commit details
    Browse the repository at this point in the history

  24. file: Rename files and extract index, target to separate files

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e4f8fc6 View commit details
    Browse the repository at this point in the history

  25. file: Extract job sum into separate method

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    d300b86 View commit details
    Browse the repository at this point in the history

  26. file: Extract split to separate file, implement bmt.SectionWriter

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    92c2c7e View commit details
    Browse the repository at this point in the history

  27. file: Add comments

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    5407b62 View commit details
    Browse the repository at this point in the history

  28. file, param: Add package for global interfaces and settings

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e5d29cb View commit details
    Browse the repository at this point in the history

  29. file: Separate splitter and hasher in different packages

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    91c0df1 View commit details
    Browse the repository at this point in the history

  30. file: Adjust interface to add context

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e1f4283 View commit details
    Browse the repository at this point in the history

  31. file: Add SectionWriter storer interface for chunks

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    835c4b4 View commit details
    Browse the repository at this point in the history

  32. file: Add test for FileStore SectionWriter

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    4f26df0 View commit details
    Browse the repository at this point in the history

  33. file: Add FileStore integration test with Splitter

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    eaf0af1 View commit details
    Browse the repository at this point in the history

  34. file, bmt: Wrap bmt.Hasher in param.SectionWriter

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e5fd998 View commit details
    Browse the repository at this point in the history

  35. file: Add test, chunk.Store sink for both data+intermediate hashers

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    a0e9ffd View commit details
    Browse the repository at this point in the history

  36. file: Add encrypt writer stub and cache store as local testutil

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    9dab8da View commit details
    Browse the repository at this point in the history

  37. file: Test encryption pipeline with one chunk

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    afc7001 View commit details
    Browse the repository at this point in the history

  38. file: Add key derivation to filehasher encrypt

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    3953663 View commit details
    Browse the repository at this point in the history

  39. file, param: Add test to verify buffer-neutral encryption

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    ef446f4 View commit details
    Browse the repository at this point in the history

  40. file: Add first part of intermediate chunk test 

    missing correctness check
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    d0c7833 View commit details
    Browse the repository at this point in the history

  41. file: Add Encryption.Reset to file/encrypt Reset

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    fef9be3 View commit details
    Browse the repository at this point in the history

  42. file: Add multi-section write capability to job

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    77d25a3 View commit details
    Browse the repository at this point in the history

  43. file, bmt, param: Add dynamic branch/section write and tests 

    encrypt and split now fails, need revisit endcount calculation
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    35b91ae View commit details
    Browse the repository at this point in the history

  44. file: Add test for endcount calculation with differing sectionsize/br… 

    …anches

Test fails; implementation not yet updated
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    3132569 View commit details
    Browse the repository at this point in the history

  45. file: Use same bmt for data and intermediate

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    d783260 View commit details
    Browse the repository at this point in the history

  46. file, param, bmt: Amend split and store after bmt change

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    e778505 View commit details
    Browse the repository at this point in the history

  47. file: Amend encrypt to compile, fix benchmarks

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    5dc2d85 View commit details
    Browse the repository at this point in the history

  48. file, bmt, param: Add Connect method to IF to set underlying writer

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    57c79ed View commit details
    Browse the repository at this point in the history

  49. file: Simplify code by adding function for BMT hashfunc create

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    edd6f6e View commit details
    Browse the repository at this point in the history

  50. file: Extract generate span sizes to separate function

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    888d38f View commit details
    Browse the repository at this point in the history

  51. file, bmt: Simplify ReferenceHasher

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    f790e33 View commit details
    Browse the repository at this point in the history

  52. file: Fix wrong call in ReferenceHasher benchmark

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    1c877e4 View commit details
    Browse the repository at this point in the history

  53. bmt, param: Use hash.Hash for SectionWriter, implement in bmt.Hasher

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    83edc4b View commit details
    Browse the repository at this point in the history

  54. file: Move reference hasher to own package, bmt compiles w changes

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    d18b51c View commit details
    Browse the repository at this point in the history

  55. bmt: Tests pass after changes to interface

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    63d2149 View commit details
    Browse the repository at this point in the history

  56. bmt: Fix zerohash bug in sync BMT

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    685fa40 View commit details
    Browse the repository at this point in the history

  57. bmt, param: Cleanup, implement proper seek for async

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    677d023 View commit details
    Browse the repository at this point in the history

  58. file: Fix file/testutillocal/Cache

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    84749e0 View commit details
    Browse the repository at this point in the history

  59. file: Move reference back, make hasher tests compile 

    Concurrency issues in asynchasher, Replace Seek with SeekSection
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    b28375a View commit details
    Browse the repository at this point in the history

  60. file, bmt: Implement param.SectionWriter for all in file

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    28e424c View commit details
    Browse the repository at this point in the history

  61. bmt: Set lock across SeekSection() and Write() for async

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    d64359a View commit details
    Browse the repository at this point in the history

  62. file: Fix dummySectionWriter test fails

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    9dc2351 View commit details
    Browse the repository at this point in the history

  63. bmt, file, param: Make all tests in file/hasher pass

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    fad18c1 View commit details
    Browse the repository at this point in the history

  64. bmt: rehabilitiate file/encrypt, make bmt tests pass 

    BMT is now slower :/
    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    94c6cb6 View commit details
    Browse the repository at this point in the history

  65. storage, bmt: Replace ResetWithLength with param.SectionWriter.SpanBytes

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    12355cb View commit details
    Browse the repository at this point in the history

  66. bmt, param: Cleanup + implement SetSpanBytes()

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    0fbe5f8 View commit details
    Browse the repository at this point in the history

  67. file, bmt: Cleanup

    @nolash
    nolash committed Dec 10, 2019
    Configuration menu
    Copy the full SHA
    4d85352 View commit details
    Browse the repository at this point in the history