Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set of offset-based APIs for thread-safe file IO #53669

Merged
merged 74 commits into from
Jun 15, 2021
Merged

Set of offset-based APIs for thread-safe file IO #53669

merged 74 commits into from
Jun 15, 2021

Commits on May 27, 2021

  1. move logic related to opening and initializing handle from FileStream… 

    …Helpers to SafeFileHandle itself
    @adamsitnik
    adamsitnik committed May 27, 2021
    Configuration menu
    Copy the full SHA
    d37d1ee View commit details
    Browse the repository at this point in the history

  2. move the arguments validation logic common for FileStream ctor and Fi… 

    …le.OpenHandle to FileStreamHelpers
    @adamsitnik
    adamsitnik committed May 27, 2021
    Configuration menu
    Copy the full SHA
    539393d View commit details
    Browse the repository at this point in the history

  3. introduce File.OpenHandle

    @adamsitnik
    adamsitnik committed May 27, 2021
    Configuration menu
    Copy the full SHA
    b60e876 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2021

  1. fix the new tests

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    c61d639 View commit details
    Browse the repository at this point in the history

  2. FileStream created from a handle should allow for bufferSize == 0

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    ad252de View commit details
    Browse the repository at this point in the history

  3. File.OpenHandle should always use full path

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    1540dba View commit details
    Browse the repository at this point in the history

  4. fix a bug in NtCreateFile mapping logic

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    d8a3113 View commit details
    Browse the repository at this point in the history

  5. always use NtCreateFile for SafeFileHandle.Open

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    02cd564 View commit details
    Browse the repository at this point in the history

  6. move the ThreadPoolBinding initialization logic to one place

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    1430dd6 View commit details
    Browse the repository at this point in the history

  7. IsAsync: use syscall when not provided and validate it early

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    ebaa92a View commit details
    Browse the repository at this point in the history

  8. read all FileOptions, not just FileOptions.Asynchronous

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    fbbcd4d View commit details
    Browse the repository at this point in the history

  9. move IsPipe CanSeek to SafeFileHandle as well (the new APIs like Rand… 

    …omFile.ReadAtOffset are going to need it)
    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    7f94f85 View commit details
    Browse the repository at this point in the history

  10. disable the Append test as it's currently not supported by design

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    33ecb8d View commit details
    Browse the repository at this point in the history

  11. expose SafeFileHandle.IsAsync and write a test for it

    @adamsitnik
    adamsitnik committed May 28, 2021
    Configuration menu
    Copy the full SHA
    f1af8f6 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2021

  1. fix compilation errors on Linux

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    a46da91 View commit details
    Browse the repository at this point in the history

  2. don't try to test DeleteOnClose for FileStreams created from handle a… 

    …s it's a Windows-specific concept
    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    9ef23eb View commit details
    Browse the repository at this point in the history

  3. take advantage of knowing the file type and avoid one syscall for Can… 

    …Seek for regular files
    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    485eb93 View commit details
    Browse the repository at this point in the history

  4. revert some ordering changes to make the code easier to review

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    c8818aa View commit details
    Browse the repository at this point in the history

  5. Apply suggestions from code review 

    Co-authored-by: Stephen Toub <stoub@microsoft.com>
    @adamsitnik @stephentoub
    adamsitnik and stephentoub committed May 31, 2021
    Configuration menu
    Copy the full SHA
    dc13de0 View commit details
    Browse the repository at this point in the history

  6. add missing ;

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    6c58252 View commit details
    Browse the repository at this point in the history

  7. simplify Windows implementation: 

    * with NtCreateFile there is no need for Validation (NtCreateFile returns error and we never create a safe file handle)
* unify status codes
* complete error mapping
    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    fb24f5c View commit details
    Browse the repository at this point in the history

  8. unify CanSeek and IsPipe for the .NET 5 Windows strategy as well

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    8baf0bd View commit details
    Browse the repository at this point in the history

  9. move the !MS_IO_REDIST methods from File.cs to File.netcoreapp.cs… 

    … file
    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    50aba1f View commit details
    Browse the repository at this point in the history

  10. RandomAccess: GetLength, Read and Write single buffer

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    143d31c View commit details
    Browse the repository at this point in the history

  11. fix Unix build

    @adamsitnik
    adamsitnik committed May 31, 2021
    Configuration menu
    Copy the full SHA
    1d41d65 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2021

  1. refactor the tests

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    2d8ac90 View commit details
    Browse the repository at this point in the history

  2. ReadAsync and WriteAsync

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    bbbd267 View commit details
    Browse the repository at this point in the history

  3. move AsyncWindowsFileStreamStrategy.ValueTaskSource to SafeFileHandle

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    46d9b6f View commit details
    Browse the repository at this point in the history

  4. ReadScatter and WriteGather

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    a946c5d View commit details
    Browse the repository at this point in the history

  5. some polishing

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    4f807ef View commit details
    Browse the repository at this point in the history

  6. ReadScatterAsync and WriteGatherAsync

    @adamsitnik
    adamsitnik committed Jun 1, 2021
    Configuration menu
    Copy the full SHA
    638e67a View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2021

  1. use ReadFileScatter and WriteFileGather when possible

    @adamsitnik
    adamsitnik committed Jun 2, 2021
    Configuration menu
    Copy the full SHA
    fe8fbd1 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2021

  1. complete Unix implementation

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    381776b View commit details
    Browse the repository at this point in the history

  2. add tests for handling already cancelled tokens

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    1febba1 View commit details
    Browse the repository at this point in the history

  3. add tests for unauthorized access

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    8db3f7a View commit details
    Browse the repository at this point in the history

  4. some polishing before sending PR

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    f33d4ca View commit details
    Browse the repository at this point in the history

  5. always treat EOF as success with 0 bytes read (read at end of file) 

    hopefully fix an edge case bug hit in the CI but not reproducible locally
    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    f8c8423 View commit details
    Browse the repository at this point in the history

  6. simplify the NoBuffering tests logic

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    2126f15 View commit details
    Browse the repository at this point in the history

  7. handle lack of preadv pwritev on OSX

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    e613e72 View commit details
    Browse the repository at this point in the history

  8. fix the Linux build

    @adamsitnik
    adamsitnik committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    73f0751 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2021

  1. Apply suggestions from code review 

    Co-authored-by: Stephen Toub <stoub@microsoft.com>
    @adamsitnik @stephentoub
    adamsitnik and stephentoub committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    940609c View commit details
    Browse the repository at this point in the history

  2. code review fixes and suggestions

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    891c2b3 View commit details
    Browse the repository at this point in the history

  3. Unix fixes

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    e462d49 View commit details
    Browse the repository at this point in the history

  4. skip the test that uses System.IO.Pipes for Browser (it's not supported)

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    253faa1 View commit details
    Browse the repository at this point in the history

  5. remove the invalid assumption that every file file opened from path m… 

    …ust be a regular file
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    fd86735 View commit details
    Browse the repository at this point in the history

  6. support ReadFileScatter and WriteFileGather on 32-bit systems as we d… 

    …on't support Itanium anymore and don't need to worry about it
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    6241b08 View commit details
    Browse the repository at this point in the history

  7. move the unpinning to finally block

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    648ab9b View commit details
    Browse the repository at this point in the history

  8. update the comment

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    e075441 View commit details
    Browse the repository at this point in the history

  9. move the call to Dispose to the catch block

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    b1d3006 View commit details
    Browse the repository at this point in the history

  10. InitThreadPoolBindingIfNeeded can dispose the handle only if we own it

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    98eaf24 View commit details
    Browse the repository at this point in the history

  11. when 0 is allowed, the exception message should ask for non-negative … 

    …number instead of positive number
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    76a4326 View commit details
    Browse the repository at this point in the history

  12. fix the casting compilation error

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    dcd9e12 View commit details
    Browse the repository at this point in the history

  13. don't make ownsHandle a protected field as it would become a part of … 

    …public contract
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    2b6a555 View commit details
    Browse the repository at this point in the history

  14. move IOVector from pal_networking.h to pal_io.h and use it in pal_io.… 

    …c to hopefully fix the FreeBSD build issue
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    4a6bb62 View commit details
    Browse the repository at this point in the history

  15. don't use nullable field to store canSeek information. Store the file… 

    … type instead (int)
    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    5ec9e1f View commit details
    Browse the repository at this point in the history

  16. update outdated debug assert

    @adamsitnik
    adamsitnik committed Jun 7, 2021
    Configuration menu
    Copy the full SHA
    4d002b2 View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2021

  1. Revert "don't use nullable field to store canSeek information. Store … 

    …the file type instead (int)"

This reverts commit 5ec9e1f.
    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    d5c9229 View commit details
    Browse the repository at this point in the history

  2. CanSeek should be thread-safe

    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    994b112 View commit details
    Browse the repository at this point in the history

  3. add more debug asserts to try to see why two CI legs have failed with… 

    … mysterious "bad file descriptor" error
    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    9405abe View commit details
    Browse the repository at this point in the history

  4. fix invalid link (it was pointing to a closed duplicate)

    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    752ff9d View commit details
    Browse the repository at this point in the history

  5. async File IO is not supported on Windows for Mono

    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    e609c06 View commit details
    Browse the repository at this point in the history

  6. async file IO is not supported on browser

    @adamsitnik
    adamsitnik committed Jun 8, 2021
    Configuration menu
    Copy the full SHA
    5ae1dfb View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2021

  1. fix WASM build

    @adamsitnik
    adamsitnik committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    d4e5b09 View commit details
    Browse the repository at this point in the history

  2. don't use preadv and pwritev on WASM as WASM implementation has bugs

    @adamsitnik
    adamsitnik committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    abc4b14 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into randomAcess 

    # Conflicts:
#	src/libraries/System.Private.CoreLib/src/System/ThrowHelper.cs
    @adamsitnik
    adamsitnik committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    043a926 View commit details
    Browse the repository at this point in the history

  4. fix a typo

    @adamsitnik
    adamsitnik committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    fc76167 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2021

  1. Merge remote-tracking branch 'upstream/main' into randomAcess 

    # Conflicts:
#	src/libraries/System.Private.CoreLib/src/System/IO/File.netcoreapp.cs
#	src/libraries/System.Runtime/ref/System.Runtime.cs
#	src/mono/mono/mini/wasm_m2n_invoke.g.h
    @adamsitnik
    adamsitnik committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    93db089 View commit details
    Browse the repository at this point in the history

  2. add missing CreateFileW default flags

    @adamsitnik
    adamsitnik committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    1014688 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/main' into randomAcess 

    # Conflicts:
#	src/libraries/System.Private.CoreLib/src/System/IO/Strategies/FileStreamHelpers.cs
    @adamsitnik
    adamsitnik committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    fde8c34 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2021

  1. reduce code duplication

    @adamsitnik
    adamsitnik committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    05327b1 View commit details
    Browse the repository at this point in the history

  2. Apply suggestions from code review 

    Co-authored-by: Stephen Toub <stoub@microsoft.com>
    @adamsitnik @stephentoub
    adamsitnik and stephentoub committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    05e65eb View commit details
    Browse the repository at this point in the history

  3. address code review feedback

    @adamsitnik
    adamsitnik committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    c87354c View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2021

  1. Apply suggestions from code review 

    Co-authored-by: Stephen Toub <stoub@microsoft.com>
    @adamsitnik @stephentoub
    adamsitnik and stephentoub committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    f80bbb7 View commit details
    Browse the repository at this point in the history

  2. address code review feedback

    @adamsitnik
    adamsitnik committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    07051fa View commit details
    Browse the repository at this point in the history