Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reader/csv: Use binary mode on Windows #2121

Merged
merged 1 commit into from
Sep 30, 2023
Merged

Commits on Sep 30, 2023

  1. reader/csv: Use binary mode on Windows

    On windows, when using `open()`, files are opened in Text mode by
    default. This causes automatic line conversion from `\r\n` to `\n`.
    We don't want this behaviour, and furthermore, it messes up our
    computation. If the file contents are `abc\r\n` and Windows is directed
    to read, it will return 4 bytes, `abc\n`. However, the file position as
    reported by `lseek` is 5, since we are five bytes into the file. This
    means we can miss rows if we get really unlucky with block offsets.
    
    Specifying `_O_BINARY` makes windows use binary mode, which is most
    compatible with the linux behaviour. Our CSV reader is prepared to
    handle carriage returns anyway.
    Riolku committed Sep 30, 2023
    Configuration menu
    Copy the full SHA
    45a2362 View commit details
    Browse the repository at this point in the history