Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I get the events from the files that are not closed on windows? #422

Open
Tatametheus opened this issue Jul 13, 2022 · 4 comments
Open
Labels

Comments

@Tatametheus
Copy link

I'm using notify to watch csv files that are generated by another app on windows. During the development stage, all the files tested on wsl, everything were good. However, when I test on windows right now, no events emit by the files,which are read-only and the app holds the writing, unless the app shut down.
I'm not familiar with the file system on windows or linux(ubuntu wsl) and not clear why it works on ubuntu wsl but not windows.
Thanks for the awesome project.

@jarruda
Copy link

jarruda commented Aug 27, 2023

I can reproduce this on Windows and on MacOS. Here's a reproduction using a python script to write:

from time import sleep

with open("test.txt", "a") as f:
    while True:
        f.write("foo\n")
        f.flush()
        print("wrote")
        sleep(2)

Using the monitor-raw.rs sample to read test.txt, no events are generated until the python script is terminated.

tail -f test.txt shows the file being written to as expected.

On Windows, if you run Get-Content -Tail 10 -Wait, then events begin to get generated by notify.

@0xpr03
Copy link
Member

0xpr03 commented Aug 27, 2023

My current guess is that windows doesn't flush the file and waits with change reports until either the file handle is closed, the application is closed or a certain time is passed.

You can also watch this happen in explorer when download/moving giant files: The file size stays at (for example) 0KB while you know that the download progress is happening, and F5/refresh in explorer won't help. But if you right click on the file and go to Properties, then suddenly the actual size is reported (but not updated unless you redo this all the time). From what I know it's kinda the same with windows defender, where closing a file handle blocks until the virus scan finished - so events are happening only then.

Whether we can change/"fix" this behaviour on windows I can't say right now.

@0xpr03 0xpr03 added A-bug os-windows Z-needs info Needs more information labels Aug 27, 2023
@jarruda
Copy link

jarruda commented Aug 30, 2023

This issue is not Windows specific. I was able to reproduce this behavior on MacOS and Windows per my comment. Further, the Get-Content -Tail -Wait behavior in Windows and tail -f behavior in MacOS implies that flushing is not the issue at hand.

@AntoineGS
Copy link

AntoineGS commented May 17, 2024

I just ran into this issue on Windows using notify to create a tail-style log parser.
Has anyone find a solution or workaround?

Edit: I noticed that if I refresh the file in Notepad++ or use tail in Powershell at the same time then an event is triggered.

Wumpf added a commit to rerun-io/rerun that referenced this issue Sep 26, 2024
…d test (#7517)

### What

Our ability to load files as they are written is limited by both
flushing & notification behavior of OS & filesystem.
This is now documented on the `save` methods.

See also
* notify-rs/notify#422

Also, fixes a test that failed because of this on Windows (nightly ci).


### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7517?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/7517?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!
* [x] If have noted any breaking changes to the log API in
`CHANGELOG.md` and the migration guide

- [PR Build Summary](https://build.rerun.io/pr/7517)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)

To run all checks from `main`, comment on the PR with `@rerun-bot
full-check`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants