Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Race condition somewhere? fetching PDFs using the Sniffer while Qiqqa updates the library (OCR!) in the background randomly locks up Qiqqa mid-activity. #18

Closed
GerHobbelt opened this issue Aug 2, 2019 · 2 comments
Labels
🐛bug Something isn't working ⛷performance Anything that's related to UX: speed of response; I/O speed, etc.
Milestone

Comments

@GerHobbelt
Copy link
Collaborator

Given the code for Library.IsBusyAddingPDFs I assume Qiqqa has more multithreading risks like that (a timeout/delay check isn't going to fly at all times and with a large library over here, those 3 seconds isn't going to cut it, resulting in a Qiqqa lock up some of the time)

Random lock up behaviour like this strongly hints towards a race condition somewhere in the code, so I guess we'll have to go through the interlocking/thread interaction inside Qiqqa. 🙉 🙈 😄

Observations

  1. These lockups almost always happen when you download a new/existing(?) PDF from the Sniffer into Qiqqa while both Qiqqa.exe and QiqqaOCR.exe are running.

  2. Dialing down the number of QiqqaOCR.exe instances to 1(ONE) on my i7 dev box seems to reduce the chance of lockup like this happening, but it certainly is still present.

  3. Using Library.IsBusyAddingPDFs in the Infrequent Background Task code seems to have minor negative = worse impact.

  4. My work on SHA-1: 8a1d766 (Fixing When re-indexing a large library, Qiqqa is unresponsive for a VERY long time (too long to wait: 1+ hours)  #17) seems to have a strong detrimental = much worse effect as the number of lockup occurrences has increased notably since that change.

    However, since that commit fixed When re-indexing a large library, Qiqqa is unresponsive for a VERY long time (too long to wait: 1+ hours)  #17 and thus prevents me from short-circuiting to the Windows Task Manager kill -9=end process tree decision, this "much worse" is quite possibly purely psychological as Qiqqa is running much longer now, per session.

GerHobbelt added a commit to GerHobbelt/qiqqa-open-source that referenced this issue Aug 2, 2019
… locks being applied correctly and completely: for example, a few places did not follow best practices by using the dissuaded `lock(this){...}` idiom (https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement)
GerHobbelt added a commit to GerHobbelt/qiqqa-open-source that referenced this issue Aug 2, 2019
… locks being applied correctly and completely. Also contains a few lines from work done before related to jimmejardine#10 et al.
GerHobbelt added a commit to GerHobbelt/qiqqa-open-source that referenced this issue Aug 7, 2019
…al sections in there were **humongous** in both code side and run-time duration; now the number of lock-ups due to *very* slow loading PDFs coming in from the Qiqqa Sniffer should be quite reduced: work related to jimmejardine#18
@GerHobbelt
Copy link
Collaborator Author

Done as per #33.

locking code has been inspected and fixed in a few places; improved in another spot (SHA-1: a80be7d)

Note: do not lock(...) on Dictionaries<...> but use a separate, dedicated 'lock' object instead: this helped remove a few additional glitches in the code.

Commits:

Revision: a80be7d
done work on #27 and on the lockups of Qiqqa (some critical sections in there were humongous in both code side and run-time duration; now the number of lock-ups due to very slow loading PDFs coming in from the Qiqqa Sniffer should be quite reduced: work related to #18

Revision: 53f2ca8
code cleanup activity (which happened while going through the code for thread safely locks inspection)

Revision: 8b2b3de
#18 work :: code review part 2, looking for thread safety locks being applied correctly and completely. Also contains a few lines from work done before related to #10 et al.

Revision: 5dcda97
#18 work :: code review part 1, looking for thread safety locks being applied correctly and completely: for example, a few places did not follow best practices by using the dissuaded lock(this){...} idiom (https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement)

@GerHobbelt GerHobbelt changed the title Race condition somewhere? fetching PDFs using the Sniffer while Qiqqa updates the library (OCR!) in the background randomly locks up Qiqqa mid-activity. ✅Race condition somewhere? fetching PDFs using the Sniffer while Qiqqa updates the library (OCR!) in the background randomly locks up Qiqqa mid-activity. Aug 8, 2019
@GerHobbelt
Copy link
Collaborator Author

Closing and decluttering the issue list so it stays workable for me: fixed in https://github.com/GerHobbelt/qiqqa-open-source mainline=master branch, pending #15 / any maintainer rights/actions.

@GerHobbelt GerHobbelt added 🐛bug Something isn't working ⛷performance Anything that's related to UX: speed of response; I/O speed, etc. labels Oct 4, 2019
@GerHobbelt GerHobbelt added this to the v82 milestone Oct 4, 2019
@GerHobbelt GerHobbelt changed the title ✅Race condition somewhere? fetching PDFs using the Sniffer while Qiqqa updates the library (OCR!) in the background randomly locks up Qiqqa mid-activity. Race condition somewhere? fetching PDFs using the Sniffer while Qiqqa updates the library (OCR!) in the background randomly locks up Qiqqa mid-activity. Oct 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛bug Something isn't working ⛷performance Anything that's related to UX: speed of response; I/O speed, etc.
Projects
None yet
Development

No branches or pull requests

1 participant