Crash on Python 3.7 related to treading / concurrent.future

[flozz]

Error encountered in Python 3.7.14. No issue in 3.10.7 though. Manage different Python versions using pyenv.

Steps to reproduce:

git clone https://github.com/flozz/yoga-image-optimizer.git
cd yoga-image-optimizer
sudo pip3 install .

Stacktrace:

$ yoga-image-optimizer
...
Exception in thread QueueManagerThread:
Traceback (most recent call last):
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/threading.py", line 926, in _bootstrap_inner                                                                                     self.run()
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/threading.py", line 870, in run self._target(*self._args, **self._kwargs)                                               File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/concurrent/futures/process.py", line 375, in _queue_management_worker
    thread_wakeup.clear()                                                                   File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/concurrent/futures/process.py", line 93, in clear
    while self._reader.poll():
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/multiprocessing/connection.py", line 255, in poll
    self._check_closed()
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

Traceback (most recent call last):
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/site-packages/yoga_image_optimizer/main_window.py", line 385, in _on_main_window_destroyed
    app.quit()
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/site-packages/yoga_image_optimizer/application.py", line 142, in quit
    self._thumbnailer.cancel_all()
  File "/home/foobar/.pyenv/versions/3.7.14/lib/python3.7/site-packages/yoga_image_optimizer/thumbnailer.py", line 156, in cancel_all
    self._executor.shutdown(wait=False, cancel_futures=True)
TypeError: shutdown() got an unexpected keyword argument 'cancel_futures'

Originally posted by @kianmeng in #30 (comment)

[flozz]

The first error is caused by a bug in the concurrent.futures.ProcessPoolExecutor implementation of Python 3.7 and 3.8 (fixed in 3.9):

https://bugs.python.org/issue36281

I can workaround by killing process before requesting the shutdown of the executor, and by adding wait=True as parameter.

---

The second error is my fault: the cancel_futures parameter does not exist before Python 3.9, so we should cancel futures one by one manually on older Python version.

---

Both errors are fixed on master :)

[kianmeng]

@flozz Thanks for the explanation and fix. I've tried and it worked without issues, 🥳 !