open_process() sometimes "swallows" exceptions

[shamrin]

I had an error in my code:

import trio

async def run():
    async with await trio.open_process('python3 -c "while True: pass"', shell=True):
        print('before')
        1 + ''
        print('after')

trio.run(run)

Running the above felt as if the program just hangs:

$ .venv/bin/python3 swallowed-error.py
before

If I press Ctrl-C, I finally get my TypeError printed out. It looks as if it was buffered before:

$ .venv/bin/python3 swallowed-error.py
before
^CTraceback (most recent call last):
  File "<string>", line 1, in <module>
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/user/src/swallowed-error.py", line 6, in run
    1 + ''
TypeError: unsupported operand type(s) for +: 'int' and 'str'

During handling of the above exception, another exception occurred:
...

(Of course, I haven't noticed TypeError at first, because the interesting part comes before deep KeyboardInterrupt traceback.)

I would expect Trio not to hide exceptions from me :-)

Trio 0.17.0, MacOS 10.14.6, Python 3.9.0.

Full output, seems to block on `await wait_child_exiting()`

$ .venv/bin/python3 swallowed-error.py
before
^CTraceback (most recent call last):
  File "<string>", line 1, in <module>
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/user/src/swallowed-error.py", line 6, in run
    1 + ''
TypeError: unsupported operand type(s) for +: 'int' and 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/src/swallowed-error.py", line 9, in <module>
    trio.run(run)
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_core/_run.py", line 1928, in run
    raise runner.main_task_outcome.error
  File "/home/user/src/swallowed-error.py", line 7, in run
    print('after')
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_abc.py", line 261, in __aexit__
    await self.aclose()
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_subprocess.py", line 198, in aclose
    await self.wait()
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_subprocess.py", line 221, in wait
    await wait_child_exiting(self)
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_subprocess_platform/kqueue.py", line 41, in wait_child_exiting
    await _core.wait_kevent(process.pid, select.KQ_FILTER_PROC, abort)
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_core/_generated_io_kqueue.py", line 30, in wait_kevent
    return await GLOBAL_RUN_CONTEXT.runner.io_manager.wait_kevent(ident, filter, abort_func)
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_core/_io_kqueue.py", line 132, in wait_kevent
    return await _core.wait_task_rescheduled(abort)
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_core/_traps.py", line 166, in wait_task_rescheduled
    return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
  File "/home/user/src/.venv/lib/python3.9/site-packages/outcome/_impl.py", line 138, in unwrap
    raise captured_error
  File "/home/user/src/.venv/lib/python3.9/site-packages/trio/_core/_run.py", line 1178, in raise_cancel
    raise KeyboardInterrupt
KeyboardInterrupt

[shamrin]

Doing infinite loop with bash also prevents it from terminating the subprocess:

async with await trio.open_process("bash -c 'while true; do sleep 1; done'", shell=True):
      ...

[shamrin]

If you are curious about the infinite loop in the subprocess, it was on purpose. I was reimplementing concurrently tool using Trio. The tool runs several long-running commands and combines their output.

Is the silent wait expected behaviour? It was confusing to me. I think I would want the subprocess to be terminated on exceptions, rather than wait forever. Alternatively, is there a way to tell trio.open_process that I don't expect command to ever finish?

In the meantime I will learn to scroll the traceback after pushing Ctrl-C on mysterious hangs :)

[shamrin]

Ok, this seems like a duplicate of #1104 and PR #1568 is supposed to fix it.