Some IPython commands are not getting executed

[akhmerov]

I noticed that nbclient does not seem to include the outputs produced by IPython's special ? command. To reproduce use the following script:

import nbformat
import nbclient

notebook = nbformat.v4.new_notebook()
notebook.cells.append(nbformat.v4.new_code_cell("?"))
notebook.cells.append(nbformat.v4.new_code_cell("print('hi')"))
print(nbclient.execute(notebook))

With IPython 8.0.0 and nbclient 0.5.9 I observe the following output:

{'nbformat': 4,
 'nbformat_minor': 5,
 'metadata': {'language_info': {'name': 'python',
   'version': '3.9.9',
   'mimetype': 'text/x-python',
   'codemirror_mode': {'name': 'ipython', 'version': 3},
   'pygments_lexer': 'ipython3',
   'nbconvert_exporter': 'python',
   'file_extension': '.py'}},
 'cells': [{'id': '82a6d8e4',
   'cell_type': 'code',
   'metadata': {'execution': {'iopub.status.busy': '2022-01-16T19:10:10.888868Z',
     'iopub.execute_input': '2022-01-16T19:10:10.894205Z',
     'iopub.status.idle': '2022-01-16T19:10:10.897155Z',
     'shell.execute_reply': '2022-01-16T19:10:10.897833Z'}},
   'execution_count': 1,
   'source': '?',
   'outputs': []},
  {'id': 'f872d7a2',
   'cell_type': 'code',
   'metadata': {'execution': {'iopub.status.busy': '2022-01-16T19:10:10.901294Z',
     'iopub.execute_input': '2022-01-16T19:10:10.901893Z',
     'shell.execute_reply': '2022-01-16T19:10:10.904409Z',
     'iopub.status.idle': '2022-01-16T19:10:10.904888Z'}},
   'execution_count': 2,
   'source': "print('hi')",
   'outputs': [{'output_type': 'stream', 'name': 'stdout', 'text': 'hi\n'}]}]}

Notice that the cell with ? has no outputs and the cell with print('hi') does. I also confirmed that in JupyterLab 3.2.8 the outputs are present in both cases, and therefore I suspect that the problem is on the nbclient side, but I am naturally not 100% sure.

[davidbrochart]

• A code cell with print('hi') has its outputs in a stream message on the IOPub channel.
• A code cell with 1+2 has its outputs in an execute_result message on the IOPub channel.
• A code cell with ? has its outputs in an execute_reply message on the shell channel.

nbclient uses output_from_msg from nbformat, which doesn't handle messages of type execute_reply (and the docstring says it only handles messages on the IOPub channel).
Maybe it's just a matter of handling this type of messages in output_from_msg.
I opened jupyter/nbformat#242.