Menus not resolving properly

[CommanderCero]

🐛 Bug

Note, I am unsure if this is an issue with NLE, Minihack, or NetHack itself.
I am reporting it here since I've only managed to reproduce this issue with MiniHack so far:
It seems like sometimes menus that ask for key-presses aren't properly resolved.
For example drinking from a fountain will ask "Drink from the fountain? [yn] (n)".
Pressing "y" should resolve it for example with "The cool draught refreshes you".
However, sometimes the same message "Drink from the fountain? [yn] (n)" just appears again.
The odd thing is that I've checked the internal flags if there is a menu open, and they are set to false.
Note that I've seen this happen with other instances like kicking open a door, or being asked if I want to attack a friendly monster.

I am working on an agent that plays NetHack using Large Language Models.
As such, these messages confuse my agent.

To Reproduce

1. Run the code below to reproduce my example from above. Note the odd seeding is just a result of my code.
2. In the for loop replace "error_action_sequence" with "working_action_sequence" to get the expected behaviour.
3. Using the error_action_sequence the last two outputs will be
   Drink from the fountain? [yn] (n) True False False
   Drink from the fountain? [yn] (n) y False False False
4. Using the working_action_sequence the last two outputs are correct.
   Drink from the fountain? [yn] (n) True False False
   The cool draught refreshes you. False False False

Note how in both cases all menu-flags are false at the end.
I also noticed that in the "error" case there is an additional "y" in the message as if I am typing.
However, I am unable to reproduce such a behaviour on alt.org/nethack.
Additionally, I am unable to confirm the message, since any additional input will be treated as if there is no menu.

import sys
from minihack.base import MiniHack
from gymnasium.utils import seeding
from nle.env.base import nethack

des_file = """
MAZE: "mylevel", ' '
FLAGS:hardfloor
INIT_MAP: solidfill,' '
GEOMETRY:center,center
MAP
--------     --------
|......|     |......|
|......|#####|......|
|......|     |......|
--------     --------
    #
    #
--------
|......|
|......|
|......|
--------
ENDMAP
REGION:(1,1,6,3),lit,"ordinary"
REGION:(14,1,19,3),lit,"ordinary"
REGION:(1,8,6,11),lit,"ordinary"

DOOR:closed,(4,4)
DOOR:closed,(7,2)
DOOR:closed,(4,7)
DOOR:closed,(13,2)

$all_rooms = selection: fillrect(1,1,6,3) & fillrect(1,8,6,10) & fillrect(14,1,19,3)
OBJECT: "full healing", rndcoord($all_rooms)
FOUNTAIN: rndcoord($all_rooms)
"""

_np_random, seed = seeding.np_random(474862)
env = MiniHack(
    des_file=des_file,
    character="@",
    actions=nethack.ACTIONS
)

seed = 474862
env.seed(core=seed, disp=seed)
obs = env.reset()
seed = _np_random.integers(sys.maxsize)
env.seed(core=seed, disp=seed)
obs = env.reset()

working_action_sequence = [3, 7, 0, 0, 0, 0, 0, 0, 0, 3, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 64, 7]
error_action_sequence = [3, 7, 0, 0, 0, 0, 0, 0, 0, 1, 4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 64, 7]

for i, action in enumerate(error_action_sequence):
    obs, reward, done, info = env.step(action)
    waiting_for_yn = bool(env.last_observation[env._internal_index][1])
    waiting_for_line = bool(env.last_observation[env._internal_index][2])
    waiting_for_space = bool(env.last_observation[env._internal_index][3])
    print("".join([chr(c) for c in obs["message"] if c != 0]), waiting_for_yn, waiting_for_line, waiting_for_space)

Environment

Collecting environment information...
/opt/conda/lib/python3.8/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
MiniHack version: 0.1.5
NLE version: 0.9.0
Gym version: 0.26.2
PyTorch version: 1.8.0a0+1606899
Is debug build: No
CUDA used to build PyTorch: 11.1

OS: Ubuntu 20.04.1 LTS
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
CMake version: version 3.18.2

Python version: 3.8
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] pytorch-transformers==1.1.0
[pip3] torch==1.8.0a0+1606899
[pip3] torchtext==0.9.0a0
[pip3] torchvision==0.9.0a0
[conda] magma-cuda110 2.5.2 5 local
[conda] mkl 2019.4 243
[conda] mkl-include 2019.4 243
[conda] nomkl 3.0 0
[conda] pytorch-transformers 1.1.0 pypi_0 pypi
[conda] torch 1.8.0a0+1606899 pypi_0 pypi
[conda] torchtext 0.9.0a0 pypi_0 pypi
[conda] torchvision 0.9.0a0 pypi_0 pypi