Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

resize handler crashes when window resized and no counters are showing #64

Closed
breathe opened this issue Dec 21, 2023 · 2 comments
Closed
Labels

Comments

@breathe
Copy link

breathe commented Dec 21, 2023

Describe the bug
When the manager still exists but the enlighten.Counter()'s have all been .close(clear=True) there's a crash that can happen inside resize handler

│ .venv/lib/python3.10/site-packages/enlighten/_manager.py:113 in │
│ _stage_resize                                                                                    │
│                                                                                                  │
│   110 │   │                                                                                      │
│   111 │   │   else:                                                                              │
│   112 │   │   │   # If not threaded, handle resize now                                           │
│ ❱ 113 │   │   │   self._resize_handler()                                                         │
│   114 │                                                                                          │
│   115 │   def _resize_handler(self):                                                             │
│   116 │   │   """                                                                                │
│                                                                                                  │
│ .venv/lib/python3.10/site-packages/enlighten/_manager.py:135 in │
│ _resize_handler                                                                                  │
│                                                                                                  │
│   132 │   │                                                                                      │
│   133 │   │   if newHeight < oldHeight:                                                          │
│   134 │   │   │   buffer.append(term.move(max(0, newHeight - self.scroll_offset), 0))            │
│ ❱ 135 │   │   │   buffer.append(u'\n' * (2 * max(self.counters.values())))                       │
│   136 │   │   elif newHeight > oldHeight and self.threaded:                                      │
│   137 │   │   │   buffer.append(term.move(newHeight, 0))                                         │
│   138 │   │   │   buffer.append(u'\n' * (self.scroll_offset - 1))                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: max() arg is an empty sequence

To Reproduce

I tried to extract a minimal repro but wasn't able to ... Logically the crash above happens where the print("crash on resize possible from here in our codebase...") message is printed ...

Found a way to repro -- code updated below

def sleep_and_print():
    import time

    time.sleep(1)
    print("text")
    import warnings

    warnings.warn("test warning")


if __name__ == "__main__":
    import enlighten, time

    total = 3
    desc = "test"
    unit = "step"

    enlighten_manager = enlighten.get_manager(
        # tried to repro with variations of these -- but can't
        # no_resize=False,
        # threaded=False,
    )

    pbar = enlighten_manager.counter(
        total=total,
        desc=desc,
        unit=unit,
        leave=False,
    )

    import concurrent.futures

    with concurrent.futures.ThreadPoolExecutor() as executor:
        # the work we do while counter exists can either launch threads or not -- added some threads here to try and induce the error
        for i in range(total):
            future = executor.submit(time.sleep, 1)
            future.result()
        pbar.update()

    print("closing")
    pbar.close(clear=True)

    print(
        "crash on resize seems to happen from here in our codebase, after closing the counter but enlighten_manager still exists ..."
    )

    with concurrent.futures.ProcessPoolExecutor() as executor:
        while True:
            future = executor.submit(sleep_and_print)
            future.result()

Failure output below

Observations

  • I seem to have to change two dimensions of the window to trigger the failure ...
  • setting threaded to True prevents repro'ing ...
  • print and warnings can be removed and error will still occur
(zephyrus-py3.10) ncohen@m1-max-toast ~/s/z/zephyrus (ben/fix-progressbar-error) [1]> python repro.py
closing
crash on resize seems to happen from here in our codebase, after closing the counter but enlighten_manager still exists ...
text
/Users/ncohen/software/zephr/zephyrus/repro.py:8: UserWarning: test warning
  warnings.warn("test warning")
text
text
text
text
text
text
text
text
text
text
text
text
Traceback (most recent call last):
  File "/Users/ncohen/software/zephr/zephyrus/repro.py", line 50, in <module>
    future.result()
  File "/Users/ncohen/.pyenv/versions/3.10.10/lib/python3.10/concurrent/futures/_base.py", line 453, in result
    self._condition.wait(timeout)
  File "/Users/ncohen/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 320, in wait
    waiter.acquire()
  File "/Users/ncohen/Library/Caches/pypoetry/virtualenvs/zephyrus--PefSXhl-py3.10/lib/python3.10/site-packages/enlighten/_manager.py", line 113, in _stage_resize
    self._resize_handler()
  File "/Users/ncohen/Library/Caches/pypoetry/virtualenvs/zephyrus--PefSXhl-py3.10/lib/python3.10/site-packages/enlighten/_manager.py", line 135, in _resize_handler
    buffer.append(u'\n' * (2 * max(self.counters.values())))
ValueError: max() arg is an empty sequence

Environment (please complete the following information):

  • Enlighten Version: 1.12.2
  • OS and version: Linux (various) MacOS Sonoma
  • Console application: [e.g. xterm, cmd, VS Code Terminal] -- iterm2
  • Special Conditions: [e.g. Running under pyinstaller] -- using matplotlib ...?

Additional context

Sorry I tried to simulate what happens in our environment to repro ... What is above is my attempt to extract a repro ...

Repro above works for me -- for additional context

The full stack trace within our application is actually within matplotlib if that provides any clues ...

(full stack trace minus our application code which calls into pandas plotting) ... Maybe the code in matplotlib is printing something?

│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_core.py:100 │
│ 0 in __call__                                                                                    │
│                                                                                                  │
│    997 │   │   │   │   │   label_name = label_kw or data.columns                                 │
│    998 │   │   │   │   │   data.columns = label_name                                             │
│    999 │   │                                                                                     │
│ ❱ 1000 │   │   return plot_backend.plot(data, kind=kind, **kwargs)                               │
│   1001 │                                                                                         │
│   1002 │   __call__.__doc__ = __doc__                                                            │
│   1003                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ __init__.py:71 in plot                                                                           │
│                                                                                                  │
│   68 │   │   │   │   ax = plt.gca()                                                              │
│   69 │   │   │   kwargs["ax"] = getattr(ax, "left_ax", ax)                                       │
│   70 │   plot_obj = PLOT_CLASSES[kind](data, **kwargs)                                           │
│ ❱ 71 │   plot_obj.generate()                                                                     │
│   72 │   plot_obj.draw()                                                                         │
│   73 │   return plot_obj.result                                                                  │
│   74                                                                                             │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:458 in generate                                                                          │
│                                                                                                  │
│    455 │   │   self._adorn_subplots()                                                            │
│    456 │   │                                                                                     │
│    457 │   │   for ax in self.axes:                                                              │
│ ❱  458 │   │   │   self._post_plot_logic_common(ax, self.data)                                   │
│    459 │   │   │   self._post_plot_logic(ax, self.data)                                          │
│    460 │                                                                                         │
│    461 │   def _args_adjust(self):                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:655 in _post_plot_logic_common                                                           │
│                                                                                                  │
│    652 │   def _post_plot_logic_common(self, ax, data):                                          │
│    653 │   │   """Common post process for each axes"""                                           │
│    654 │   │   if self.orientation == "vertical" or self.orientation is None:                    │
│ ❱  655 │   │   │   self._apply_axis_properties(ax.xaxis, rot=self.rot, fontsize=self.fontsize)   │
│    656 │   │   │   self._apply_axis_properties(ax.yaxis, fontsize=self.fontsize)                 │
│    657 │   │   │                                                                                 │
│    658 │   │   │   if hasattr(ax, "right_ax"):                                                   │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/pandas/plotting/_matplotlib/ │
│ core.py:744 in _apply_axis_properties                                                            │
│                                                                                                  │
│    741 │   │   """                                                                               │
│    742 │   │   if rot is not None or fontsize is not None:                                       │
│    743 │   │   │   # rot=0 is a valid setting, hence the explicit None check                     │
│ ❱  744 │   │   │   labels = axis.get_majorticklabels() + axis.get_minorticklabels()              │
│    745 │   │   │   for label in labels:                                                          │
│    746 │   │   │   │   if rot is not None:                                                       │
│    747 │   │   │   │   │   label.set_rotation(rot)                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1413 in   │
│ get_majorticklabels                                                                              │
│                                                                                                  │
│   1410 │                                                                                         │
│   1411 │   def get_majorticklabels(self):                                                        │
│   1412 │   │   """Return this Axis' major tick labels, as a list of `~.text.Text`."""            │
│ ❱ 1413 │   │   self._update_ticks()                                                              │
│   1414 │   │   ticks = self.get_major_ticks()                                                    │
│   1415 │   │   labels1 = [tick.label1 for tick in ticks if tick.label1.get_visible()]            │
│   1416 │   │   labels2 = [tick.label2 for tick in ticks if tick.label2.get_visible()]            │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1264 in   │
│ _update_ticks                                                                                    │
│                                                                                                  │
│   1261 │   │   """                                                                               │
│   1262 │   │   major_locs = self.get_majorticklocs()                                             │
│   1263 │   │   major_labels = self.major.formatter.format_ticks(major_locs)                      │
│ ❱ 1264 │   │   major_ticks = self.get_major_ticks(len(major_locs))                               │
│   1265 │   │   self.major.formatter.set_locs(major_locs)                                         │
│   1266 │   │   for tick, loc, label in zip(major_ticks, major_locs, major_labels):               │
│   1267 │   │   │   tick.update_position(loc)                                                     │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1602 in   │
│ get_major_ticks                                                                                  │
│                                                                                                  │
│   1599 │   │                                                                                     │
│   1600 │   │   while len(self.majorTicks) < numticks:                                            │
│   1601 │   │   │   # Update the new tick label properties from the old.                          │
│ ❱ 1602 │   │   │   tick = self._get_tick(major=True)                                             │
│   1603 │   │   │   self.majorTicks.append(tick)                                                  │
│   1604 │   │   │   self._copy_tick_props(self.majorTicks[0], tick)                               │
│   1605                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:1551 in   │
│ _get_tick                                                                                        │
│                                                                                                  │
│   1548 │   │   │   │   f"The Axis subclass {self.__class__.__name__} must define "               │
│   1549 │   │   │   │   "_tick_class or reimplement _get_tick()")                                 │
│   1550 │   │   tick_kw = self._major_tick_kw if major else self._minor_tick_kw                   │
│ ❱ 1551 │   │   return self._tick_class(self.axes, 0, major=major, **tick_kw)                     │
│   1552 │                                                                                         │
│   1553 │   def _get_tick_label_size(self, axis_name):                                            │
│   1554 │   │   """                                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:417 in    │
│ __init__                                                                                         │
│                                                                                                  │
│    414 │   __name__ = 'xtick'                                                                    │
│    415 │                                                                                         │
│    416 │   def __init__(self, *args, **kwargs):                                                  │
│ ❱  417 │   │   super().__init__(*args, **kwargs)                                                 │
│    418 │   │   # x in data coords, y in axes coords                                              │
│    419 │   │   ax = self.axes                                                                    │
│    420 │   │   self.tick1line.set(                                                               │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/axis.py:156 in    │
│ __init__                                                                                         │
│                                                                                                  │
│    153 │   │   │   grid_alpha = mpl.rcParams["grid.alpha"]                                       │
│    154 │   │   grid_kw = {k[5:]: v for k, v in kwargs.items()}                                   │
│    155 │   │                                                                                     │
│ ❱  156 │   │   self.tick1line = mlines.Line2D(                                                   │
│    157 │   │   │   [], [],                                                                       │
│    158 │   │   │   color=color, linestyle="none", zorder=zorder, visible=tick1On,                │
│    159 │   │   │   markeredgecolor=color, markersize=size, markeredgewidth=width,                │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/_api/deprecation. │
│ py:454 in wrapper                                                                                │
│                                                                                                  │
│   451 │   │   │   │   "positionally is deprecated since Matplotlib %(since)s; the "              │
│   452 │   │   │   │   "parameter will become keyword-only %(removal)s.",                         │
│   453 │   │   │   │   name=name, obj_type=f"parameter of {func.__name__}()")                     │
│ ❱ 454 │   │   return func(*args, **kwargs)                                                       │
│   455 │                                                                                          │
│   456 │   # Don't modify *func*'s signature, as boilerplate.py needs it.                         │
│   457 │   wrapper.__signature__ = signature.replace(parameters=[                                 │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/lines.py:381 in   │
│ __init__                                                                                         │
│                                                                                                  │
│    378 │   │                                                                                     │
│    379 │   │   self.set_markevery(markevery)                                                     │
│    380 │   │   self.set_antialiased(antialiased)                                                 │
│ ❱  381 │   │   self.set_markersize(markersize)                                                   │
│    382 │   │                                                                                     │
│    383 │   │   self._markeredgecolor = None                                                      │
│    384 │   │   self._markeredgewidth = None                                                      │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/matplotlib/lines.py:1254 in  │
│ set_markersize                                                                                   │
│                                                                                                  │
│   1251 │   │   │   self.stale = True                                                             │
│   1252 │   │   self._markeredgewidth = ew                                                        │
│   1253 │                                                                                         │
│ ❱ 1254 │   def set_markersize(self, sz):                                                         │
│   1255 │   │   """                                                                               │
│   1256 │   │   Set the marker size in points.                                                    │
│   1257                                                                                           │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/enlighten/_manager.py:113 in │
│ _stage_resize                                                                                    │
│                                                                                                  │
│   110 │   │                                                                                      │
│   111 │   │   else:                                                                              │
│   112 │   │   │   # If not threaded, handle resize now                                           │
│ ❱ 113 │   │   │   self._resize_handler()                                                         │
│   114 │                                                                                          │
│   115 │   def _resize_handler(self):                                                             │
│   116 │   │   """                                                                                │
│                                                                                                  │
│ /app/.venv/lib/python3.10/site-packages/enlighten/_manager.py:135 in │
│ _resize_handler                                                                                  │
│                                                                                                  │
│   132 │   │                                                                                      │
│   133 │   │   if newHeight < oldHeight:                                                          │
│   134 │   │   │   buffer.append(term.move(max(0, newHeight - self.scroll_offset), 0))            │
│ ❱ 135 │   │   │   buffer.append(u'\n' * (2 * max(self.counters.values())))                       │
│   136 │   │   elif newHeight > oldHeight and self.threaded:                                      │
│   137 │   │   │   buffer.append(term.move(newHeight, 0))                                         │
│   138 │   │   │   buffer.append(u'\n' * (self.scroll_offset - 1))                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: max() arg is an empty sequence
@breathe breathe added the bug label Dec 21, 2023
@avylove
Copy link
Contributor

avylove commented Dec 22, 2023

Good catch! Thanks for reporting!

I can reproduce with:

import signal
import sys
import time

import enlighten

# Create a counter and close it
manager = enlighten.get_manager()
with manager.counter(leave=False) as pbar:
    pass

# Resize window and trigger signal
sys.stdout.write(f'\033[8;{manager.term.height - 2};{manager.term.width}t')
signal.raise_signal(signal.SIGWINCH)

# Wait for error
time.sleep(3)

I think it's an easy fix, but will need to come up with test code. I'll try to have a new release out soon.

@avylove
Copy link
Contributor

avylove commented Dec 25, 2023

This should be fixed in 1.12.4.
Please try it out and let me know. Thanks!

@avylove avylove closed this as completed Dec 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants