Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asyncio (mode) debugger support #362

Draft
wants to merge 25 commits into
base: ctx_cancel_semantics_and_overruns
Choose a base branch
from

Commits on Jun 15, 2023

  1. Initial module import from piker.data._sharemem

    More or less a verbatim copy-paste minus some edgy variable naming and
    internal `piker` module imports. There is a bunch of OHLC related
    defaults that need to be dropped and we need to adjust to an optional
    dependence on `numpy` by supporting shared lists as per the mp docs.
    goodboy committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    9716d86 View commit details
    Browse the repository at this point in the history
  2. Add ShmList wrapping the stdlib's ShareableList

    First attempt at getting `multiprocessing.shared_memory.ShareableList`
    working; we wrap the stdlib type with a readonly attr and a `.key` for
    cross-actor lookup. Also, rename all `numpy` specific routines to have
    a `ndarray` suffix in the func names.
    goodboy committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    7147729 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c32b21b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    339d787 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    edb82fd View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    1713ecd View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b52ff27 View commit details
    Browse the repository at this point in the history
  8. Parametrize rw test with variable frame sizes

    Demonstrates fixed size frame-oriented reads by the child where the
    parent only transmits a "read" stream msg on "frame fill events" such
    that the child incrementally reads the shm list data (much like in
    a real-time-buffered streaming system).
    goodboy committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    a9fc4c1 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    e0bf964 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    f9a84f0 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    4f442ef View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    f745da9 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2023

  1. Configuration menu
    Copy the full SHA
    ebcb275 View commit details
    Browse the repository at this point in the history
  2. Add a debug-mode-breakpoint-causes-hang case!

    Only found this by luck more or less (while working on something in
    a client project) and it turns out we can actually get to (yet another)
    hang state where SIGINT will be ignored by the root actor on teardown..
    
    I've added all the necessary logic flags to reproduce. We obviously need
    a follow up bug issue and a test suite to replicate!
    
    It appears as though the following are required based on very light
    tinkering:
    - infected asyncio mode active
    - debug mode active
    - the `trio` context must breakpoint *before* `.started()`-ing
    - the `asyncio` must **not** error
    goodboy committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    ee87cf0 View commit details
    Browse the repository at this point in the history
  3. First proto: use greenback for sync func breakpointing

    This works now for supporting a new `tractor.pause_from_sync()`
    `tractor`-aware-replacement for `Pdb.set_trace()` from sync functions
    which are also scheduled from our runtime. Uses `greenback` to do all
    the magic of scheduling the bg `tractor._debug._pause()` task and
    engaging the normal TTY locking machinery triggered by `await
    tractor.breakpoint()`
    
    Further this starts some public API renaming, making a switch to
    `tractor.pause()` from `.breakpoint()` which IMO much better expresses
    the semantics of the runtime intervention required to suffice
    multi-process "breakpointing"; it also is an alternate name for the same
    in computer science more generally: https://en.wikipedia.org/wiki/Breakpoint
    It also avoids using the same name as the `breakpoint()` built-in which
    is important since there **is alot more going on** when you call our
    equivalent API.
    
    Deats of that:
    - add deprecation warning for `tractor.breakpoint()`
    - add `tractor.pause()` and a shorthand, easier-to-type, alias `.pp()`
      for "pause-point" B)
    - add `pause_from_sync()` as the new `breakpoint()`-from-sync-function
      hack which does all the `greenback` stuff for the user.
    
    Still TODO:
    - figure out where in the runtime and when to call
      `greenback.ensure_portal()`.
    - fix the frame selection issue where
      `trio._core._ki._ki_protection_decorator:wrapper` seems to be always
      shown on REPL start as the selected frame..
    goodboy committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    fc56971 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2023

  1. Configuration menu
    Copy the full SHA
    ac695a0 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. Configuration menu
    Copy the full SHA
    565d7c3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    46972df View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    98a7326 View commit details
    Browse the repository at this point in the history
  4. Fix frame-selection display on first REPL entry

    For whatever reason pdb(p), and in general, will show the frame of the
    *next* python instruction/LOC on initial entry (at least using
    `.set_trace()`), as such remove the `try/finally` block in the sync
    code entrypoint `.pause_from_sync()`, and also since doesn't seem like
    we really need it anyway.
    
    Further, and to this end:
    - enable hidden frames support in our default config.
    - fix/drop/mask all the frame ref-ing/mangling we had prior since it's no
      longer needed as well as manual `Lock` releasing which seems to work
      already by having the `greenback` spawned task do it's normal thing?
    - move to no `Union` type annots.
    - hide all frames that can add "this is the runtime confusion" to
      traces.
    goodboy committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    4ace8f6 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b36b3d5 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Make NamespacePath work on object refs

    Detect if the input ref is a non-func (like an `object` instance) in
    which case grab its type name using `type()`. Wrap all the name-getting
    into a new `_mk_fqpn()` static meth: gets the "fully qualified path
    name" and returns path and name in tuple; port other methds to use it.
    Refine and update the docs B)
    goodboy committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    bee2c36 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. Configuration menu
    Copy the full SHA
    e03bec5 View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Configuration menu
    Copy the full SHA
    1102843 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    22c14e2 View commit details
    Browse the repository at this point in the history