Skip to content

v0.20.0

Latest
Compare
Choose a tag to compare
@github-actions github-actions released this 24 Sep 09:34

HyperQueue 0.20.0

New features

  • It is now possible to dynamically submit new tasks into an existing job (we call this concept "Open jobs").
    See Open jobs documentation

  • Worker streaming. Before, you could stream task stderr/stdout to the server over the network using the --log parameter of hq submit.
    This approach had various issues and was not scalable. Therefore, we have replaced this functionality with worker streaming,
    where the streaming of task output to a set of files on disk is performed by workers instead.
    This new streaming approach creates more files than original solution (where it was always one file per job),
    but the number of files stays small and independent on the number of executed tasks.
    The new architecture also allows parallel I/O writing and storing of multiple job streams in one stream handle.
    You can use worker streaming using the --stream parameter of hq submit. Check out the documentation for more information.

  • Optimization of journal size

  • Tasks' crash counters are not increased when worker is stopped by hq worker stop or by time limit.

Removed

  • Because worker streaming fully replaces original streaming, the original server streaming was removed.
    For most cases, you can rename --log to --stream and hq log to hq output-log. See the docs for more details.

Fixes

  • HQ should no longer crash while printing job info when a failed task does not have any workers
    attached (#731).

Note

  • Dashboard still not enabled in this version

Artifact summary:

  • hq-v0.20.0-*: Main HyperQueue build containing the hq binary. Download this archive to
    use HyperQueue from the command line
    .
  • hyperqueue-0.20.0-*: Wheel containing the hyperqueue package with HyperQueue Python
    bindings.