Skip to content

DeepSparse v0.7.0
Compare
Choose a tag to compare
@jeanniefinks jeanniefinks released this 13 Sep 16:45
· 909 commits to main since this release
v0.7.0
c325595
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

New Features:

  • Operators optimized for Engine support:
    • Where*
    • Cast*
    • IntegerMatMul*
    • QLinearMatMul*
    • Gather (for scalar indices)
      *optimized only for AVX-512 support
  • Flag created to disable any batch size overrides, setting the environment variable "NM_DISABLE_BATCH_OVERRIDE=1".
  • Warnings display when emulating quantized operations on machines without VNNI instructions.
  • Support added for Python 3.9.
  • Support added for ONNX versions 1.8 - 1.10.

Changes:

  • Performance improvements made for sparse quantized transformer models.
  • Documentation updates made for examples/ultralytics-yolo to include YOLOv5.

Resolved Issues:

  • A crash could result with an uninitialized memory read. A check is now in place before trying to access it.
  • Engine output_shape functions corrected on multi-socket systems when the output dimensions are not statically known.

Known Issues:

  • BERT models with quantized embeds currently segfault on AVX2 machines. Workaround is to run on a VNNI-compatible machine.
Assets 7
Loading