FFmpeg video analytics release v0.5

• Support OpenVINO 2020.2 release
• Enable I420Blob support and utilize IE for pre-processing.
• Move IE C-API dependency to native C-API library by OpenVINO IE 2020.2.
• Add person-detection-retail-0002.json(Faster RCNN) support.
• Orchestration – providing latency for components in VA pipeline

release v0.4.2

minor change since v0.4.1:

• update dockerfile(package) to 2020.1
• add third-party-programs
• fix opencv pkgconfig

release for OpenVINO 2020.1

• Changed C API for OpenVINO release 2020.1

FFmpeg video analytics release v0.4

This release contains FFmpeg* Video Analytics plugins that bring Deep Learning Inference capabilities to open-source framework FFmpeg* and helps developers to build highly efficient and scalable video analytics applications.

This release is targeting the following platforms:

Server platforms with Intel® Xeon^™ CPU and Linux OS.
Desktop platforms with Intel® Core^™ CPU and integrated graphics and Linux* OS.
Linux OS platform with Intel® Movidius^™ Neural Compute Stick and Intel® Movidius^™ Neural Compute
Stick 2.
VCAC-A Accelerator Card

New in This Release

1. Migrate to FFmpeg v4.2 release
2. Support OpenVINO^™ 2019 R3 release and above.
3. Adopt official IE C API and use inference request callback mechanism
4. Support person re-identification model
5. Refine “metaconvert” filter to convert the inference results in AVFrame’s sidedata to a consolidated
   metadata format.
6. Replace “iemetadata” muxer with “metapublish” to mux JSON metadata to file or Kafka® streams.
7. Add one option to do filter pre-initialization before pipeline starts to avoid “first frame wait initialization”
   such as scenario of RTSP streaming, which lower the latency of the pipeline and avoid possible real
   time streaming corruption.
8. Support drawing more output info overlay with original video through OpenCV
9. Add a cpp sample to demonstrate how to use FFmpeg API in addition to video analytics filter plugins

Known Issues/limitations

This release is subject to the following limitations:

1. Running the pipeline with Gen-accelerated hardware decoding and inference on Gen (GPU), there
   might be segment fault issue happen (This issue doesn’t exist on OpenVINO^™ 2019 R1).
2. Some models from Open Model Zoo (For example, vehicle-license-plate-detection-barrier-0106.xml,
   and license-plate-recognition-barrier-0001.xml) doesn’t support batch-size setting. This could be
   workaround by setting batch-size as 1.
3. When GPU-accelerated hardware decoding is enabled in the ffmpeg command line, there might be
   issue reported that hardware surface is not available. This could be workaround by:
   1. Setting appropriate “-extra_hw_frames” numbers and “nireq” numbers for each inference filter.
   2. Setting “-threads 1” to disable multiple-threading for decoding.
4. The inference output supports limited number of pre-defined metadata format for use cases, including
   objects detection, emotion, age, gender, license plate, etc. and the format may not be identical with
   lower versions.
5. There is memory leak to run inference on GPU with OpenVINO^™ 2019 R3 release.

Release details are available in the attached release notes. Getting started is available on the Wiki or in the attached user guide.

v0.4 rc.1 pre-release

v0.4-rc.1

Add cpp sample deep-player

FFmpeg video analytics release v0.3

This release contains FFmpeg* Video Analytics plugins that bring Deep Learning Inference capabilities to open-source framework FFmpeg* and helps developers to build highly efficient and scalable video analytics applications.

New in This Release

In this release, it enables FFmpeg* analytics pipeline with the elementary inference features, including:

• Support OpenVINO^™ 2019 R2/R3 release.
• Support Intel® VCAC-A (Harker Height) accelerator card, which is one standard PCIe form factor
  accelerator card. It is equipped with Intel® Core^™ i3 CPU and Movidius^™ Myriad* X VPU.
• Enable the “throughput” performance mode, executing multiple infer requests asynchronously to
  improve the performance, on both CPU (for example, on XEON® E5) and VCAC-A (with 12 Myraid* X VPUs).
• Support cropping feature in “detect” filter, so user could specify part of the image for inference.
• Support HETERO plugin.
• Support Multi-Device plugin to run inference on multiple devices for higher throughput.
• New “metaconvert” filter to convert the inference results in AVFrame’s sidedata to pre-defined metadata
  format.
• The "detect" filter supports YOLO*v3 model for object detection.
• Support specifying configurations for different inference devices.
• Support asynchronous pre-processing in the inference filters.
• Support model reshape API (which is introduced in OpenVINO^™ 2019 R2)

Known Issues/limitations

This release is subject to the following limitations:

• Running the pipeline with Gen^™-accelerated hardware decoding and inference on Gen^™(GPU), there
  might be segfault issue happen (This issue doesn’t exist on OpenVINO^™ 2019 R1).
• Some models from Open Model Zoo (example, vehicle-license-plate-detection-barrier-0106.xml,
  license-plate-recognition-barrier-0001.xml) doesn’t support batch-size setting. This could be
  workarounded by setting batch-size as 1.
• When GPU-accelerated hardware decoding is enabled in the ffmpeg command line, there might be
  issue reported that hardware surface is not available. This could be workarounded by
  • setting appropriate “-extra_hw_frames” numbers and “nireq” numbers for each inference filter.
  • setting “-threads 1” to disable multiple-threading for decoding.
• OpenVINO^™ doesn't support C API up to 2019 R2 release, so we provided the C-API wrapper for the
  inference engine. This C API wrapper will be replaced when OpenVINO^™ support IE C API
  officially.
• The inference output supports limited number of pre-defined metadata format for use cases, including
  objects detection, emotion, age, gender, license plate, etc.

Release details are available in the attached release notes. Getting started is available on the Wiki or in the attached user guide.

FFmpeg video analytics release v0.2

New in Release v0.2

• In this initial release, it enables FFmpeg analytics pipeline with the elementary inference features, including:

1. Support OpenVINO 2019 R1.1 release.
2. Support Inference acceleration on Intel hardware (including CPU, GPU, VPU) supported by Intel®
   OpenVINOTM Toolkit.
3. The "detect" filter supports the following DL models:
   a. face-detection-adas-0001
   b. face-detection-retail-0004
   c. face-person-detection-retail-0002
   d. person-detection-retail-0013
   e. pedestrian-detection-adas-0002
   f. pedestrian-and-vehicle-detector-adas-0001
   g. vehicle-detection-adas-0002
   h. person-vehicle-bike-detection-crossroad-0078
   i. vehicle-license-plate-detection-barrier-0106
   j. mobilenet-ssd
4. The "classify" filter supports the following models:
   a. age-gender-recognition-retail-0013
   b. emotions-recognition-retail-0003
   c. face-reidentification-retail-0095
   d. vehicle-attributes-recognition-barrier-0039
   e. license-plate-recognition-barrier-0001
   f. person-attributes-recognition-crossroad-0230
5. For each inference filter, it supports “interval” option which specify the feature to do inference for every
   N frame.

Known Issues/limitations

• This release is subject to the following limitations:

1. The inference filters work on latency mode only, it doesn’t support IE’s batch processing mode and
   async API, so throughput mode is not supported in this release. Which means, running on Xeon CPU
   or accelerator cards which contains multiple VPUs can't get best performance of throughput.
2. OpenVINO doesn't support C API up to 2019 R1.1 release, so we provided the C-Wrap API for the
   inference engine at https://github.com/VCDP/FFmpeg-patch/tree/master/thirdparty. This C API of IE is
   subject to update accordingly and it would be removed when OpenVINO support IE’s C-API officially.
3. “iemetadata” format supports limited number of pre-defined metadata format for use cases, including
   objects detection, emotion, age, gender.