examples: add cache_primitive_desc example

[shu1chen]

Description

This C++ API example demonstrates how to implement a cache mechanism for primitive descriptors. The purpose of this cache is to prevent the repeated construction of the same primitive descriptor for identical shapes. This is particularly beneficial when creating and executing a large number of convolution primitives, as it can significantly reduce the overhead of construction of the primitive descriptors.

Key optimizations included in this example:

• Utilizing hash values of keys for faster unordered_map queries.
• A Least Recently Used (LRU) list to maintain key order and ensure frequently used primitive descriptors remain within the MAX_CACHE_SIZE limit.

An example of the output:

Conv primitive_desc newly constructed - IC: 32, Time for conv primitive_desc: 859.98 us
Conv primitive_desc newly constructed - IC: 64, Time for conv primitive_desc: 329.964 us
Conv primitive_desc newly constructed - IC: 128, Time for conv primitive_desc: 285.738 us
Conv primitive_desc cache hit - IC: 32, Time for conv primitive_desc: 3.322 us
Conv primitive_desc cache hit - IC: 64, Time for conv primitive_desc: 1.785 us
Conv primitive_desc cache hit - IC: 128, Time for conv primitive_desc: 1.258 us
Conv primitive_desc newly constructed - IC: 256, Time for conv primitive_desc: 352.398 us
Example passed on CPU.

Checklist

General

• Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• Have you formatted the code using clang-format?

[vpirogov]

oneDNN offers to options to amortize memory descriptor creation cost:

• Retaining primitive descriptors on the application side.
• Using primitive cache feature, which is enabled in default builds.

Introducing additional caching mechanism on the user side is not recommended, as it will inevitably rely on the library implementation details and thus would be fragile. It still worth investigating underlying performance issue in oneDNN that triggered this work.