Give names to configs in e2e test framework

[pzread]

This change gives the name to config objects in e2e test framework (especially the ones with unique IDs).

This is required to generate benchmark names in the new benchmark suites, and later will be used and shared across the benchmark tools and uploaded to the perf dashboard. Currently we generate the benchmark names in the benchmark tools, which is not an ideal place.

They can be also used in the CMake comments/names of the module generation rules to give friendly information about where the rules come from (see "Tracability in E2E Test Artifacts" in #12215 (comment))

Right now it generates the names from config fields such as architectures and tags. Ideally I want users to manually assign a more concise name for each config (we already do that to name the unique ID constants, might be able to reuse). But currently the perf dashboard heavily relies on the benchmark names to filter benchmarks, so the tags need to be part of the name. We need a dashboard to support filtering on metadata before we can move to a more concise naming schema.

LMLWYT about the naming schema

Name format:

ImportedModel:
<model_name>(<import_config_name>)

CompileConfig:
[<compile_target_arch>,...][<compile_config_tag>,...]

ModuleExecutionConfig:
<runtime_driver>(<runtime_loader>)[<run_config_tag>,...]

DeviceSpec:
<device_name>[<device_tag>,...]

ModuleGenerationConfig
<model_name>(<import_config_name>) [<compile_target_arch>,...][<compile_config_tag>,...] 

E2EModelRunConfig
<model_name>(<import_config_name>) [<compile_target_arch>,...][<compile_config_tag>,...] <runtime_driver>(<runtime_loader>)[<run_config_tag>,...] with <input_data_name> @ <device_name>[<device_tag>,...]

Execution benchmark names:

BertForMaskedLMTF(tf_v2) [x86_64-cascadelake-linux_gnu-llvm_cpu][default-flags] local_sync(embedded_elf)[full-inference,default-flags] with zeros @ c2-standard-16[cpu]

BertLargeTF(tf_v1) [x86_64-cascadelake-linux_gnu-llvm_cpu][experimental-flags,fuse-padding] local_task(embedded_elf)[8-thread,full-inference,default-flags] with zeros @ c2-standard-16[cpu]

MobileBertSquad_int8(tflite) [armv8.2-a-generic-linux_android29-llvm_cpu][experimental-flags,mmt4d,dotprod] local_task(embedded_elf)[4-thread,full-inference,default-flags] with zeros @ Pixel-6-Pro[big-core]

# The current longest name (218 characters)

MobileBertSquad_fp16(tflite) [valhall-mali-vulkan_android31-vulkan_spirv][experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16] vulkan(none)[full-inference,experimental-flags] with zeros @ Pixel-6-Pro[gpu]

Compilation benchmark names:

EfficientNet_int8(tflite) [x86_64-cascadelake-linux_gnu-llvm_cpu][experimental-flags,fuse-padding,compile-stats]

MobileBertSquad_fp16(tflite) [valhall-mali-vulkan_android31-vulkan_spirv][experimental-flags,fuse-padding,repeated-kernel,demote-f32-to-f16,compile-stats]

[GMNGeoffrey]

Let's just make sure that these aren't being used as keys. We can change the names without modifying the server, correct?

[github-actions]

Abbreviated Benchmark Summary

@ commit dd9191c6f8fff98337b7e441496592ce480b706b (vs. base 713f9851eda694abe664103bcafafcf846390546)

No improved or regressed benchmarks 🏖️

No improved or regressed compilation metrics 🏖️

For more information:

• Full benchmark result tables
• Build

Source Workflow Run

[pzread]

Let's just make sure that these aren't being used as keys. We can change the names without modifying the server, correct?

That's correct.