Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preparing TPC for weights per-attribute quantization #925

Merged
merged 36 commits into from
Jan 31, 2024

Conversation

ofirgo
Copy link
Collaborator

@ofirgo ofirgo commented Jan 21, 2024

Pull Request Description:

Preparing the ground for all weight attributes quantization in the TPC:

  • Weights configuration in OpQuantizationConfig are extracted to a new class named AttributeQuantizationConfig which holds the weights quantization configuration per-attribute.
  • Each OpQuantizationConfig now includes a default_attribute_config and an attributes_config_mapping which maps an attribute to the attribute's specific quantization configuration. The default config is then used to quantize all non-specified weight attributes.
  • By default, we add Kernel and Bias attributes to all our TP models base op config. The kernel is quantized similarly to the way we have quantized weights so far. The bias quantization is disabled.
  • In addition, currently, the default attribute config disables weights quantization (it would change in a future change to enable multi-attribute quantization.

To enable attribute quantization with specific config per attribute, we created a mapping mechanism between a general attribute name (e.g., "KERNEL_ATTR") to this attribute name in the framework (e.g., "kernel" for keras Conv2d or "depthwise_kernel" for keras DepthwiseConv2d). This mapping is activated once initializing the TPC (in the framework).

Additional modifications:

  • In MCT core side - when building the nodes candidates config we now need to consider config per attribute. This is not yet implemented - this PR just prepared the ground for implementing it in a future change, and currently, just extracting the kernel config from the given op configs in the TPC.
  • This TPC change required many modifications in tests and tests TPCs.

Checklist before requesting a review:

  • I set the appropriate labels on the pull request.
  • I have added/updated the release note draft (if necessary).
  • I have updated the documentation to reflect my changes (if necessary).
  • All function and files are well documented.
  • All function and classes have type hints.
  • There is a licenses in all file.
  • The function and variable names are informative.
  • I have checked for code duplications.
  • I have added new unittest (if necessary).

Ofir Gordon added 2 commits January 24, 2024 13:26
… modify tests TPCs and update the function that doing the connection)
@ofirgo ofirgo merged commit d8e441d into sony:main Jan 31, 2024
24 checks passed
lior-dikstein pushed a commit that referenced this pull request Feb 25, 2024
* Weights configuration in OpQuantizationConfig are extracted to a new class named AttributeQuantizationConfig which holds the weights quantization configuration per-attribute.
* Each OpQuantizationConfig now includes a default_attribute_config and an attributes_config_mapping which maps an attribute to the attribute's specific quantization configuration. The default config is then used to quantize all non-specified weight attributes.
* By default, we add Kernel and Bias attributes to all our TP models base op config. The kernel is quantized similarly to the way we have quantized weights so far. The bias quantization is disabled.
* To enable attribute quantization with specific config per attribute, we created a mapping mechanism between a general attribute name (e.g., "KERNEL_ATTR") to this attribute name in the framework

---------

Co-authored-by: Ofir Gordon <Ofir.Gordon@altair-semi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants