MMRazor Release V1.0.0rc1

Changelog of v1.0.0rc1

v1.0.0rc1 (27/10/2022)

We are excited to announce the release of MMRazor 1.0.0rc1.

Highlights

• New Pruning Framework：We have systematically refactored the Pruning module. The new Pruning module can more automatically resolve the dependencies between channels and cover more corner cases.

New Features

Pruning

• A new pruning framework is released in this release. (#311, #313)
  It consists of five core modules, including Algorithm, ChannelMutator, MutableChannelUnit, MutableChannel and DynamicOp.

• MutableChannelUnit is introduced for the first time. Each MutableChannelUnit manages all channels with channel dependency.

  from mmrazor.registry import MODELS
  
  ARCHITECTURE_CFG = dict(
      _scope_='mmcls',
      type='ImageClassifier',
      backbone=dict(type='MobileNetV2', widen_factor=1.5),
      neck=dict(type='GlobalAveragePooling'),
      head=dict(type='mmcls.LinearClsHead', num_classes=1000, in_channels=1920))
  model = MODELS.build(ARCHITECTURE_CFG)
  from mmrazor.models.mutators import ChannelMutator
  
  channel_mutator = ChannelMutator()
  channel_mutator.prepare_from_supernet(model)
  units = channel_mutator.mutable_units
  print(units[0])
  # SequentialMutableChannelUnit(
  #   name=backbone.conv1.conv_(0, 48)_48
  #   (output_related): ModuleList(
  #     (0): Channel(backbone.conv1.conv, index=(0, 48), is_output_channel=true, expand_ratio=1)
  #     (1): Channel(backbone.conv1.bn, index=(0, 48), is_output_channel=true, expand_ratio=1)
  #     (2): Channel(backbone.layer1.0.conv.0.conv, index=(0, 48), is_output_channel=true, expand_ratio=1)
  #     (3): Channel(backbone.layer1.0.conv.0.bn, index=(0, 48), is_output_channel=true, expand_ratio=1)
  #   )
  #   (input_related): ModuleList(
  #     (0): Channel(backbone.conv1.bn, index=(0, 48), is_output_channel=false, expand_ratio=1)
  #     (1): Channel(backbone.layer1.0.conv.0.conv, index=(0, 48), is_output_channel=false, expand_ratio=1)
  #     (2): Channel(backbone.layer1.0.conv.0.bn, index=(0, 48), is_output_channel=false, expand_ratio=1)
  #     (3): Channel(backbone.layer1.0.conv.1.conv, index=(0, 48), is_output_channel=false, expand_ratio=1)
  #   )
  #   (mutable_channel): SquentialMutableChannel(num_channels=48, activated_channels=48)
  # )

Our new pruning algorithm can help you develop pruning algorithm more fluently. Pelease refer to our documents PruningUserGuide for model detail.

Distillation

• Support CRD, a distillation algorithm based on contrastive representation learning. (#281)

• Support PKD, a distillation algorithm that can be used in MMDetection and MMDetection3D. #304

• Support DEIT, a classic Transformer distillation algorithm.(#332)

• Add a more powerful baseline setting for KD. (#305)

• Add MethodInputsRecorder and FuncInputsRecorder to record the input of a class method or a function.(#320)

NAS

• Support DSNAS, a nas algorithm that does not require retraining. (#226 )

Tools

• Support configurable immediate feature map visualization. (#293 )
  A useful tool is supported in this release to visualize the immediate features of a neural network. Please refer to our documents VisualizationUserGuide for more details.

Bug Fixes

• Fix the bug that FunctionXXRecorder and FunctionXXDelivery can not be pickled. (#320)

Ongoing changes

• Quantization: We are developing the basic interface of PTQ and QAT. RFC(Request for Comments) will be released soon.
• AutoSlim: AutoSlim is not yet available and is being refactored.
• Fx Pruning Tracer: Currently, the model topology can only be resolved through the backward tracer. In the future, both backward tracer and fx tracer will be supported.
• More Algorithms: BigNAS、AutoFormer、GreedyNAS and Resrep will be released in the next few versions.
• Documentation: we will add more design docs, tutorials, and migration guidance so that the community can deep dive into our new design, participate the future development, and smoothly migrate downstream libraries to MMRazor 1.x.

Contributors

A total of 12 developers contributed to this release.
Thanks @FreakieHuang @gaoyang07 @HIT-cwh @humu789 @LKJacky @pppppM @pprp @spynccat @sunnyxiaohu @wilxy @kitecats @SheffieldCao

New Contributors

• @kitecats made their first contribution in #334
• @SheffieldCao made their first contribution in #299