[Feature] Add TIMM and HuggingFace wrappers to build classifiers from them directly.

[mzr1996]

Motivation

Pytorch-image-models and HuggingFace are famous model hubs, this PR can integrate their models in mmcls and use mmcls to train and inference models come from timm and huggingface directly.

Use cases

To train a model from TIMM

For example, to train a ResNet-50 model from timm with our vit-mae schedule.
Use config file as below

# configs/resnet/resnet50_timm.py
_base_ = ['../vision_transformer/vit-base-p16_pt-32xb128-mae_in1k-224.py']

model = dict(
    type='TimmClassifier',
    model_name='resnet50',
    loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
    train_cfg=dict(augments=[
        dict(type='Mixup', alpha=0.8),
        dict(type='CutMix', alpha=1.0)
    ]),
    _delete_=True)

To fine-tune a model from hugging-face

For example, to fine-tune a ResNet-50 model from hugging face on CIFAR dataset.
Use config file as below

# configs/resnet/resnet50_huggingface_cifar.py
_base_ = [
    '../_base_/datasets/cifar10_bs16.py',
    '../_base_/schedules/cifar10_bs128.py',
    '../_base_/default_runtime.py',
]

model = dict(
    type='HuggingFaceClassifier',
    model_name='microsoft/resnet-50',
    pretrained=True,
    loss=dict(type='CrossEntropyLoss', loss_weight=1.0),
    num_labels=10,
    ignore_mismatched_sizes=True,
)

To inference a TIMM model

>>> from mmengine import Config
>>> from mmcls.apis import init_model, inference_model
>>> from mmcls.utils import register_all_modules
>>> from mmcls.datasets.categories import IMAGENET_CATEGORIES
>>> register_all_modules()
>>> # Use data pipeline from the resnet config
>>> cfg = Config.fromfile("./configs/resnet/resnet50_8xb32_in1k.py")
>>> cfg.model = dict(type='TimmClassifier', model_name='resnet50', pretrained=True)
>>> model = init_model(cfg)
>>> pred_label = inference_model(model, 'demo/demo.JPEG')['pred_label']
>>> print(IMAGENET_CATEGORIES[pred_label])
'sea snake'

To inference a HuggingFace model

>>> from mmengine import Config
>>> from mmcls.apis import init_model, inference_model
>>> from mmcls.utils import register_all_modules
>>> from mmcls.datasets.categories import IMAGENET_CATEGORIES
>>> register_all_modules()
>>> # Use data pipeline from the resnet config
>>> cfg = Config.fromfile("./configs/resnet/resnet50_8xb32_in1k.py")
>>> cfg.model = dict(type='HuggingFaceClassifier', model_name='microsoft/resnet-50', pretrained=True)
>>> model = init_model(cfg)
>>> pred_label = inference_model(model, 'demo/demo.JPEG')['pred_label']
>>> print(IMAGENET_CATEGORIES[pred_label])
'sea snake'

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• CLA has been signed and all committers have signed the CLA in this PR.

[codecov]

Codecov Report

Base: 0.02% // Head: 90.19% // Increases project coverage by +90.17% 🎉

Coverage data is based on head (617b741) compared to base (b8b31e9).
Patch has no changes to coverable lines.

Additional details and impacted files

@@             Coverage Diff              @@
##           dev-1.x    #1102       +/-   ##
============================================
+ Coverage     0.02%   90.19%   +90.17%     
============================================
  Files          121      140       +19     
  Lines         8217    10383     +2166     
  Branches      1368     1645      +277     
============================================
+ Hits             2     9365     +9363     
+ Misses        8215      798     -7417     
- Partials         0      220      +220     

Flag	Coverage Δ
unittests	90.19% <ø> (+90.17%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcls/apis/inference.py	0.00% <0.00%> (ø)
mmcls/datasets/transforms/compose.py
mmcls/models/necks/reduction.py	100.00% <0.00%> (ø)
mmcls/models/retrievers/__init__.py	100.00% <0.00%> (ø)
mmcls/models/backbones/mvit.py	92.46% <0.00%> (ø)
mmcls/models/backbones/edgenext.py	95.20% <0.00%> (ø)
mmcls/structures/utils.py	77.77% <0.00%> (ø)
mmcls/models/backbones/mobileone.py	94.47% <0.00%> (ø)
mmcls/models/backbones/swin_transformer_v2.py	89.47% <0.00%> (ø)
mmcls/models/heads/arcface_head.py	85.48% <0.00%> (ø)
... and 130 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[Ezra-Yu]

LGTM.