[Feature] Support EVA

[okotaku]

Motivation

paper: https://arxiv.org/abs/2211.07636
code: https://github.com/baaivision/EVA
issue: #1219

Modification

In addition, I fixed a bug in BEiT that caused ffn's add_identity to be True. fixed in #1234
And fixed sklearn deprecated error.

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• CLA has been signed and all committers have signed the CLA in this PR.

[codecov]

Codecov Report

Base: 0.02% // Head: 88.61% // Increases project coverage by +88.58% 🎉

Coverage data is based on head (c3c6bd8) compared to base (b8b31e9).
Patch has no changes to coverable lines.

❗ Current head c3c6bd8 differs from pull request most recent head 9f54c80. Consider uploading reports for the commit 9f54c80 to get more accurate results

Additional details and impacted files

@@             Coverage Diff              @@
##           dev-1.x    #1239       +/-   ##
============================================
+ Coverage     0.02%   88.61%   +88.58%     
============================================
  Files          121      151       +30     
  Lines         8217    11811     +3594     
  Branches      1368     1893      +525     
============================================
+ Hits             2    10466    +10464     
+ Misses        8215     1043     -7172     
- Partials         0      302      +302     

Flag	Coverage Δ
unittests	88.61% <ø> (+88.58%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcls/datasets/transforms/compose.py
mmcls/models/backbones/mobilevit.py	91.15% <0.00%> (ø)
mmcls/utils/progress.py	66.66% <0.00%> (ø)
mmcls/models/backbones/beit.py	57.06% <0.00%> (ø)
mmcls/models/retrievers/__init__.py	100.00% <0.00%> (ø)
mmcls/models/backbones/efficientformer.py	95.08% <0.00%> (ø)
mmcls/apis/model.py	81.60% <0.00%> (ø)
mmcls/models/backbones/edgenext.py	95.20% <0.00%> (ø)
mmcls/models/backbones/revvit.py	64.39% <0.00%> (ø)
mmcls/models/classifiers/timm.py	25.97% <0.00%> (ø)
... and 142 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[Ezra-Yu]

the official repo https://github.com/baaivision/EVA has released MAE style EVA model, Could you convert them all together?

[okotaku]

@Ezra-Yu
You mean 'eva_psz14' and 'eva_psz14to16' ?
https://github.com/baaivision/EVA/tree/master/eva

I thought this convert script was fine for its, any improvements?

[Ezra-Yu]

@Ezra-Yu You mean 'eva_psz14' and 'eva_psz14to16' ? https://github.com/baaivision/EVA/tree/master/eva

this https://github.com/baaivision/EVA/tree/master/eva#eva-l-learning-better-mim-representations-from-eva-clip; eva_l_psz14 and eva_l_psz14_21k_ft;
it looks like the convert script works fine for that, only need to add some configs for that.

[okotaku]

I got it.

[okotaku]

@Ezra-Yu Done!

[mzr1996]

@okotaku Hello, I have uploaded the checkpoints and added some pre-trained models in the metafile. For those pre-trained models, I named the config files as headless and those config files have only model, but no dataset nor optim_wrapper.
With this design, you can access these pre-trained models from mmcls.get_model API, like

>>> import mmcls
>>> import torch
>>> extractor = mmcls.get_model('eva-l-p14_3rdparty-mim_in21k', pretrained=True)
>>> inputs = torch.rand(1, 3, 224, 224)
>>> outs = extractor.extract_feat(inputs)  # Or `extractor(inputs)`
>>> print(outs[0].shape)
torch.Size([1, 1024])

What do you think about it?

[okotaku]

@mzr1996 It looks good, thank you.

[mzr1996]

LGTM