EVA 02 vit_h pre training weight loading problem

[lyc200522]

Branch

main branch (mmpretrain version)

Describe the bug

size mismatch for backbone.rope.freqs_sin: copying a param with shape torch.Size([16, 16, 64]) from checkpoint, the shape in current model is torch.Size([256, 64]).
missing keys in source state_dict: backbone.ln2.weight, backbone.ln2.bias

Environment

import torch
from mmpretrain import get_model
model = get_model('vit-large-p14_eva02-pre_in21k', pretrained=True)

Other information

No response