Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Update analyze_results.py for dev-1.x #1071

Merged
merged 5 commits into from
Oct 11, 2022

Conversation

okotaku
Copy link
Collaborator

@okotaku okotaku commented Oct 3, 2022

Motivation

Update analyze_results.py for mmcls v1.0.

Results

mim test mmcls configs/resnet/resnet18_8xb32_in1k.py --checkpoint https://download.openmmlab.com/mmclassification/v0/resnet/resnet18_8xb32_in1k_20210831-fbbb1da6.pth --dump result.pkl
mim run mmcls analyze_results configs/resnet/resnet18_8xb32_in1k.py result.pkl --out-dir show

ILSVRC2012_val_00002087 JPEG_0

ILSVRC2012_val_00001520 JPEG_0

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
  • CLA has been signed and all committers have signed the CLA in this PR.

@okotaku okotaku changed the title [Enhancement] Update analyze_results.py for mmcls v1.0 [Enhancement] Update analyze_results.py for dev-1.x Oct 3, 2022
@Ezra-Yu Ezra-Yu self-requested a review October 8, 2022 03:17
result['gt_label']).set_pred_label(
result['pred_label']).set_pred_score(
torch.Tensor(result['pred_scores']))
img = mmcv.imread(result['img_path'], channel_order='rgb')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some images are too small to visualize, better to provide --rescale-factor option to rescale images before visualizing them. Please refers to https://github.com/open-mmlab/mmclassification/blob/ae37d7fd276d983eba3acc2676a2f6e2d720bd69/tools/visualizations/browse_dataset.py#L64-L69

Comment on lines 77 to 83
output['img_path'] = outputs[i]['img_path']
output['filename'] = Path(outputs[i]['img_path']).name
output['gt_label'] = int(outputs[i]['gt_label']['label'][0])
output['pred_score'] = float(
torch.max(outputs[i]['pred_label']['score']).item())
output['pred_scores'] = outputs[i]['pred_label']['score'].tolist()
output['pred_label'] = int(outputs[i]['pred_label']['label'][0])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all datasets have the img_path key. For example, the CIFAR dataset directly loads all images during initialization and only has img key.
Better to use sample_idx to get the data info from the dataset directly.

Suggested change
output['img_path'] = outputs[i]['img_path']
output['filename'] = Path(outputs[i]['img_path']).name
output['gt_label'] = int(outputs[i]['gt_label']['label'][0])
output['pred_score'] = float(
torch.max(outputs[i]['pred_label']['score']).item())
output['pred_scores'] = outputs[i]['pred_label']['score'].tolist()
output['pred_label'] = int(outputs[i]['pred_label']['label'][0])
output['sample_idx'] = outputs[i]['sample_idx']
output['gt_label'] = outputs[i]['gt_label']['label']
output['pred_score'] = float(
torch.max(outputs[i]['pred_label']['score']).item())
output['pred_scores'] = outputs[i]['pred_label']['score']
output['pred_label'] = outputs[i]['pred_label']['label']

And in the save_imgs:

    for result in results:
        data_sample = ClsDataSample()\
            .set_gt_label(result['gt_label'])\
            .set_pred_label(result['pred_label'])\
            .set_pred_score(result['pred_scores'])
        data_info = dataset.get_data_info(result['sample_idx'])
        if 'img' in data_info:
            img = data_info['img']
            name = str(result['sample_idx'])
        elif 'img_path' in data_info:
            img = mmcv.imread(data_info['img_path'], channel_order='rgb')
            name = Path(data_info['img_path']).name
        else:
            raise ValueError('Cannot load images from the dataset infos.')
        if rescale_factor is not None:
            img = mmcv.imrescale(img, rescale_factor)
        vis.add_datasample(name, img, data_sample)

        for k, v in result.items():
            if isinstance(v, torch.Tensor):
                result[k] = v.tolist()

@mzr1996 mzr1996 merged commit 9bc5874 into open-mmlab:dev-1.x Oct 11, 2022
mzr1996 pushed a commit to mzr1996/mmpretrain that referenced this pull request Nov 24, 2022
* update analyze_results

* lint

* add --rescale-factor and fix filename logic

* lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants