Questions About the feature_visualization Function

[Zengyf-CVer]

@glenn-jocher

This is my first question:

I tried the feature_visualization function and found some problems.
First, when I run this detection code, two directories are generated in runs/features: exp and exp2, as shown in the figure:

python detect.py --weights ./weight/yolov5s.pt --source ./train2017/000000000081.jpg

Display in the shell:

The picture above is the first saved file, in exp.
In the display of the shell, there is another one that processes the input image, as shown in the figure:

So I checked the specific feature map generated:
In exp:

In exp2:

So what I want to ask is, why are two directories exp and exp2 generated at the same time? What do these two directories represent?

This is my second question:

I checked the feature_visualization function in plots.py, there are two valuable codes:

yolov5/utils/plots.py

Line 465 in 33202b7

blocks = torch.chunk(x, channels, dim=1) # block by channel dimension

and

yolov5/utils/plots.py

Line 468 in 33202b7

feature = transforms.ToPILImage()(blocks[i].squeeze())

I am not sure, whether the generated two directories exp and exp2 are related to the models type I set?

yolov5/models/yolo.py

Lines 158 to 159 in 33202b7

	if feature_vis and m.type == 'models.common.SPP':
	feature_visualization(x, m.type, m.i)

What else are settings like models.common.SPP?
I am not sure about the specific architecture of yolov5. Is there any relevant architecture diagram? Like this, but the official version?

Looking forward to your reply, thank you very much.

[github-actions]

👋 Hello @Zengyf-CVer, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@Zengyf-CVer the first features are created when the model is run once to initialize GPUs for consistent speeds in later images:

yolov5/detect.py

Lines 90 to 92 in 33202b7

	# Run inference
	if device.type != 'cpu':
	model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters()))) # run once

We've made visualizing YOLOv5 🚀 architectures super easy. There are two main ways:

model.yaml

Each model has a corresponding yaml file that displays the model architecture. Here is YOLOv5s, defined by yolov5s.yaml:

yolov5/models/yolov5s.yaml

Lines 12 to 48 in 831773f

	# YOLOv5 backbone
	backbone:
	# [from, number, module, args]
	[[-1, 1, Focus, [64, 3]], # 0-P1/2
	[-1, 1, Conv, [128, 3, 2]], # 1-P2/4
	[-1, 3, C3, [128]],
	[-1, 1, Conv, [256, 3, 2]], # 3-P3/8
	[-1, 9, C3, [256]],
	[-1, 1, Conv, [512, 3, 2]], # 5-P4/16
	[-1, 9, C3, [512]],
	[-1, 1, Conv, [1024, 3, 2]], # 7-P5/32
	[-1, 1, SPP, [1024, [5, 9, 13]]],
	[-1, 3, C3, [1024, False]], # 9
	]
	# YOLOv5 head
	head:
	[[-1, 1, Conv, [512, 1, 1]],
	[-1, 1, nn.Upsample, [None, 2, 'nearest']],
	[[-1, 6], 1, Concat, [1]], # cat backbone P4
	[-1, 3, C3, [512, False]], # 13
	[-1, 1, Conv, [256, 1, 1]],
	[-1, 1, nn.Upsample, [None, 2, 'nearest']],
	[[-1, 4], 1, Concat, [1]], # cat backbone P3
	[-1, 3, C3, [256, False]], # 17 (P3/8-small)
	[-1, 1, Conv, [256, 3, 2]],
	[[-1, 14], 1, Concat, [1]], # cat head P4
	[-1, 3, C3, [512, False]], # 20 (P4/16-medium)
	[-1, 1, Conv, [512, 3, 2]],
	[[-1, 10], 1, Concat, [1]], # cat head P5
	[-1, 3, C3, [1024, False]], # 23 (P5/32-large)
	[[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5)
	]

TensorBoard Graph

Simply start training a model, and then view the TensorBoard Graph for an interactive view of the model architecture. This example shows YOLOv5s viewed in our Notebook –

# Tensorboard
%load_ext tensorboard
%tensorboard --logdir runs/train

# Train YOLOv5s on COCO128 for 3 epochs
!python train.py --weights yolov5s.pt --epochs 3

[glenn-jocher]

@Zengyf-CVer good news 😃! Your original issue may now be fixed ✅ in PR #3920. This is a complete revamp of feature visualization with many fixes and improvements.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload with model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!