Yolov5 classification model support #1082

xiang-wuu · 2022-08-22T15:52:26Z

This PR will update the Yolov5 TRT network to support Yolov5 classification model serialization and inference as per the recent yolov5-v6.2, this PR can be linked to issue #1077, and cover below tasks.

Update gen_wts.py script to support Yolov5 classification model export.
Update build_engine TRT network definition to initially support Yolov5s model
Test the Yolov5s based classification model serialization and infer testing.
Update build_engine TRT network definition to support all models
Validate the infer results of PyTorch with TRT

wang-xinyu · 2022-08-23T03:09:59Z

@xiang-wuu Awesome, looking forward to your implementation.

wang-xinyu · 2022-08-24T07:08:55Z

@xiang-wuu Hi, for classification model, can you write a separate .cpp file with main() function, and build another executable file? That will simplify the logic.

yolov5/yolov5.cpp

…l alongwith existing detection model.

…he infer script.

xiang-wuu · 2022-08-24T16:20:25Z

Update build_engine TRT network definition to support all models

As the p6 scale models are not provided officially for Yolov5 classification models, so this can be done afterwards once the support is added. The normal scale models are working fine.

…, hence hardcoded value provided to support all model varients

xiang-wuu · 2022-08-25T10:45:18Z

Evaluation Metrics

Model	Size	Framework/IE	Architecture(GPU/CPU)	Precision	Accuracy(Top-1)	F1-score	Latency
YOLOv5s-cls	224	TensorRT	Nvidia GTX 1060	FP32	69.892	0.6952	1.40 ms
YOLOv5s-cls	224	PyTorch	Nvidia GTX 1060	FP32	71.724	0.7136	3.07 ms

Validate the infer results of PyTorch with TRT

As per the last subtask, the model is been inferred separately using PyTorch and TRT over 50K validation dataset from imagenet, also the time complexity is averaged over 5K dry runs for both model implementations. The slight reduction in accuracy numbers of TRT model could possibly caused due to pre-processing or post-processing implementations.

…changes

sctrueew · 2022-08-26T05:40:23Z

@wang-xinyu Hi,

I appreciate your implementation. In C++, how do I display className and prob after the infer? In the Python file, there is an example, but I haven't seen it in the C++ file.

Should I change these lines to:

`for (int b = 0; b < fcount; b++) {
auto & res = batch_res[b];
cv::Mat img = imgs_buffer[b];
for (size_t j = 0; j < res.size(); j++) {
cv::Rect r = get_rect(img, res[j].bbox);
cv::rectangle(img, r, cv::Scalar(0x27, 0xC1, 0x36), 2);
cv::putText(img, std::to_string((int) res[j].class_id), cv::Point(r.x, r.y - 1), cv::FONT_HERSHEY_PLAIN, 1.2, cv::Scalar(0xFF, 0xFF, 0xFF), 2);
}
cv::imwrite("_" + file_names[f - fcount + 1 + b], img);

}`

To:
vector classes;
`for (int b = 0; b < fcount; b++) {
auto & res = batch_res[b];
cv::Mat img = imgs_buffer[b];
for (size_t j = 0; j < res.size(); j++) {
auto className = classes[res[j].class_id];
auto prob = res[j].prob;
}

}`

wang-xinyu

Thanks a lot. I left few comments.

yolov5/CMakeLists.txt

yolov5/gen_wts.py

yolov5/yolov5.cpp

yolov5/yolov5_cls.cpp

yolov5/yolov5_trt.py

xiang-wuu · 2022-08-26T08:24:20Z

@wang-xinyu Hi,

I appreciate your implementation. In C++, how do I display className and prob after the infer? In the Python file, there is an example, but I haven't seen it in the C++ file.

Should I change these lines to:

`for (int b = 0; b < fcount; b++) { auto & res = batch_res[b]; cv::Mat img = imgs_buffer[b]; for (size_t j = 0; j < res.size(); j++) { cv::Rect r = get_rect(img, res[j].bbox); cv::rectangle(img, r, cv::Scalar(0x27, 0xC1, 0x36), 2); cv::putText(img, std::to_string((int) res[j].class_id), cv::Point(r.x, r.y - 1), cv::FONT_HERSHEY_PLAIN, 1.2, cv::Scalar(0xFF, 0xFF, 0xFF), 2); } cv::imwrite("_" + file_names[f - fcount + 1 + b], img);

}`

To: vector classes; `for (int b = 0; b < fcount; b++) { auto & res = batch_res[b]; cv::Mat img = imgs_buffer[b]; for (size_t j = 0; j < res.size(); j++) { auto className = classes[res[j].class_id]; auto prob = res[j].prob; }

}`

de-serialization is not handled in the CPP code, it can be easily implemented from the python infer script.

xiang-wuu · 2022-08-26T10:44:10Z

@wang-xinyu 根据您的建议，所有更改都已完成

wang-xinyu · 2022-08-28T06:09:27Z

@xiang-wuu 我的意思是，不要修改yolov5.cpp和yolov5_trt.py这两个文件，现在PR里面包含了这两个文件的一点改动。

xiang-wuu · 2022-08-28T07:14:19Z

@wang-xinyu both the files yolov5.cpp and yolov5_trt.py are reverted back to the original state, there still might be some formatting changes visible in git diff, kindly ignore them!

wang-xinyu · 2022-08-29T08:46:28Z

@xiang-wuu Even format differences, can you revert them? Some lines don't need line-break, but line-breaks are applied. Especially in yolov5_trt.py.

xiang-wuu · 2022-08-29T16:47:43Z

@wang-xinyu as you mention to revert the format changes in yolov5_trt.py file it would be difficult to revert back to prior commit state for single file, however removing all formatting changes back to original state won't revert the git blame authorship to the previous commit author. If you still want me to do the changes manually then let me know.

wang-xinyu · 2022-08-30T03:29:54Z

@xiang-wuu Can you please download the yolov5_trt.py from master branch and overwrite it to your current yolov5_trt.py? That will be easier. No need to consider the git commit state.

xiang-wuu · 2022-08-30T09:55:56Z

@wang-xinyu Reverted back to prior commit.

wang-xinyu · 2022-08-30T10:50:00Z

@xiang-wuu Thanks.

wang-xinyu · 2022-08-30T10:51:23Z

@xiang-wuu So you are using v6.2, right? Have you tested the v6.2 detection model with your branch?

sctrueew · 2022-08-30T12:27:07Z

@wang-xinyu Hi,

I'd like to print the classification result in C++ but the result it's not correct:

`

	auto start = std::chrono::system_clock::now();
	doInference(*context, stream, (void**)buffers, prob, BATCH_SIZE);
	auto end = std::chrono::system_clock::now();
	std::cout << "inference time: " << std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count() << "ms" << std::endl;
	float maxp = 0;
	int index = 0;
	for (int b = 0; b < fcount; b++) {
		for (int j = 0; j < 1000; ++j)
		{
			float p = prob[b * OUTPUT_SIZE + j];
			if (p > maxp)
			{
				maxp = p;
				index = j;
			}
		}
	}
	std::cout << "out index: " << index << " prob: " << maxp << std::endl;

`

sctrueew · 2022-08-30T13:00:17Z

I changed my code to

`

	std::unordered_map<float, int> prob_index;
	for (int j = 0; j < OUTPUT_SIZE; ++j) {
		prob_index[prob[j]] = j;
	}
	std::sort(prob, prob + OUTPUT_SIZE, [](int x, int y) { return x > y; });
	for (int a = 0; a < 10; ++a) {
		int label = prob_index[prob[a]];
		auto pr = prob[a];
		std::string s = std::to_string(pr);
		std::cout << "topk label: " << label << std::endl;
		std::cout << "topk prob: " << s << std::endl;
	}

`

but the result always are:

inference time: 12ms
topk label: 623
topk prob: 4.086990
topk label: 844
topk prob: 3.013781
topk label: 499
topk prob: 3.963979
topk label: 769
topk prob: 3.052796
topk label: 699
topk prob: 3.400196
topk label: 596
topk prob: 3.919277
topk label: 902
topk prob: 3.024654
topk label: 551
topk prob: 3.231964
topk label: 846
topk prob: 3.721910
topk label: 763
topk prob: 3.045218

Would you be able to give me some tips on how to resolve this?

wang-xinyu · 2022-08-31T03:15:57Z

@sctrueew Have you try to use the .py to run inference.

sctrueew · 2022-08-31T06:01:08Z

@wang-xinyu No, I haven't. It is my goal to use C++.

xiang-wuu · 2022-08-31T06:41:08Z

@xiang-wuu So you are using v6.2, right? Have you tested the v6.2 detection model with your branch?

@wang-xinyu as per the official release notes of v6.2 there is no significant architecture changes for detection models since release v6.0 hence detection models from v6.2 works here without any issue.

xiang-wuu · 2022-08-31T06:42:38Z

@wang-xinyu also let me know , do you want me to update the README file for this PR specific changes, then i can raise separate PR for the same.

wang-xinyu · 2022-09-01T03:25:09Z

@xiang-wuu If you can verify the detection model on v6.2 branch, that will be great. We can update the readme after that.
If you are not interested in that, I will verify it later, and also update readme later.

xiang-wuu · 2022-09-01T04:49:19Z

@xiang-wuu If you can verify the detection model on v6.2 branch, that will be great. We can update the readme after that. If you are not interested in that, I will verify it later, and also update readme later.

As i mentioned earlier, i verified that and it's working fine.

sctrueew · 2022-09-05T12:26:55Z

@xiang-wuu,
I couldn't find any solution. Could you please give us some tips to get results in C++?

wang-xinyu · 2022-09-06T12:43:21Z

@sctrueew @xiang-wuu I have updated the cpp code, the c++ inference is working now, can you check?

sctrueew · 2022-09-06T13:17:41Z

@wang-xinyu Thanks. I've updated the code to the latest version and tested it but I don't know how to display the result after doInference.

I'm using this to display: Is it correct?

`

std::unordered_map<float, int> prob_index;

	for (int j = 0; j < OUTPUT_SIZE; ++j) {
		prob_index[prob[j]] = j;
	}
	std::sort(prob, prob + OUTPUT_SIZE, [](int x, int y) { return x > y; });
	for (int a = 0; a < 10; ++a) {
		int label = prob_index[prob[a]];
		auto pr = prob[a];
		std::string s = std::to_string(pr);
		std::cout << "topk label: " << label << std::endl;
		std::cout << "topk prob: " << s << std::endl;
	}`

wang-xinyu · 2022-09-07T08:07:25Z

@sctrueew Cpp postprocessing is also added, please check.

sctrueew · 2022-09-07T12:49:17Z

@wang-xinyu Thanks for the implementation. I've tested but the result is always this for all images:

../giraffe.jpg
letter opener, paper knife, paperknife 0.0260404
cleaver, meat cleaver, chopper 0.0230263
hatchet 0.0220196

* updated gen_wts script to support yolov5 classfication model export * updated yolov5-s architecture to support classification head. * updated yolov5_trt infer script to support yolov5 classification model alongwith existing detection model. * added imagenet_classes file to load list of 1k classes required for the infer script. * final conv block doesn't require dynamic scale factor for out channel, hence hardcoded value provided to support all model varients * yolov5.cpp reverted back to original state, with explicit code changes * python infer script reverted back to prior state, with explicit code changes * seperate cpp file added for yolov5 classification task * cmake updated for yolo5 classification file * seperate python infer script for classification task * cmake updated by replacing cuda_add_executable with add_executable * default value added to the type argument * post-processing removed from yolov5 clasisifcation module * pre-processing for yolov5 classification inferencing * classification macro removed from the original yolov5 detection cpp file * reverted some extremely minor formatting changes. * reverted back to prior state by removing all formatting changes.

updated gen_wts script to support yolov5 classfication model export

bd76d45

updated yolov5-s architecture to support classification head.

975b78f

xiang-wuu commented Aug 24, 2022

View reviewed changes

yolov5/yolov5.cpp Outdated Show resolved Hide resolved

xiang-wuu added 2 commits August 24, 2022 19:52

updated yolov5_trt infer script to support yolov5 classification mode…

e15e6d3

…l alongwith existing detection model.

added imagenet_classes file to load list of 1k classes required for t…

9e4538b

…he infer script.

final conv block doesn't require dynamic scale factor for out channel…

69a246e

…, hence hardcoded value provided to support all model varients

xiang-wuu added 5 commits August 25, 2022 23:46

yolov5.cpp reverted back to original state, with explicit code changes

7bfbe80

python infer script reverted back to prior state, with explicit code …

69e059d

…changes

seperate cpp file added for yolov5 classification task

2aa861f

cmake updated for yolo5 classification file

bfef04b

seperate python infer script for classification task

b235a8d

xiang-wuu requested a review from wang-xinyu August 25, 2022 19:11

wang-xinyu requested changes Aug 26, 2022

View reviewed changes

xiang-wuu added 2 commits August 26, 2022 13:51

cmake updated by replacing cuda_add_executable with add_executable

e858eb4

default value added to the type argument

695ee74

xiang-wuu added 2 commits August 26, 2022 15:14

post-processing removed from yolov5 clasisifcation module

6645805

pre-processing for yolov5 classification inferencing

0589aa1

classification macro removed from the original yolov5 detection cpp file

9e11275

xiang-wuu requested a review from wang-xinyu August 28, 2022 07:15

reverted some extremely minor formatting changes.

5f06f87

reverted back to prior state by removing all formatting changes.

b277925

wang-xinyu approved these changes Aug 30, 2022

View reviewed changes

wang-xinyu merged commit a772e46 into wang-xinyu:master Aug 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yolov5 classification model support #1082

Yolov5 classification model support #1082

xiang-wuu commented Aug 22, 2022 •

edited

Loading

wang-xinyu commented Aug 23, 2022

wang-xinyu commented Aug 24, 2022

xiang-wuu commented Aug 24, 2022 •

edited

Loading

xiang-wuu commented Aug 25, 2022 •

edited

Loading

sctrueew commented Aug 26, 2022 •

edited

Loading

wang-xinyu left a comment

xiang-wuu commented Aug 26, 2022

xiang-wuu commented Aug 26, 2022

wang-xinyu commented Aug 28, 2022

xiang-wuu commented Aug 28, 2022

wang-xinyu commented Aug 29, 2022

xiang-wuu commented Aug 29, 2022

wang-xinyu commented Aug 30, 2022

xiang-wuu commented Aug 30, 2022

wang-xinyu commented Aug 30, 2022

wang-xinyu commented Aug 30, 2022

sctrueew commented Aug 30, 2022

sctrueew commented Aug 30, 2022

wang-xinyu commented Aug 31, 2022

sctrueew commented Aug 31, 2022

xiang-wuu commented Aug 31, 2022

xiang-wuu commented Aug 31, 2022

wang-xinyu commented Sep 1, 2022

xiang-wuu commented Sep 1, 2022

sctrueew commented Sep 5, 2022

wang-xinyu commented Sep 6, 2022

sctrueew commented Sep 6, 2022 •

edited

Loading

wang-xinyu commented Sep 7, 2022

sctrueew commented Sep 7, 2022

Yolov5 classification model support #1082

Yolov5 classification model support #1082

Conversation

xiang-wuu commented Aug 22, 2022 • edited Loading

wang-xinyu commented Aug 23, 2022

wang-xinyu commented Aug 24, 2022

xiang-wuu commented Aug 24, 2022 • edited Loading

xiang-wuu commented Aug 25, 2022 • edited Loading

Evaluation Metrics

sctrueew commented Aug 26, 2022 • edited Loading

wang-xinyu left a comment

Choose a reason for hiding this comment

xiang-wuu commented Aug 26, 2022

xiang-wuu commented Aug 26, 2022

wang-xinyu commented Aug 28, 2022

xiang-wuu commented Aug 28, 2022

wang-xinyu commented Aug 29, 2022

xiang-wuu commented Aug 29, 2022

wang-xinyu commented Aug 30, 2022

xiang-wuu commented Aug 30, 2022

wang-xinyu commented Aug 30, 2022

wang-xinyu commented Aug 30, 2022

sctrueew commented Aug 30, 2022

sctrueew commented Aug 30, 2022

wang-xinyu commented Aug 31, 2022

sctrueew commented Aug 31, 2022

xiang-wuu commented Aug 31, 2022

xiang-wuu commented Aug 31, 2022

wang-xinyu commented Sep 1, 2022

xiang-wuu commented Sep 1, 2022

sctrueew commented Sep 5, 2022

wang-xinyu commented Sep 6, 2022

sctrueew commented Sep 6, 2022 • edited Loading

wang-xinyu commented Sep 7, 2022

sctrueew commented Sep 7, 2022

xiang-wuu commented Aug 22, 2022 •

edited

Loading

xiang-wuu commented Aug 24, 2022 •

edited

Loading

xiang-wuu commented Aug 25, 2022 •

edited

Loading

sctrueew commented Aug 26, 2022 •

edited

Loading

sctrueew commented Sep 6, 2022 •

edited

Loading