detect.py and train_aux.py don't work on yolov7 base and custom weights on GTX 16xx

[Dyupich]

Hello! I faced the following problem. This command works fine. However, when viewing the resulting photo, it turns out that nothing was found.

python detect.py --weights yolov7x.pt --conf 0.1 --img-size 640 --source data\val\test.jpg --no-trace --project checking_detect --name images

I tried on a few images with other weights (yolov7x, yolov7-e6e, custom-yolov7), but result is one. No classes, no bounding boxes.

I tried it on Windows 10 with all requirements installed.
Version of torch: 1.11.0+cu113
Version of torchvision: 0.12.0+cu113
Python version: 3.9.5
GPU: GTX 1660

(torch) C:\Users\Andrey\Desktop\yolov7-custom>python detect.py --weights yolov7-e6e.pt --conf 0.5 --img-size 640 --source data\val\04501.jpg --no-trace --project checking_detect --name images
Namespace(weights=['yolov7-e6e.pt'], source='data\val\04501.jpg', img_size=640, conf_thres=0.5, iou_thres=0.45, device='', view_img=False, save_txt=False, save_conf=False, nosave=False, classes=None, agnostic_nms=False, augment=False, update=False, project='checking_detect', name='images', exist_ok=False, no_trace=True)
YOLOR 2022-8-16 torch 1.11.0+cu113 CUDA:0 (NVIDIA GeForce GTX 1660, 6143.6875MB)

Fusing layers...
Model Summary: 792 layers, 151687420 parameters, 817020 gradients
C:\Users\Andrey\anaconda3\envs\torch\lib\site-packages\torch\functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:2228.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Done. (327.0ms) Inference, (0.0ms) NMS
The image with the result is saved in: checking_detect\images2\04501.jpg
Done. (1.319s)

When I try to train on custom dataset I got this...

I tried on Linux with this same version of torch and torchvision on another device and all works fine. ( On custom dataset on 0 and 1 epoch I got mAP=0.4 and mAP=0.5, respectively, and all other weights works normal and shows me bboxes and labels)

Is there a way to make everything work on Windows with my setup?

[puresox]

I had the same problem and I found out that it was my GPU that did not support half precision.You can try to add the "--device cpu" argument or set the variable "half" to False in detect.py.

[Dyupich]

I had the same problem and I found out that it was my GPU that did not support half precision.You can try to add the "--device cpu" argument or set the variable "half" to False in detect.py.

Thanks for your answer. I tried your method. Yes, it works on CPU (with cli flag --device cpu and variable half = False) for detect.py.

But I want to use my GPU :-)
I checkout information about 16xx series. And I found this issue at yolov5 repository.

ultralytics/yolov5#7908

For training I found method (based on this issue) to solve the problem with awful mAP but it did not help.
I added parameter "half_precision=False" in two points of train_aux.py

Start on Line 412

            results, maps, times = test.test(data_dict,
                                             batch_size=batch_size * 2,
                                             imgsz=imgsz_test,
                                             model=ema.ema,
                                             single_cls=opt.single_cls,
                                             dataloader=testloader,
                                             save_dir=save_dir,
                                             verbose=nc < 50 and final_epoch,
                                             plots=plots and final_epoch,
                                             wandb_logger=wandb_logger,
                                             compute_loss=compute_loss,
                                             is_coco=is_coco,
                                             half_precision=False) #here <---------------

Start on Line 492

            results, _, _ = test.test(opt.data,
                                      batch_size=batch_size * 2,
                                      imgsz=imgsz_test,
                                      conf_thres=0.001,
                                      iou_thres=0.7,
                                      model=attempt_load(m, device).half(),
                                      single_cls=opt.single_cls,
                                      dataloader=testloader,
                                      save_dir=save_dir,
                                      save_json=True,
                                      plots=False,
                                      is_coco=is_coco,
                                      half_precision=False) #here <---------------

Hope this information will help to solve the problem

[Dyupich]

I downgraded torch, torchvision, cudatoolkit, cudnn based on issue: openai/glide-text2im#31 (comment)
After it detect.py and train_aux.py works fine without any changes in code.

conda activate myvenv
pip install torch==1.10.1+cu102 torchvision==0.11.2+cu102 -f https://download.pytorch.org/whl/torch_stable.html
conda install cudatoolkit=10.2 cudnn=8.0 -c=conda-forge
pip install setuptools==59.5.0