Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ValueError: signal number 32 out of range and the training progress bar not moving when running training command "python rotate_train.py" #10

Open
hopesala opened this issue Oct 10, 2021 · 2 comments

Comments

@hopesala
Copy link

python rotate_train.py --weights rotate-yolov5s-ucas.pt --cfg rotate_yolov5s_ucas.yaml --data rotate_ucas.yaml --hyp hyp.ucas.yaml --img-size 1024 --epochs 3 --batch-size 4 --noautoanchor --rotate --cache
YOLOv5 🚀 2021-8-18 torch 1.7.0 CUDA:0 (NVIDIA GeForce GTX 1080 Ti, 11170.0625MB)

Namespace(adam=False, artifact_alias='latest', batch_size=4, bbox_interval=-1, bucket='', cache_images=True, cfg='./models/rotate_yolov5s_ucas.yaml', data='./data/rotate_ucas.yaml', device='', entity=None, epochs=3, evolve=False, exist_ok=False, global_rank=-1, hyp='./data/hyp.ucas.yaml', image_weights=False, img_size=[1024, 1024], label_smoothing=0.0, linear_lr=False, local_rank=-1, multi_scale=False, name='exp', noautoanchor=True, nosave=False, notest=False, project='runs/train', quad=False, rect=False, resume=False, rotate=True, save_dir='runs/train/exp6', save_period=-1, single_cls=False, sync_bn=False, total_batch_size=4, upload_dataset=False, weights='rotate-yolov5s-ucas.pt', workers=4, world_size=1)
tensorboard: Start with 'tensorboard --logdir runs/train', view at http://localhost:6006/
hyperparameters: lr0=0.01, lrf=0.2, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=0.05, cls=0.5, cls_pw=1.0, obj=1.0, obj_pw=1.0, iou_t=0.1, anchor_t=4.0, fl_gamma=0.0, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=30.0, translate=0.1, scale=0.0, shear=5.0, perspective=0.0005, flipud=0.5, fliplr=0.5, mosaic=0.0, mixup=0.0
wandb: Install Weights & Biases for YOLOv5 logging with 'pip install wandb' (recommended)

             from  n    params  module                                  arguments                     

0 -1 1 3520 models.common.Focus [3, 32, 3]
1 -1 1 18560 models.common.Conv [32, 64, 3, 2]
2 -1 1 18816 models.common.C3 [64, 64, 1]
3 -1 1 73984 models.common.Conv [64, 128, 3, 2]
4 -1 1 156928 models.common.C3 [128, 128, 3]
5 -1 1 295424 models.common.Conv [128, 256, 3, 2]
6 -1 1 625152 models.common.C3 [256, 256, 3]
7 -1 1 1180672 models.common.Conv [256, 512, 3, 2]
8 -1 1 656896 models.common.SPP [512, 512, [5, 9, 13]]
9 -1 1 1182720 models.common.C3 [512, 512, 1, False]
10 -1 1 131584 models.common.Conv [512, 256, 1, 1]
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 [-1, 6] 1 0 models.common.Concat [1]
13 -1 1 361984 models.common.C3 [512, 256, 1, False]
14 -1 1 33024 models.common.Conv [256, 128, 1, 1]
15 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
16 [-1, 4] 1 0 models.common.Concat [1]
17 -1 1 90880 models.common.C3 [256, 128, 1, False]
18 -1 1 147712 models.common.Conv [128, 128, 3, 2]
19 [-1, 14] 1 0 models.common.Concat [1]
20 -1 1 296448 models.common.C3 [256, 256, 1, False]
21 -1 1 590336 models.common.Conv [256, 256, 3, 2]
22 [-1, 10] 1 0 models.common.Concat [1]
23 -1 1 1182720 models.common.C3 [512, 512, 1, False]
24 [17, 20, 23] 1 40455 models.yolo.Rotate_Detect [2, [[27, 26, 20, 40, 44, 19, 34, 34, 25, 47], [55, 24, 44, 38, 31, 61, 50, 50, 63, 45], [65, 62, 88, 60, 84, 79, 113, 85, 148, 122]], [128, 256, 512]]
Model Summary: 283 layers, 7087815 parameters, 7087815 gradients

Transferred 360/362 items from rotate-yolov5s-ucas.pt
Scaled weight_decay = 0.0005
Optimizer groups: 62 .bias, 62 conv.weight, 59 other
train: Scanning '../UCAS50/train.cache' images and labels... 38 found, 0 missing
train: Caching images (0.1GB): 100%|███████████| 38/38 [00:00<00:00, 177.94it/s]
val: Scanning '../UCAS50/val.cache' images and labels... 10 found, 0 missing, 0
0%| | 0/10 [00:00<?, ?it/s]Exception in thread Thread-14:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

val: Caching images (0.0GB): 100%|██████████████| 10/10 [00:00<00:00, 99.27it/s]
Exception in thread Thread-14:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Exception in thread Thread-14:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Exception in thread Thread-14:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Plotting labels...
Exception in thread Thread-26:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Exception in thread Thread-26:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Exception in thread Thread-26:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Exception in thread Thread-26:
Traceback (most recent call last):
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/cao/anaconda3/envs/stark/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/cao/anaconda3/envs/stark/lib/python3.6/multiprocessing/resource_sharer.py", line 139, in _serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
File "/home/cao/anaconda3/envs/stark/lib/python3.6/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

Image sizes 1024 train, 1024 test
Using 4 dataloader workers
Logging results to runs/train/exp6
Starting training for 3 epochs...

 Epoch   gpu_mem       box       obj       cls     total    labels  img_size

0%| | 0/10 [00:00<?, ?it/s]

What could be the reason and how to fix this problem? thanks for replying~

@hopesala
Copy link
Author

Besides, when I run command "python rotate_detect.py --weights rotate-yolov5s-ucas.pt --img 1024 --conf 0.75
--source data/images/UCAS-AOD --iou-thres 0.4 --hide-labels"
no objects are detected(no red rectangles are found in the runs/detect/expX folder output imgs)

@XinzeLee
Copy link
Owner

It seems like a bug in python. Please refer to this link for detailed explainations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants