How to specify the learning rate decay strategy during training? How to achieve it?

[Wanghe1997]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

For YOLOv5-v6.0
Such as StepLR，ExpLR，MultiStepLR，CosineLR and so on. In YOLOv5-v6.0，which learning rate decay strategy is used by default? If I want to change, which lines of code in which file should I change? thanks！

Additional

No response

[glenn-jocher]

@Wanghe1997 YOLOv5 uses cosine LR scheduler by default:

yolov5/train.py

Lines 167 to 172 in 8df64a9

	# Scheduler
	if opt.linear_lr:
	lf = lambda x: (1 - x / (epochs - 1)) * (1.0 - hyp['lrf']) + hyp['lrf'] # linear
	else:
	lf = one_cycle(1, hyp['lrf'], epochs) # cosine 1->hyp['lrf']
	scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lf) # plot_lr_scheduler(optimizer, scheduler, epochs)

[Wanghe1997]

@Wanghe1997 YOLOv5 uses cosine LR scheduler by default:

yolov5/train.py

Lines 167 to 172 in 8df64a9

	# Scheduler
	if opt.linear_lr:
	lf = lambda x: (1 - x / (epochs - 1)) * (1.0 - hyp['lrf']) + hyp['lrf'] # linear
	else:
	lf = one_cycle(1, hyp['lrf'], epochs) # cosine 1->hyp['lrf']
	scheduler = lr_scheduler.LambdaLR(optimizer, lr_lambda=lf) # plot_lr_scheduler(optimizer, scheduler, epochs)

In addition to cosine LR scheduler, are there other strategies written in the program?

[glenn-jocher]

@Wanghe1997 linear is written in with python train.py --linear, but you can also use any other custom scheduler you want with a lambda function or any premade schedulers here:
https://pytorch.org/docs/stable/optim.html

[Wanghe1997]

@Wanghe1997 linear is written in with python train.py --linear, but you can also use any other custom scheduler you want with a lambda function or any premade schedulers here: https://pytorch.org/docs/stable/optim.html

Thank you for your reply. Can you answer my supplementary question in another issue?
#5653 (comment)

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[adityatandon]

For anyone looking for an updated answer in 2022, from YOLOv5-version 6.1 onwards, the default learning rate scheduler has been changed to linear instead of the cosine lr scheduler.

[glenn-jocher]

@adityatandon thank you for the update! The YOLOv5 team is constantly improving the framework based on the latest research and user feedback. If you have any further questions or need assistance with anything else, feel free to ask here.