Add QFocalLoss()

[yxNONG]

implement the quality focal loss which is a more general case of focal loss
more detail in https://arxiv.org/abs/2006.04388

In the obj loss (or the case cls loss with label smooth), the targets is no long barely be 0 or 1 (can be 0.7), in this case, the normal focal loss is not work accurately
quality focal loss in behave the same as focal loss when the target is equal to 0 or 1, and work accurately when targets in (0, 1)

example:

targets:
tensor([[0.6225, 0.0000, 0.0000],
[0.9000, 0.0000, 0.0000],
[1.0000, 0.0000, 0.0000]])

---

pred_prob:
tensor([[0.6225, 0.2689, 0.1192],
[0.7773, 0.5000, 0.2227],
[0.8176, 0.8808, 0.1978]])

---

focal_loss
tensor([[0.0937, 0.0328, 0.0039],
[0.0166, 0.1838, 0.0199],
[0.0039, 1.3186, 0.0145]])

---

qfocal_loss
tensor([[7.5373e-08, 3.2768e-02, 3.9179e-03],
[4.8601e-03, 1.8380e-01, 1.9857e-02],
[3.9233e-03, 1.3186e+00, 1.4545e-02]])

we can see that targets[0][0] = 0.6255 is almost the same as pred_prob[0][0] = 0.6225,
the targets[1][0] = 0.9 is greater then pred_prob[1][0] = 0.7773 by 0.1227
however, the focal loss[0][0] = 0.0937 larger then focal loss[1][0] = 0.0166 (which against the purpose of focal loss)

for the quality focal loss , it implement the case of targets not equal to 0 or 1

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Implementation of Quality Focal Loss (QFL) in the YOLOv5 loss computation.

📊 Key Changes

• 🆕 Introduced a new class QFocalLoss as a wrapper around the existing loss function.
• ⚙️ QFocal loss is an adaptation of traditional focal loss that incorporates a quality factor.
• 🧮 Customizes the original loss by scaling it with a modulating factor based on prediction accuracy and an alpha factor for class imbalance.

🎯 Purpose & Impact

• 💡 The purpose is to enhance the model's accuracy by focusing training on hard examples and reducing the impact of easy negatives.
• 🎚 The impact could include improved performance in scenarios with class imbalance or when detecting challenging objects.
• 🔄 Users may observe changes in loss magnitudes and training dynamics due to the new loss component.

[glenn-jocher]

@yxNONG very interesting, thanks for the PR. It is very much the case in YOLOv5 that objectness target values are between 0-1 since they are set to the achieved the IoU (YOLOv3 paper set them to 1.0). This means that objectness targets start near 0 at the beginning of training and trend towards 1 as the regression output becomes more accurate in the later stages of training. Fractional objectness target values like 0.7 are very common.

[yxNONG]

@glenn-jocher i notice that it can be not equal 0 and 1 so submit the PR, i will double check if the implement is correct late

[glenn-jocher]

/rebase

[glenn-jocher]

/rebase

[glenn-jocher]

@yxNONG is this PR ready to merge?

[yxNONG]

@glenn-jocher , yes i think it work correctly now

targets:
tensor([[0.6225, 0.0000, 0.0000],
[0.9000, 0.0000, 0.0000],
[0.5000, 1.0000, 0.0000]])
pred_prob:
tensor([[0.6225, 0.2689, 0.1192],
[0.7773, 0.5000, 0.2227],
[0.5000, 0.8808, 0.1978]])

---

normal_loss:
tensor([[0.6628, 0.3133, 0.1269],
[0.3769, 0.6931, 0.2519],
[0.6931, 0.1269, 0.2204]])

---

focal_loss
tensor([[0.0937, 0.0328, 0.0039],
[0.0166, 0.1838, 0.0199],
[0.1225, 0.0013, 0.0145]])

---

qfocal_loss
tensor([[7.5373e-08, 3.2768e-02, 3.9179e-03],
[4.8601e-03, 1.8380e-01, 1.9857e-02],
[0.0000e+00, 1.3060e-03, 1.4545e-02]])

---

[glenn-jocher]

@yxNONG great! I will merge this PR then.

Thank you for your contributions.

[zehuichen123]

hi, it seems that this QFL does not yield an API for us to use it?

[nikbobo]

In the paper, I only see the hyperparameters is only the gamma, but in your implements it has two hyperparameters, alpha and gamma, which different about that, and how to control the this two hyperparameters?

[glenn-jocher]

@nikbobo, the Quality Focal Loss indeed consists of the gamma parameter, which controls the balance between easy and hard examples, and the alpha parameter, which controls the class balance. These two hyperparameters can be adjusted to control the behavior of the loss function. To use QFocalLoss, please refer to the documentation at https://docs.ultralytics.com/yolov5/. Thank you for bringing this to our attention.