This project was done in terms of The 80th BSU Scientific Conference of Students and Postgraduates. The text in Russian is available here. There is also a more detailed version of this text here.
Source jupyter notebook in English with main aspects of the project is available here.
The example of landmarks detection on the video can be found here.
The main aim of the project was to compare different neural networks architectures (ResNet18, MobileNetV2, EfficientNetB0, MobileNetV3-Large, MobileViT-Small) and the influence of various aspects in the training process on the task of regression of 2D facial landmarks. The LaPa dataset was used to train the networks and estimate the quality of the results.
On the next plot you can see the dependence of the NME (normalized mean error) on the test part of the dataset from various parameters: the inference time on the CPU, on the GPU, GFLOPS, the number of trained parameters:
The influence and importance of various aspects in the training process and data preparation was analyzed on the base of MobileNetV3:
The amazing quality improvement was reached after replacing the MSE loss with the Wing loss:
The example of landmarks detection on the video with the EfficientNetB0 and the Wing loss can be found here.