Facial Keypoint Detection

In this project, I combine my knowledge of computer vision techniques and deep learning to build an end-to-end facial keypoint recognition system. Facial keypoints include points around the eyes, nose, and mouth on any face and are used in many applications, from facial tracking to emotion recognition. My completed solution can take in any image containing faces and identify the location of each face and their facial keypoints, as shown below.

The project is broken up into a few main parts in one Jupyter notebook:

Part 1 : Investigating OpenCV, pre-processing, and face detection

Part 2 : Training a Convolutional Neural Network (CNN) to detect facial keypoints

Part 3 : Putting parts 1 and 2 together to identify facial keypoints on any image!

Data

I am using the facial keypoints detection dataset from Kaggle. If you'd like to re-train the model, you'll need to first download the training/test sets and place them in the subdirectory data.

Step 1: Face detection and Eye detection

Implement face and eye detection.

Step 2: De-noise an image for better face detection

De-noise an image for better face detection.

Step 3: Canny Edge Detection and Gaussian Blur

Blur and edge detect a test image.

Step 4: Automatically hide the identity of a person (blur a face)

Automatically detect the face of a person in a test image, then blur their face to mask their identity.

Step 5: Build a Convolutional Neural Network (CNN)

Design a convolutional network architecture for learning correspondence between input faces and facial keypoints. Compile and train the CNN on the dataset. Visualize the loss function.

Step 6: Complete a facial keypoints detector and complete the CV pipeline

Combine OpenCV face detection with trained convnet facial keypoint detector to detect facial keypoints on any image that contains one or multiple human faces.