Pytorch implementation for Forgery-Domain-Supervised Deepfake Detection with Non-negative Constraint. [Paper Link]
Please install Pytorch first. This repo has been developed with python version 3.6, pytorch version 1.8.0, CUDA 10.2 and torchvision 0.9.0. For convenience, use requirements to create a conda environment.
We use the well-known public dataset FaceForensics++
for training and evaluation. Check the github repo and paper link for more details.
Following the settings, we use dlib
to detect the face, enlarge the face region 1.3 times and crop it. The whole dataset contains 1000 videos, in which 720 for training, 140 for validation and 140 for testing. Here are split rules.
After preprocessing, the dataset should be organized as following:
|-- dataset
| |-- train
| | |-- real
| | | |-- 000
| | | | |-- frame0.jpg
| | | | |-- frame1.jpg
| | | | |-- ...
| | | |-- 001
| | | |-- ...
| | |-- fake
| | |-- Deepfakes
| | | |-- 000_167
| | | | |-- frame0.jpg
| | | | |-- frame1.jpg
| | | | |-- ...
| | | |-- 001_892
| | | |-- ...
| | |-- Face2Face
| | | |-- ...
| | |-- FaceSwap
| | |-- NeuralTextures
| |-- valid
| | |-- real
| | | |-- ...
| | |-- fake
| | |-- ...
| |-- test
| | |-- ...
We recommend to use our implemented get_FF_dataset
and get_FF_5class
functions to load pytorch datasets. Here's an simple example.
train_dataset = get_FF_5_class(path_to_dataset, mode='train', isTrain=True)
valid_dataset = get_FF_dataset(path_to_dataset, mode='valid', isTrain=False, img_size=img_size, drop_rate=0.8)
The function get_FF_5class
returns a 5-class label for multi-class supervision in our methods and get_FF_dataset
returns vanilla binary label.
For more details about the function, param or customizing dataset, please refer to dataset.md.
For a glimpse at the documentation training of our method, please run:
python main.py --help
With default parameters setting, to traing a model with AIM
and NCC
from the scratch, please run:
python main.py --output-dir /path/to/checkpoint/ --backbone-name resnet --dataset-path /path/to/dataset/ --use-ncc --use-aim --use-mc
If a pretrained backbone is loaded, we recommend to set warmup-iters
to 20000.
To evalulate on FaceForensics++
dataset, please run:
python evaluation.py --ckpt-path /path/to/checkpoint/ --backbone-name resnet --dataset-path /path/to/dataset/ --use-ncc --use-aim --use-mc
We compare the baseline (vanilla xception, binary supervision) with our method (xception, multi-class supervision, with NCC and AIM).
The performance evaluation is conducted on FF++ c40
test split. Execution time is measured with batch size 1, 20 runs averaged.
Methods | AUC | Params(M) | MACs(G) | CPU total(s) | GPU total(ms) |
---|---|---|---|---|---|
Xception | 88.65 | 20.811 | 6.012 | 2.083 | 121.743 |
Ours | 92.15 | 37.605 | 18.085 | 2.404 | 272.773 |
Note: Params and MACs are measured by pytorch-OpCounter. CPU total and GPU total are measured by pytorch profiler. The CPU total time includes CUDALauchKernel. Check comp_cost.py and runtime_cost.py for more details.