Skip to content

Code for the ICML 2020 paper: Leveraging Frequency Analysis for Deep Fake Image Recognition.

License

Notifications You must be signed in to change notification settings

RUB-SysSec/GANDCTAnalysis

Repository files navigation

Leveraging Frequency Analysis for Deep Fake Image Recognition

logo

This is the code repository accompaning our ICML 2020 paper Leveraging Frequency Analysis for Deep Fake Image Recognition.

Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for untrained humans to distinguish them from actual photos. These achievementshave been largely made possible by Generative Adversarial Networks (GANs). While these deepfake images have been thoroughly investigatedin the image domain—a classical approach from the area of image forensics—an analysis in the frequency domain has been missing. This paper addresses this shortcoming and our results reveal, that in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified. We perform a comprehensive analysis, showing that these artifacts are consistent across different neural network architectures, data sets,and resolutions. In a further investigation, we demonstrate that these artifacts are caused by upsampling operations found in all current GAN architectures, indicating a structural and fundamental problem in the way images are generatedvia GANs. Based on this analysis, we demonstrate how the frequency representation can be used to automatically identify deep fake images, surpassing state-of-the-art methods.

Prerequisites

For ease of use we provide a Dockerfile which builds a container in which you can execute all experiments. Additionally, we provide a shell script for ease of use:

Choose: docker.sh {build|download|convert|shell}
    build - Build the Dockerfile.
    shell - Spawn a shell inside the docker container.
    tests - Spawn Docker instance for pytest.
    clean - Cleanup directories from training.

Otherwise you will need a recent Python 3 version, tensorflow 2.0+ with CUDA compatibility. See requirements.txt for packages needed.

Datasets

We utilize these three popular datasets:

Additionally, we utilize the pre-trained models from these repositories:

Dataset preparation

The datasets have to be converted beforehand. First run crop_celeba.py or crop_lsun.py depending on your dataset. This will create a new folder which has cropped instances of the training data to 128x128. Then run prepare_dataset.py, depending on the mode selection the script expects different input. Note FFHQ is distributed in a cropped version.

The scripts expects one directory as input, containing multiple directories each with at least 27,000 images. These directories will get encoded with labels in the order of appearence, i.e., encoded as follows:

data
 |--- A_lsun 	-> label 0
 |--- B_ProGAN 	-> label 1
 	...

It converts all images to dct encoded numpy arrays/tfrecords, depending on the mode selected. Saving the output in three directories train (100,000), val (10,000) and test (25,000).

usage: prepare_dataset.py [-h] [--raw] [--log] [--color] [--normalize]
                          DIRECTORY {normal,tfrecords} ...

positional arguments:
  DIRECTORY           Directory to convert.
  {normal,tfrecords}  Select the mode {normal|tfrecords}

optional arguments:
  -h, --help          show this help message and exit
  --raw, -r           Save image data as raw image.
  --log, -l           Log scale Images.
  --color, -c         Compute as color instead.
  --normalize, -n     Normalize data.

Example:
python prepare_dataset.py ~/datasets/GANFingerprints/perturbed_experiments/lsun/blur/ -lnc normal

Computing Statistics

To compute all of our statistics we utilize the compute_statistics.py script. This script is run on the raw (cropped) image files.

usage: compute_statistics.py [-h] [--output OUTPUT] [--color]
                             AMOUNT [DATASETS [DATASETS ...]]

positional arguments:
  AMOUNT                The amount of images to load.
  DATASETS              Path to datasets. The first entry is assumed to be the
                        referrence one.

optional arguments:
  -h, --help            show this help message and exit
  --output OUTPUT, -o OUTPUT
                        Output directory. Default: {output_default}.
  --color, -c           Plot for each color channel seperate.
  
Example:
python compute_statistics.py 10000 ~/datasets/ffhq/real,REAL ~/datasets/ffhq/fake,FAKE

Experiments

Training your own models

After you have converted the data files as laid out above, you can train a new classifier:

usage: classifer.py train [-h] [--debug] [--epochs EPOCHS]
                          [--image_size IMAGE_SIZE]
                          [--early_stopping EARLY_STOPPING]
                          [--classes CLASSES] [--grayscale]
                          [--batch_size BATCH_SIZE] [--l1 L1] [--l2 L2]
                          MODEL TRAIN_DATASET VAL_DATASET

positional arguments:
  MODEL                 Select model to train {resnet, cnn, nn, log, log1,
                        log2, log3}.
  TRAIN_DATASET         Dataset to load.
  VAL_DATASET           Dataset to load.

optional arguments:
  -h, --help            show this help message and exit
  --debug, -d           Debug mode.
  --epochs EPOCHS, -e EPOCHS
                        Epochs to train for; Default: 50.
  --image_size IMAGE_SIZE
                        Image size. Default: [128, 128, 3]
  --early_stopping EARLY_STOPPING
                        Early stopping criteria. Default: 5
  --classes CLASSES     Classes. Default: 5
  --grayscale, -g       Train on grayscaled images.
  --batch_size BATCH_SIZE, -b BATCH_SIZE
                        Batch size. Default: 32
  --l1 L1               L1 reguralizer intensity. Default: 0.01
  --l2 L2               L2 reguralizer intensity. Default: 0.01
 
Example:

python classifer.py train log2 datasets/ffhq/data_raw_color_train_tf/data.tfrecords datasets/ffhq/data_raw_color_val_tf/data.tfrecords -b 32 -e 100 --l2 0.01 --classes 1 --image_size 1024

Testing

You can also use our pre-trained models.

usage: classifer.py test [-h] [--image_size IMAGE_SIZE] [--grayscale]
                         [--batch_size BATCH_SIZE]
                         MODEL TEST_DATASET

positional arguments:
  MODEL                 Path to model.
  TEST_DATASET          Dataset to load.

optional arguments:
  -h, --help            show this help message and exit
  --image_size IMAGE_SIZE
                        Image size. Default: [128, 128, 3]
  --grayscale, -g       Test on grayscaled images.
  --batch_size BATCH_SIZE, -b BATCH_SIZE
                        Batch size. Default: 32
                        
Example:
python classifer.py test path/to/classifier datasets/ffhq/data_raw_color_test_tf/data.tfrecords -b 32 --image_size 1024

If you simply want to clasify a directory of images, you can do so by utilizing our run_classifer.py script:

usage: run_classifier.py [-h] [--size SIZE] [--batch_size BATCH_SIZE]
                         [--dct DCT]
                         MODEL DATA

positional arguments:
  MODEL                 Model to evaluate.
  DATA                  Directory to classify.

optional arguments:
  -h, --help            show this help message and exit
  --size SIZE, -s SIZE  Only use this amount of images.
  --batch_size BATCH_SIZE, -b BATCH_SIZE
                        Batch size to use; Default: {batch_size}.
  --dct DCT, -d DCT     DCT input

Example:

python run_classifier.py submission_models/ffhq/ridge_pixel ~/datasets/ffhq/fake -s 1000

If you use a DCT-variant of the classifer, you also have to provide an estimate of the mean and variance of the dataset so it can be normalized. They can also be found in the google drive:

python run_classifier.py submission_models/ffhq/ridge_dct ~/datasets/ffhq/fake -s 1000 -d mean_var/ffhq_mean_var

Baselines

Basesline experiments are located in baselines directory. Each experiment (i.e., PRNU, Eigenfaces, and kNN) can be executed using the common baseline.py script.

usage: baselines.py [-h] [--command {train,test,grid_search}]
                    [--n_jobs N_JOBS] --datasets DATASETS --datasets_dir
                    DATASETS_DIR --output_dir OUTPUT_DIR
                    [--classifier_name CLASSIFIER_NAME]
                    {knn,prnu,eigenfaces} ...

Example (grid-search):
  python baselines.py --n_jobs 64 --command grid_search \
    --output_dir baselines/results \
    --datasets_dir datasets \
    --datasets lsun_raw_color_raw_normalized \
    --datasets lsun_raw_color_dct_log_scaled_normalized \
    --datasets celeba_raw_color_raw_normalized \
    --datasets celeba_raw_color_dct_log_scaled_normalized \
    knn

Example (train):
  python baselines.py --command train \
    --output_dir baselines/results \
    --datasets_dir datasets \
    --datasets lsun_raw_color_raw_normalized \
    eigenfaces \
    --pca_target_variance 0.95 \
    --C 0.01

Example (test):
  python3 baselines.py --command test \
    --output_dir baselines/results \
    --datasets_dir datasets \
    --classifier_name classifier_lsun_raw_color_raw_prnu_levels.3_sigma.0.8 \
    --datasets lsun_raw_color_raw \
    prnu

About

Code for the ICML 2020 paper: Leveraging Frequency Analysis for Deep Fake Image Recognition.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages