Skip to content

Source code for the paper: Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

License

Notifications You must be signed in to change notification settings

HoiM/MobileFSGAN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

Introduction

This is the repository for the source code of the papar
Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

Accepted to IEEE International Conference on Multimedia and Expo 2022

[ICME 2022] [arXiv]

Framework overview

Selected results

Preferable configurations

torch==1.6.0

torchvision==0.7.0

cuda version: 10.2

How to use the code

Firstly, I strongly recommend implementing your own dataset class for loading training data, since my implemenation is based the structure of my data folder.

class FaceShifterDataset(torch.utils.data.TensorDataset):
    def __init__(self):
        super(FaceShifterDataset, self).__init__()
        # TODO: add your code

    def __getitem__(self, item):
        # TODO: add your code
        return Xs, Xt, GT, with_gt, src_as_true

    def __len__(self):
        # TODO: add your code
        return len(self.data_list)

Basically, the __getitem__ function returns five elements:

  • Xs: the source image.
  • Xt: the target image.
  • GT: the ground truth image. If none, an equivalent tensor with all values being -1.
  • with_gt: (torch.float32) whether the training data has a ground truth label (the value can be either 1 or 0).
  • src_as_true: (bool) used to indicate whether the source or the target is a real image. Selected real images are used to feed the discrimiator.

Note that images are with shape being (3, 256, 256) and values in (-1, 1). Images are in format torch.tensor with dtype being torch.float32.

After that, reset the arugments inside main.py and refer to run_training.sh. Note that my implementation uses DistributedDataParallel on 4 GPUs within one machine. Please adjust the settings based on your own hardware.

Performance

Qualitative performance

Comparisons of model size
Model Model Size (MB)
FaceShifter (PyTorch) 669.0 + 64.0 + 168.0
MegaFS (PyTorch) 3529.0 + 364.0
SimSwap (PyTorch) 766.9 + 220.2
Ours (Pytorch) 10.2 + 4.9
Ours (CoreML) 15.0
Ours (TNN) 14.8

Citation

If you find my work useful, please cite:

@article{mobilefsgan,
author = {Haiming Yu and Hao Zhu and Xiangju Lu and Junhui Liu},
title = {Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution},
journal = {in ICME}, 
year = 2022
}

About

Source code for the paper: Migrating Face Swap to Mobile Devices: A lightweight Framework and A Supervised Training Solution

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published