Skip to content

Occlusion Aware Unsupervised Learning of Optical Flow From Video

Notifications You must be signed in to change notification settings

HaiyangPeng/UnOpticalFlow

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

50 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

We proposed a new method of dealing with occulsion problem in unsupervised learning of optical flow by calculating occlusion mask. Compared with UnFlow(AAAI 2018) and OAFlow(CVPR 2018), we achieved more precise results in KITTI dataset.

method KITTI 2012 KITTI 2015 train set KITTI 2015 test set
UnFlow 3.78 8.80 23.27%
OAFlow 3.55 8.88 31.2%
Ours 2.67 7.1 22%

Installation

The code is based on Python3.6. You could use either virtualenv or conda to setup a specified environment. And then run:

pip install -r requirements.txt

Run experiments

Prepare training data:

  1. Download KITTI raw dataset using the script provided on the official website. You also need to download KITTI 2015 dataset to evaluate the predicted optical flow.

Training:

  1. Modify the configuration file in the ./config directory to set up your path. The config file contains the important paths and default hyper-parameters used in the training process.
1. python train.py --config_file ./config/kitti.yaml --gpu [gpu_id] --mode flow --prepared_save_dir [name_of_your_prepared_dataset] --model_dir [your/directory/to/save/training/models]

If you are running experiments on the dataset for the first time, it would first process data and save in the [prepared_base_dir] path defined in your config file.

Evaluation:

the network weights after training on kitti raw data is here

  1. To evaluate the optical flow estimation on KITTI 2015, run:
python test.py --config_file ./config/kitti.yaml --gpu [gpu_id] --mode flow --task kitti_flow --pretrained_model [path/to/your/model]

Acknowledgement

We implemented our idea based on TrainFlow

Cite

@inproceedings{10.1117/12.2588381, author = {Jianfeng Li and Junqiao Zhao and Shuangfu Song and Tiantian Feng}, title = {{Occlusion aware unsupervised learning of optical flow from video}}, volume = {11605}, booktitle = {Thirteenth International Conference on Machine Vision}, editor = {Wolfgang Osten and Dmitry P. Nikolaev and Jianhong Zhou}, organization = {International Society for Optics and Photonics}, publisher = {SPIE}, pages = {224 -- 231}, year = {2021}, doi = {10.1117/12.2588381}, URL = {https://doi.org/10.1117/12.2588381} }

About

Occlusion Aware Unsupervised Learning of Optical Flow From Video

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%