Skip to content

Latest commit

 

History

History
125 lines (116 loc) · 5.34 KB

README.md

File metadata and controls

125 lines (116 loc) · 5.34 KB
Raw

IRR

Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation

Abstract

Deep learning approaches to optical flow estimation have seen rapid progress over the recent years. One common trait of many networks is that they refine an initial flow estimate either through multiple stages or across the levels of a coarse-to-fine representation. While leading to more accurate results, the downside of this is an increased number of parameters. Taking inspiration from both classical energy minimization approaches as well as residual networks, we propose an iterative residual refinement (IRR) scheme based on weight sharing that can be combined with several backbone networks. It reduces the number of parameters, improves the accuracy, or even achieves both. Moreover, we show that integrating occlusion prediction and bi-directional flow estimation into our IRR scheme can further boost the accuracy. Our full network achieves state- of-the-art results for both optical flow and occlusion estimation across several standard datasets.

Results and Models

Models Training datasets FlyingChairsOcc Sintel (training) KITTI2015 (training) Log Config Download
clean final Fl-all EPE
IRR-PWC FlyingChairsOcc 1.44 2.38 3.86 - - log Config Model
IRR-PWC FlyingChairsOcc + FlyingThing3d subset - 1.79 3.38 25.06% 8.32 log Config Model
IRR-PWC-ft FlyingChairsOcc + FlyingThing3d subset + Sintel - 1.51 2.18 - - log Config Model
IRR-PWC-ft-final FlyingChairsOcc + FlyingThing3d subset + Sintel final - 1.71 1.94 - - log Config Model
IRR-PWC-ft FlyingChairsOcc + FlyingThing3d subset + KITTI - - - 8.51% 2.19 log Config Model

Citation

@inproceedings{hur2019iterative,
  title={Iterative residual refinement for joint optical flow and occlusion estimation},
  author={Hur, Junhwa and Roth, Stefan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={5754--5763},
  year={2019}
}