Method

OccInpFlow: Occlusion-Inpainting Optical FlowEstimation by Unsupervised Learning [OccInpFlow]
https://github.com/coolbeam/OccInpFlow

Submitted on 3 Sep. 2020 05:20 by
Kunming Luo (Megvii)

Running time:0.02 s
Environment:1 core @ 2.5 Ghz (Python)

Method Description:
Occlusion is an inevitable and critical problem
in unsupervised optical flow learning. Existing
methods either treat occlusions equally as non-
occluded regions or simply remove them to avoid
incorrectness. However, the occlusion regions
can provide effective information for optical
flow learning. In this paper, we present
OccInpFlow, an occlusion-inpainting framework
to make full use of occlusion regions.
Specifically, a new appearance-flow network is
proposed to inpaint occluded flows based on the
image content. Moreover, a boundary warp is
proposed to deal with occlusions caused by
displacement beyond image border. We conduct
experiments on multiple leading flow benchmark
data sets such as Flying Chairs, KITTI and MPI-
Sintel, which demonstrate that the performance
is significantly improved by our proposed
occlusion handling framework.
Parameters:
modele size=5.26 M
Latex Bibtex:

Detailed Results

This page provides detailed results for the method(s) selected. For each of the first 20 test images, the number of erroneous pixels at all thresholds is depicted in the table. Underneath, the left input image, the disparity / end-point error map and the estimated (and interpolated) disparity / optical flow map are shown. The error map scales linearly between 0 (black) and >=5 (white) pixels error. Red denotes all occluded pixels, falling outside the image boundaries. The false color map is scaled to the largest ground truth disparity / flow value.

Test Set Average

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.73 % 10.74 % 1.0 px 1.6 px
3 pixels 4.43 % 7.52 % 1.0 px 1.6 px
4 pixels 3.36 % 5.87 % 1.0 px 1.6 px
5 pixels 2.74 % 4.86 % 1.0 px 1.6 px
This table as LaTeX

Reflective Regions

This table as LaTeX

Test Image 0

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 5.17 % 8.16 % 0.7 px 1.0 px
3 pixels 3.30 % 5.04 % 0.7 px 1.0 px
4 pixels 2.33 % 3.83 % 0.7 px 1.0 px
5 pixels 1.79 % 3.09 % 0.7 px 1.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 1

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.69 % 6.14 % 0.6 px 0.7 px
3 pixels 3.10 % 3.89 % 0.6 px 0.7 px
4 pixels 2.22 % 2.73 % 0.6 px 0.7 px
5 pixels 1.82 % 2.13 % 0.6 px 0.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 2

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 7.40 % 10.29 % 0.8 px 1.2 px
3 pixels 5.02 % 7.71 % 0.8 px 1.2 px
4 pixels 3.74 % 6.29 % 0.8 px 1.2 px
5 pixels 2.82 % 5.23 % 0.8 px 1.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 3

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 7.52 % 11.19 % 0.8 px 0.9 px
3 pixels 4.76 % 6.86 % 0.8 px 0.9 px
4 pixels 2.83 % 3.95 % 0.8 px 0.9 px
5 pixels 1.79 % 2.42 % 0.8 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 4

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.78 % 3.11 % 0.5 px 0.6 px
3 pixels 1.08 % 1.66 % 0.5 px 0.6 px
4 pixels 0.61 % 0.94 % 0.5 px 0.6 px
5 pixels 0.49 % 0.71 % 0.5 px 0.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 5

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 5.74 % 9.66 % 1.0 px 1.3 px
3 pixels 3.60 % 6.05 % 1.0 px 1.3 px
4 pixels 2.59 % 4.32 % 1.0 px 1.3 px
5 pixels 2.10 % 3.30 % 1.0 px 1.3 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 6

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.35 % 10.78 % 0.8 px 1.0 px
3 pixels 3.86 % 6.70 % 0.8 px 1.0 px
4 pixels 2.72 % 4.14 % 0.8 px 1.0 px
5 pixels 2.13 % 2.89 % 0.8 px 1.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 7

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 8.58 % 13.80 % 1.2 px 1.9 px
3 pixels 5.93 % 10.47 % 1.2 px 1.9 px
4 pixels 4.55 % 8.47 % 1.2 px 1.9 px
5 pixels 3.75 % 7.33 % 1.2 px 1.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 8

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.66 % 8.25 % 0.9 px 1.4 px
3 pixels 2.79 % 5.35 % 0.9 px 1.4 px
4 pixels 2.05 % 4.08 % 0.9 px 1.4 px
5 pixels 1.58 % 3.34 % 0.9 px 1.4 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 9

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 10.51 % 16.81 % 1.2 px 2.0 px
3 pixels 6.31 % 11.60 % 1.2 px 2.0 px
4 pixels 4.42 % 8.16 % 1.2 px 2.0 px
5 pixels 3.44 % 6.17 % 1.2 px 2.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 10

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.17 % 10.10 % 0.6 px 0.9 px
3 pixels 2.01 % 5.76 % 0.6 px 0.9 px
4 pixels 1.33 % 3.66 % 0.6 px 0.9 px
5 pixels 1.05 % 2.48 % 0.6 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 11

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 8.16 % 11.08 % 1.0 px 1.1 px
3 pixels 4.95 % 7.71 % 1.0 px 1.1 px
4 pixels 3.12 % 5.38 % 1.0 px 1.1 px
5 pixels 2.29 % 3.77 % 1.0 px 1.1 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 12

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 8.56 % 15.72 % 1.4 px 2.1 px
3 pixels 6.74 % 12.47 % 1.4 px 2.1 px
4 pixels 5.82 % 10.71 % 1.4 px 2.1 px
5 pixels 5.20 % 9.56 % 1.4 px 2.1 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 13

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.11 % 8.66 % 0.8 px 1.6 px
3 pixels 3.03 % 6.74 % 0.8 px 1.6 px
4 pixels 2.51 % 5.69 % 0.8 px 1.6 px
5 pixels 2.19 % 4.87 % 0.8 px 1.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 14

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.54 % 10.07 % 0.7 px 0.9 px
3 pixels 3.24 % 5.61 % 0.7 px 0.9 px
4 pixels 1.66 % 3.19 % 0.7 px 0.9 px
5 pixels 1.21 % 2.11 % 0.7 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 15

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 11.88 % 18.23 % 1.5 px 2.3 px
3 pixels 8.05 % 13.28 % 1.5 px 2.3 px
4 pixels 6.07 % 10.29 % 1.5 px 2.3 px
5 pixels 4.64 % 8.17 % 1.5 px 2.3 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 16

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 2.78 % 7.51 % 0.6 px 0.8 px
3 pixels 1.01 % 4.92 % 0.6 px 0.8 px
4 pixels 0.59 % 3.79 % 0.6 px 0.8 px
5 pixels 0.32 % 2.55 % 0.6 px 0.8 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 17

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 7.96 % 12.59 % 1.0 px 2.0 px
3 pixels 5.02 % 9.65 % 1.0 px 2.0 px
4 pixels 3.95 % 8.45 % 1.0 px 2.0 px
5 pixels 3.23 % 7.46 % 1.0 px 2.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 18

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 0.59 % 2.69 % 0.4 px 0.5 px
3 pixels 0.28 % 1.39 % 0.4 px 0.5 px
4 pixels 0.14 % 0.62 % 0.4 px 0.5 px
5 pixels 0.08 % 0.23 % 0.4 px 0.5 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 19

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.77 % 4.19 % 0.4 px 0.6 px
3 pixels 1.21 % 2.79 % 0.4 px 0.6 px
4 pixels 0.98 % 2.02 % 0.4 px 0.6 px
5 pixels 0.84 % 1.53 % 0.4 px 0.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot





eXTReMe Tracker