Method

Hybrid Learning of Optical Flow and Next Frame Prediction [Next-Flow]
https://lmb.informatik.uni-freiburg.de/Publications/2017/SZB17/

Submitted on 16 Mar. 2017 16:30 by
Nima Sedaghat (Freiburg University)

Running time:0.1 s
Environment:GPU @ 1.0 Ghz (C/C++)

Method Description:
We boost CNN-based optical flow estimation in real scenes with the help of the freely available *self-supervised* task of next-frame prediction. We train the network in a hybrid way, providing it with a mixture of synthetic and real videos. With the help of a sample-variant multi-tasking architecture, the network is trained on different tasks depending on the availability of ground-truth.
Parameters:
Latex Bibtex:
@TechReport{SZB17,
author = "N. Sedaghat and M. Zolfaghari and T. Brox",
title = "Hybrid Learning of Optical Flow and Next Frame Prediction to Boost Optical Flow in the Wild",
institution = "arXiv:1612.03777",
month = " ",
year = "2017",
note = "https://arxiv.org/abs/1612.03777",
url = "http://lmb.informatik.uni-freiburg.de/Publications/2017/SZB17"
}

Detailed Results

This page provides detailed results for the method(s) selected. For each of the first 20 test images, the number of erroneous pixels at all thresholds is depicted in the table. Underneath, the left input image, the disparity / end-point error map and the estimated (and interpolated) disparity / optical flow map are shown. The error map scales linearly between 0 (black) and >=5 (white) pixels error. Red denotes all occluded pixels, falling outside the image boundaries. The false color map is scaled to the largest ground truth disparity / flow value.

Test Set Average

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 50.97 % 56.85 % 5.1 px 9.2 px
3 pixels 39.12 % 46.37 % 5.1 px 9.2 px
4 pixels 31.37 % 39.39 % 5.1 px 9.2 px
5 pixels 25.88 % 34.34 % 5.1 px 9.2 px
This table as LaTeX

Reflective Regions

This table as LaTeX

Test Image 0

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 68.40 % 72.57 % 5.4 px 8.4 px
3 pixels 55.26 % 60.90 % 5.4 px 8.4 px
4 pixels 44.27 % 50.99 % 5.4 px 8.4 px
5 pixels 31.60 % 39.69 % 5.4 px 8.4 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 1

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 44.67 % 52.19 % 3.2 px 4.5 px
3 pixels 35.17 % 43.66 % 3.2 px 4.5 px
4 pixels 27.49 % 36.60 % 3.2 px 4.5 px
5 pixels 21.68 % 30.72 % 3.2 px 4.5 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 2

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 51.26 % 59.05 % 3.5 px 6.7 px
3 pixels 36.29 % 46.35 % 3.5 px 6.7 px
4 pixels 26.85 % 38.13 % 3.5 px 6.7 px
5 pixels 20.86 % 32.64 % 3.5 px 6.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 3

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 55.20 % 62.59 % 3.6 px 6.1 px
3 pixels 36.00 % 46.51 % 3.6 px 6.1 px
4 pixels 26.77 % 38.59 % 3.6 px 6.1 px
5 pixels 21.08 % 33.44 % 3.6 px 6.1 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 4

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 42.04 % 46.62 % 3.0 px 3.6 px
3 pixels 28.51 % 33.33 % 3.0 px 3.6 px
4 pixels 21.77 % 26.59 % 3.0 px 3.6 px
5 pixels 17.83 % 22.20 % 3.0 px 3.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 5

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 54.82 % 61.78 % 5.5 px 10.6 px
3 pixels 42.40 % 51.13 % 5.5 px 10.6 px
4 pixels 34.53 % 44.28 % 5.5 px 10.6 px
5 pixels 28.72 % 39.06 % 5.5 px 10.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 6

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 50.79 % 56.77 % 3.6 px 5.2 px
3 pixels 38.57 % 45.29 % 3.6 px 5.2 px
4 pixels 29.45 % 36.08 % 3.6 px 5.2 px
5 pixels 21.73 % 28.38 % 3.6 px 5.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 7

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 53.89 % 61.54 % 5.4 px 11.3 px
3 pixels 41.42 % 51.14 % 5.4 px 11.3 px
4 pixels 32.84 % 43.99 % 5.4 px 11.3 px
5 pixels 25.33 % 37.69 % 5.4 px 11.3 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 8

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 57.40 % 64.93 % 5.9 px 13.6 px
3 pixels 41.76 % 51.95 % 5.9 px 13.6 px
4 pixels 32.79 % 44.49 % 5.9 px 13.6 px
5 pixels 27.13 % 39.78 % 5.9 px 13.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 9

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 51.28 % 58.07 % 5.5 px 10.0 px
3 pixels 40.21 % 48.14 % 5.5 px 10.0 px
4 pixels 33.98 % 42.43 % 5.5 px 10.0 px
5 pixels 28.70 % 37.58 % 5.5 px 10.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 10

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 54.45 % 62.13 % 6.2 px 12.9 px
3 pixels 41.79 % 51.42 % 6.2 px 12.9 px
4 pixels 34.17 % 44.86 % 6.2 px 12.9 px
5 pixels 29.29 % 40.60 % 6.2 px 12.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 11

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 44.49 % 52.58 % 3.3 px 6.9 px
3 pixels 30.11 % 39.96 % 3.3 px 6.9 px
4 pixels 21.60 % 32.10 % 3.3 px 6.9 px
5 pixels 17.09 % 27.54 % 3.3 px 6.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 12

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 62.03 % 67.91 % 8.4 px 13.3 px
3 pixels 50.21 % 57.78 % 8.4 px 13.3 px
4 pixels 41.31 % 50.17 % 8.4 px 13.3 px
5 pixels 36.39 % 45.82 % 8.4 px 13.3 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 13

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 43.29 % 50.06 % 4.2 px 6.9 px
3 pixels 33.99 % 41.57 % 4.2 px 6.9 px
4 pixels 28.68 % 36.32 % 4.2 px 6.9 px
5 pixels 23.93 % 31.58 % 4.2 px 6.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 14

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 63.44 % 69.66 % 8.7 px 15.8 px
3 pixels 54.48 % 62.23 % 8.7 px 15.8 px
4 pixels 47.53 % 56.45 % 8.7 px 15.8 px
5 pixels 41.21 % 51.20 % 8.7 px 15.8 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 15

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 66.14 % 71.92 % 6.1 px 12.8 px
3 pixels 54.13 % 61.69 % 6.1 px 12.8 px
4 pixels 45.99 % 54.76 % 6.1 px 12.8 px
5 pixels 39.71 % 49.47 % 6.1 px 12.8 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 16

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 63.78 % 67.39 % 3.9 px 5.7 px
3 pixels 43.94 % 49.50 % 3.9 px 5.7 px
4 pixels 31.88 % 38.57 % 3.9 px 5.7 px
5 pixels 24.09 % 31.41 % 3.9 px 5.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 17

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 56.14 % 62.99 % 4.1 px 9.2 px
3 pixels 42.88 % 51.77 % 4.1 px 9.2 px
4 pixels 31.60 % 42.22 % 4.1 px 9.2 px
5 pixels 24.09 % 35.86 % 4.1 px 9.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 18

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 68.63 % 71.84 % 6.0 px 9.2 px
3 pixels 51.91 % 56.83 % 6.0 px 9.2 px
4 pixels 40.23 % 46.34 % 6.0 px 9.2 px
5 pixels 33.56 % 40.36 % 6.0 px 9.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 19

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 34.17 % 41.19 % 2.7 px 4.4 px
3 pixels 21.95 % 29.31 % 2.7 px 4.4 px
4 pixels 16.00 % 23.30 % 2.7 px 4.4 px
5 pixels 12.66 % 19.86 % 2.7 px 4.4 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot





eXTReMe Tracker