Method

Horn-Schunck Optical Flow with a Multi-Scale Strategy [H+S_ROB]
http://www.ipol.im/pub/art/2013/20/

Submitted on 26 Mar. 2018 15:39 by
Alexander Brock (HCI)

 Running time: 8 s Environment: 4 cores @ 2.5 Ghz (C/C++)

 Method Description: The seminal work of Horn and Schunck is the first variational method for optical flow estimation. It introduced a novel framework where the optical flow is computed as the solution of a minimization problem. From the assumption that pixel intensities do not change over time, the optical flow constraint equation is derived. This equation relates the optical flow with the derivatives of the image. There are infinitely many vector fields that satisfy the optical flow constraint, thus the problem is ill-posed. To overcome this problem, Horn and Schunck introduced an additional regularity condition that restricts the possible solutions. Their method minimizes both the optical flow constraint and the magnitude of the variations of the flow field, producing smooth vector fields. One of the limitations of this method is that, typically, it can only estimate small motions. In the presence of large displacements, this method fails when the gradient of the image is not smooth enough. In this work, we describe an implementation of the original Horn and Schunck method and also introduce a multi-scale strategy in order to deal with larger displacements. For this multi-scale strategy, we create a pyramidal structure of downsampled images and change the optical flow constraint equation with a nonlinear formulation. In order to tackle this nonlinear formula, we linearize it and solve the method iteratively in each scale. In this sense, there are two common approaches: one approach that computes the motion increment in the iterations; or the one we follow, that computes the full flow during the iterations. The solutions are incrementally refined over the scales. This pyramidal structure is a standard tool in many optical flow methods. Parameters: processors: 4 alpha: 100 nscales: 10 zoom_factor: .95 nwarps: 5 TOL: 0.001 maxiter: 100 Latex Bibtex: @article{ipol.2013.20, title = {{Horn-Schunck Optical Flow with a Multi-Scale Strategy}}, author = {Meinhardt-Llopis, Enric and Sánchez Pérez, Javier and Kondermann, Daniel}, journal = {{Image Processing On Line}}, volume = {3}, pages = {151--172}, year = {2013}, doi = {10.5201/ipol.2013.20}, }

Detailed Results

This page provides detailed results for the method(s) selected. For the first 20 test images, the percentage of erroneous pixels is depicted in the table. We use the error metric described in Object Scene Flow for Autonomous Vehicles (CVPR 2015), which considers a pixel to be correctly estimated if the disparity or flow end-point error is <3px or <5% (for scene flow this criterion needs to be fulfilled for both disparity maps and the flow map). Underneath, the left input image, the estimated results and the error maps are shown (for disp_0/disp_1/flow/scene_flow, respectively). The error map uses the log-color scale described in Object Scene Flow for Autonomous Vehicles (CVPR 2015), depicting correct estimates (<3px or <5% error) in blue and wrong estimates in red color tones. Dark regions in the error images denote the occluded pixels which fall outside the image boundaries. The false color maps of the results are scaled to the largest ground truth disparity values / flow magnitudes.

Test Set Average

 Error Fl-bg Fl-fg Fl-all All / All 68.22 76.49 69.60 All / Est 68.22 76.49 69.60 Noc / All 62.55 74.96 64.80 Noc / Est 62.55 74.96 64.80
This table as LaTeX

Test Image 0

 Error Fl-bg Fl-fg Fl-all All / All 87.44 100.00 89.16 All / Est 87.44 100.00 89.16 Noc / All 85.88 100.00 88.02 Noc / Est 85.88 100.00 88.02
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 1

 Error Fl-bg Fl-fg Fl-all All / All 77.94 100.00 80.40 All / Est 77.94 100.00 80.40 Noc / All 75.26 100.00 78.32 Noc / Est 75.26 100.00 78.32
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 2

 Error Fl-bg Fl-fg Fl-all All / All 81.18 99.23 82.06 All / Est 81.18 99.23 82.06 Noc / All 77.25 99.23 78.53 Noc / Est 77.25 99.23 78.53
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 3

 Error Fl-bg Fl-fg Fl-all All / All 85.15 96.81 86.23 All / Est 85.15 96.81 86.23 Noc / All 81.65 95.74 82.86 Noc / Est 81.65 95.74 82.86
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 4

 Error Fl-bg Fl-fg Fl-all All / All 67.95 93.88 72.25 All / Est 67.95 93.88 72.25 Noc / All 61.61 93.66 67.60 Noc / Est 61.61 93.66 67.60
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 5

 Error Fl-bg Fl-fg Fl-all All / All 67.09 0.00 61.06 All / Est 67.09 0.00 61.06 Noc / All 61.80 0.00 55.45 Noc / Est 61.80 0.00 55.45
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 6

 Error Fl-bg Fl-fg Fl-all All / All 65.22 3.54 58.72 All / Est 65.22 3.54 58.72 Noc / All 60.12 3.54 53.38 Noc / Est 60.12 3.54 53.38
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 7

 Error Fl-bg Fl-fg Fl-all All / All 2.68 100.00 21.73 All / Est 2.68 100.00 21.73 Noc / All 2.68 100.00 20.87 Noc / Est 2.68 100.00 20.87
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 8

 Error Fl-bg Fl-fg Fl-all All / All 5.58 100.00 23.02 All / Est 5.58 100.00 23.02 Noc / All 5.58 100.00 23.02 Noc / Est 5.58 100.00 23.02
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 9

 Error Fl-bg Fl-fg Fl-all All / All 8.01 100.00 31.51 All / Est 8.01 100.00 31.51 Noc / All 8.01 100.00 31.51 Noc / Est 8.01 100.00 31.51
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 10

 Error Fl-bg Fl-fg Fl-all All / All 67.01 95.29 73.47 All / Est 67.01 95.29 73.47 Noc / All 61.14 95.29 70.02 Noc / Est 61.14 95.29 70.02
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 11

 Error Fl-bg Fl-fg Fl-all All / All 68.87 83.63 71.52 All / Est 68.87 83.63 71.52 Noc / All 64.10 83.62 68.03 Noc / Est 64.10 83.62 68.03
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 12

 Error Fl-bg Fl-fg Fl-all All / All 76.10 81.08 76.44 All / Est 76.10 81.08 76.44 Noc / All 71.53 81.08 72.29 Noc / Est 71.53 81.08 72.29
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 13

 Error Fl-bg Fl-fg Fl-all All / All 78.27 100.00 80.94 All / Est 78.27 100.00 80.94 Noc / All 73.76 100.00 76.81 Noc / Est 73.76 100.00 76.81
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 14

 Error Fl-bg Fl-fg Fl-all All / All 79.79 98.37 80.11 All / Est 79.79 98.37 80.11 Noc / All 75.70 98.37 76.17 Noc / Est 75.70 98.37 76.17
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 15

 Error Fl-bg Fl-fg Fl-all All / All 89.46 78.81 88.49 All / Est 89.46 78.81 88.49 Noc / All 86.25 78.81 85.40 Noc / Est 86.25 78.81 85.40
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 16

 Error Fl-bg Fl-fg Fl-all All / All 79.91 100.00 82.87 All / Est 79.91 100.00 82.87 Noc / All 74.48 100.00 79.07 Noc / Est 74.48 100.00 79.07
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 17

 Error Fl-bg Fl-fg Fl-all All / All 83.58 87.23 83.96 All / Est 83.58 87.23 83.96 Noc / All 79.29 87.23 80.32 Noc / Est 79.29 87.23 80.32
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 18

 Error Fl-bg Fl-fg Fl-all All / All 80.90 100.00 89.97 All / Est 80.90 100.00 89.97 Noc / All 74.63 100.00 81.63 Noc / Est 74.63 100.00 81.63
This table as LaTeX

Input Image

Flow Result

Flow Error

Test Image 19

 Error Fl-bg Fl-fg Fl-all All / All 83.47 89.49 84.15 All / Est 83.47 89.49 84.15 Noc / All 78.07 89.49 79.72 Noc / Est 78.07 89.49 79.72
This table as LaTeX

Input Image

Flow Result

Flow Error