Method

OAS-Net: Occlusion Aware Sampling Network for Accurate Optical Flow [OAS-Net]


Submitted on 14 Aug. 2020 03:58 by
Lingtong Kong (SJTU)

Running time:0.03 s
Environment:NVIDIA GTX 1080 Ti

Method Description:
Optical flow is an essential step for many
computer vision tasks. Existing deep networks have
achieved satisfactory results by mostly employing
a pyramidal coarse-to-fine paradigm, where a key
process is to adopt warped target feature based on
previous flow prediction to correlate with source
feature for building 3D matching cost volume.
However, the warping operation can lead to
troublesome ghosting problem that results in
ambiguity. Moreover, occluded areas are treated
equally with non occluded regions in most existing
works, which may cause performance degradation. To
deal with these challenges, we propose a
lightweight yet efficient optical flow network,
named OAS-Net for accurate optical flow. First, a
new sampling based correlation layer is employed
without noisy warping operation. Second, a novel
occlusion aware module is presented to make raw
cost volume conscious of occluded regions. Third,
a shared flow and occlusion awareness decoder is
adopted for structure compactness.
Parameters:
OAS-Net
Latex Bibtex:
@inproceedings{kong2021oasnet,
author={Lingtong Kong and Xiaohang Yang and Jie
Yang},
booktitle={IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP)},
title={OAS-Net: Occlusion Aware Sampling Network
for Accurate Optical Flow},
year={2021},}

Detailed Results

This page provides detailed results for the method(s) selected. For each of the first 20 test images, the number of erroneous pixels at all thresholds is depicted in the table. Underneath, the left input image, the disparity / end-point error map and the estimated (and interpolated) disparity / optical flow map are shown. The error map scales linearly between 0 (black) and >=5 (white) pixels error. Red denotes all occluded pixels, falling outside the image boundaries. The false color map is scaled to the largest ground truth disparity / flow value.

Test Set Average

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.66 % 9.25 % 0.7 px 1.4 px
3 pixels 2.88 % 6.41 % 0.7 px 1.4 px
4 pixels 2.13 % 5.01 % 0.7 px 1.4 px
5 pixels 1.70 % 4.14 % 0.7 px 1.4 px
This table as LaTeX

Reflective Regions

This table as LaTeX

Test Image 0

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.15 % 8.91 % 0.7 px 1.2 px
3 pixels 2.65 % 6.36 % 0.7 px 1.2 px
4 pixels 2.02 % 4.95 % 0.7 px 1.2 px
5 pixels 1.79 % 4.08 % 0.7 px 1.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 1

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.72 % 8.42 % 0.6 px 0.9 px
3 pixels 2.76 % 5.76 % 0.6 px 0.9 px
4 pixels 1.96 % 4.67 % 0.6 px 0.9 px
5 pixels 1.29 % 3.76 % 0.6 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 2

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 2.64 % 6.77 % 0.5 px 0.9 px
3 pixels 1.22 % 3.89 % 0.5 px 0.9 px
4 pixels 0.65 % 2.77 % 0.5 px 0.9 px
5 pixels 0.52 % 2.37 % 0.5 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 3

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 2.38 % 7.05 % 0.4 px 0.7 px
3 pixels 1.48 % 4.54 % 0.4 px 0.7 px
4 pixels 1.03 % 3.30 % 0.4 px 0.7 px
5 pixels 0.74 % 2.53 % 0.4 px 0.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 4

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.08 % 2.87 % 0.4 px 0.5 px
3 pixels 0.49 % 1.20 % 0.4 px 0.5 px
4 pixels 0.36 % 0.59 % 0.4 px 0.5 px
5 pixels 0.28 % 0.44 % 0.4 px 0.5 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 5

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.79 % 8.00 % 0.7 px 1.8 px
3 pixels 2.32 % 5.64 % 0.7 px 1.8 px
4 pixels 1.78 % 4.62 % 0.7 px 1.8 px
5 pixels 1.41 % 4.00 % 0.7 px 1.8 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 6

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.33 % 8.10 % 0.6 px 0.9 px
3 pixels 2.19 % 4.75 % 0.6 px 0.9 px
4 pixels 1.26 % 3.25 % 0.6 px 0.9 px
5 pixels 1.02 % 2.35 % 0.6 px 0.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 7

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.62 % 11.85 % 0.9 px 1.5 px
3 pixels 3.36 % 7.52 % 0.9 px 1.5 px
4 pixels 2.08 % 5.43 % 0.9 px 1.5 px
5 pixels 1.78 % 4.53 % 0.9 px 1.5 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 8

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.60 % 10.83 % 0.9 px 1.6 px
3 pixels 2.80 % 7.58 % 0.9 px 1.6 px
4 pixels 2.11 % 5.97 % 0.9 px 1.6 px
5 pixels 1.79 % 4.90 % 0.9 px 1.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 9

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 5.42 % 11.16 % 0.8 px 1.4 px
3 pixels 2.57 % 7.02 % 0.8 px 1.4 px
4 pixels 1.45 % 5.01 % 0.8 px 1.4 px
5 pixels 1.01 % 3.78 % 0.8 px 1.4 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 10

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.07 % 9.65 % 0.6 px 1.0 px
3 pixels 1.87 % 5.60 % 0.6 px 1.0 px
4 pixels 1.35 % 3.81 % 0.6 px 1.0 px
5 pixels 1.04 % 2.81 % 0.6 px 1.0 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 11

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.04 % 8.77 % 0.6 px 1.2 px
3 pixels 2.11 % 5.75 % 0.6 px 1.2 px
4 pixels 1.53 % 4.32 % 0.6 px 1.2 px
5 pixels 1.30 % 3.14 % 0.6 px 1.2 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 12

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 9.07 % 13.89 % 1.3 px 1.8 px
3 pixels 7.15 % 11.41 % 1.3 px 1.8 px
4 pixels 6.21 % 10.04 % 1.3 px 1.8 px
5 pixels 5.72 % 9.14 % 1.3 px 1.8 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 13

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.55 % 8.61 % 0.9 px 1.7 px
3 pixels 3.58 % 5.96 % 0.9 px 1.7 px
4 pixels 3.10 % 4.81 % 0.9 px 1.7 px
5 pixels 2.85 % 4.28 % 0.9 px 1.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 14

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.44 % 6.49 % 0.4 px 0.6 px
3 pixels 0.75 % 3.71 % 0.4 px 0.6 px
4 pixels 0.54 % 2.20 % 0.4 px 0.6 px
5 pixels 0.41 % 1.26 % 0.4 px 0.6 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 15

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 8.65 % 15.25 % 0.9 px 1.9 px
3 pixels 5.63 % 10.62 % 0.9 px 1.9 px
4 pixels 4.43 % 8.34 % 0.9 px 1.9 px
5 pixels 3.67 % 6.86 % 0.9 px 1.9 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 16

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 2.86 % 7.18 % 0.5 px 0.7 px
3 pixels 1.22 % 4.69 % 0.5 px 0.7 px
4 pixels 0.78 % 3.42 % 0.5 px 0.7 px
5 pixels 0.56 % 2.44 % 0.5 px 0.7 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 17

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.92 % 12.55 % 0.9 px 2.1 px
3 pixels 4.86 % 9.52 % 0.9 px 2.1 px
4 pixels 3.97 % 8.28 % 0.9 px 2.1 px
5 pixels 3.34 % 7.56 % 0.9 px 2.1 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 18

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 0.54 % 2.35 % 0.3 px 0.4 px
3 pixels 0.25 % 1.06 % 0.3 px 0.4 px
4 pixels 0.16 % 0.47 % 0.3 px 0.4 px
5 pixels 0.06 % 0.21 % 0.3 px 0.4 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot



Test Image 19

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.45 % 3.34 % 0.4 px 0.5 px
3 pixels 0.83 % 2.05 % 0.4 px 0.5 px
4 pixels 0.61 % 1.29 % 0.4 px 0.5 px
5 pixels 0.44 % 0.80 % 0.4 px 0.5 px
This table as LaTeX





This figure as: png eps pdf txt gnuplot





eXTReMe Tracker