Stereo Evaluation 2012


The stereo / flow benchmark consists of 194 training image pairs and 195 test image pairs, saved in loss less png format. Our evaluation server computes the average number of bad pixels for all non-occluded or occluded (=all groundtruth) pixels. We require that all methods use the same parameter set for all test pairs. Our development kit provides details about the data format as well as MATLAB / C++ utility functions for reading and writing disparity maps and flow fields.

Our evaluation table ranks all methods according to the number of non-occluded erroneous pixels at the specified disparity / end-point error threshold. All methods providing less than 100 % density have been interpolated using simple background interpolation as explained in the corresponding header file in the development kit. For each method we show:

  • Out-Noc: Percentage of erroneous pixels in non-occluded areas
  • Out-All: Percentage of erroneous pixels in total
  • Avg-Noc: Average disparity / end-point error in non-occluded areas
  • Avg-All: Average disparity / end-point error in total
  • Density: Percentage of pixels for which ground truth has been provided by the method

Note: On 04.11.2013 we have improved the ground truth disparity maps and flow fields leading to slightly improvements for all methods. Please download the stereo/flow dataset with the improved ground truth for training again, if you have downloaded the dataset prior to 04.11.2013. Please consider reporting these new number for all future submissions. Links to last leaderboards before the updates: stereo and flow!

Important Policy Update: As more and more non-published work and re-implementations of existing work is submitted to KITTI, we have established a new policy: from now on, only submissions with significant novelty that are leading to a peer-reviewed paper in a conference or journal are allowed. Minor modifications of existing algorithms or student research projects are not allowed. Such work must be evaluated on a split of the training set. To ensure that our policy is adopted, new users must detail their status, describe their work and specify the targeted venue during registration. Furthermore, we will regularly delete all entries that are 6 months old but are still anonymous or do not have a paper associated with them. For conferences, 6 month is enough to determine if a paper has been accepted and to add the bibliography information. For longer review cycles, you need to resubmit your results.
Additional information used by the methods
  • Flow: Method uses optical flow (2 temporally adjacent images)
  • Multiview: Method uses more than 2 temporally adjacent images
  • Motion stereo: Method uses epipolar geometry for computing optical flow
  • Additional training data: Use of additional data sources for training (see details)

Table        Error threshold        Evaluation area

Method Setting Code Out-Noc Out-All Avg-Noc Avg-All Density Runtime Environment
1 0.00 % 0.00 % 0.0 px 0.0 px 0.00 %
2 0.00 % 0.00 % 0.0 px 0.0 px 0.00 %
3 AcfNet code 1.17 % 1.54 % 0.5 px 0.5 px 100.00 % 0.48 s 1 core @ 2.5 Ghz (Python)
Y. Zhang, Y. Chen, X. Bai, S. Yu, K. Yu, Z. Li and K. Yang: Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching. AAAI 2020.
4 GANet-deep code 1.19 % 1.60 % 0.4 px 0.5 px 100.00 % 1.8 s GPU @ 2.5 Ghz (Python)
F. Zhang, V. Prisacariu, R. Yang and P. Torr: GA-Net: Guided Aggregation Net for End-to-end Stereo Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
5 M2S_CSPN 1.19 % 1.53 % 0.4 px 0.5 px 100.00 % 0.5 s GPU @ 2.5 Ghz (C/C++)
X. Cheng, P. Wang and R. Yang: Learning Depth with Convolutional Spatial Propagation Network. arXiv preprint arXiv:1810.02695 2018.
6 Dedge-AGMNet 1.26 % 1.64 % 0.4 px 0.5 px 100.00 % 0.9 s 1 core @ 2.5 Ghz (Python)
7 EMCUA 1.26 % 1.64 % 0.4 px 0.5 px 100.00 % 0.9 s GPU @ 1.5 Ghz (Python + C/C++)
8 NCA-Net 1.28 % 1.68 % 0.5 px 0.5 px 100.00 % 0.5 s GPU @ 2.5 Ghz (Python)
9 MCUA 1.30 % 1.70 % 0.5 px 0.5 px 100.00 % 0.40 s 1 core @ 2.5 Ghz (C/C++)
10 PhvNet 1.30 % 1.68 % 0.5 px 0.5 px 100.00 % 52 s 1 core @ 2.5 Ghz (Java + C/C++)
11 GDNet 1.31 % 1.72 % 0.4 px 0.5 px 100.00 % 0.45 s GPU @ 2.5 Ghz (Python)
12 AMNet 1.32 % 1.73 % 0.5 px 0.5 px 100.00 % 0.9 s GPU @ 2.5 Ghz (Python)
X. Du, M. El-Khamy and J. Lee: AMNet: Deep Atrous Multiscale Stereo Disparity Estimation Networks. 2019.
13 NLCA-Net 1.32 % 1.72 % 0.4 px 0.5 px 100.00 % 0.6 s GPU @ 2.5 Ghz (C/C++)
14 GwcNet-gc code 1.32 % 1.70 % 0.5 px 0.5 px 100.00 % 0.32 s GPU @ 2.0 Ghz (Java + C/C++)
X. Guo, K. Yang, W. Yang, X. Wang and H. Li: Group-wise correlation stereo network. CVPR 2019.
15 HcNet 1.32 % 1.71 % 0.5 px 0.5 px 100.00 % 0.48 s GPU @ 2.5 Ghz (Python)
16 SDCNN 1.34 % 1.78 % 0.5 px 0.6 px 100.00 % 0.5 s GPU @ 3.0 Ghz (Python)
17 APMNet 1.34 % 1.74 % 0.5 px 0.5 px 100.00 % 0.5 s GPU @ 2.5 Ghz (Python)
18 MAN 1.35 % 1.76 % 0.5 px 0.5 px 100.00 % 1.5 s 1 core @ 2.5 Ghz (Python)
19 oos 1.35 % 1.81 % 0.5 px 0.6 px 100.00 % 0.4 s GPU @ 2.5 Ghz (Python + C/C++)
20 GANet-15 code 1.36 % 1.80 % 0.5 px 0.5 px 100.00 % 0.36 s 1 core @ 2.5 Ghz (C/C++)
F. Zhang, V. Prisacariu, R. Yang and P. Torr: GA-Net: Guided Aggregation Net for End-to-end Stereo Matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
21 MS-Net 1.38 % 1.79 % 0.5 px 0.5 px 100.00 % 0.75 s 1 core @ 2.5 Ghz (C/C++)
22 PCF-S 1.38 % 1.79 % 0.5 px 0.5 px 100.00 % 0.11 s GPU @ 2.5 Ghz (Python)
23 SDEA 1.38 % 1.80 % 0.5 px 0.5 px 100.00 % 0.40 s 1 core @ 2.5 Ghz (Python)
24 DSM 1.39 % 1.79 % 0.5 px 0.5 px 100.00 % 0.4 s 1 core @ 2.5 Ghz (C/C++)
25 SLED-Net 1.39 % 1.80 % 0.5 px 0.5 px 100.00 % 0.75 s 1 core @ 2.5 Ghz (C/C++)
26 PSMNet_R 1.40 % 1.81 % 0.5 px 0.5 px 100.00 % 0.5 s GPU @ 2.5 Ghz (Python)
27 HD^3-Stereo code 1.40 % 1.80 % 0.5 px 0.5 px 100.00 % 0.14 s NVIDIA Pascal Titan XP
Z. Yin, T. Darrell and F. Yu: Hierarchical Discrete Distribution Decomposition for Match Density Estimation. CVPR 2019.
28 DFNet 1.41 % 1.82 % 0.5 px 0.5 px 100.00 % 0.7 s 1 core @ 2.5 Ghz (Python)
29 MPA-Net 1.41 % 1.83 % 0.5 px 0.5 px 100.00 % 0.6 s 1 core @ 2.5 Ghz (C/C++)
30 CFP-Net code 1.41 % 1.83 % 0.5 px 0.5 px 100.00 % 0.95 s 8 cores @ 2.5 Ghz (Python)
Z. Zhu, M. He, Y. Dai, Z. Rao and B. Li: Multi-scale Cross-form Pyramid Network for Stereo Matching. arXiv preprint 2019.
31 WSMCnetEB_S2C3 code 1.42 % 1.90 % 0.6 px 0.6 px 100.00 % 0.39 s GPU @ Nvidia GTX 1070 (Pytorch)
32 Stereo-DRNet 1.42 % 1.83 % 0.5 px 0.5 px 100.00 % 0.23 s GPU @ 2.5 Ghz (Python)
33 CANet 1.45 % 1.89 % 0.5 px 0.6 px 100.00 % 4.7 s 1 core @ 2.5 Ghz (Python)
34 cnet code 1.46 % 1.96 % 0.5 px 0.5 px 100.00 % 0.8 s 1 core @ 2.5 Ghz (C/C++)
35 unet 1.46 % 1.88 % 0.5 px 0.5 px 100.00 % 0.6 s GPU @ 2.5 Ghz (Python)
36 EdgeStereo-V2 1.46 % 1.83 % 0.4 px 0.5 px 100.00 % 0.32 s Nvidia GTX Titan Xp
X. Song, X. Zhao, L. Fang and H. Hu: EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detection. arXiv preprint arXiv:1903.01700 2019.
37 ALab 1.47 % 1.88 % 0.5 px 0.5 px 100.00 % 0.6 s 1 core @ 2.5 Ghz (C/C++)
38 WCRAPNet 1.47 % 1.88 % 0.5 px 0.5 px 100.00 % 0.5 s 1 core @ 2.5 Ghz (Python)
39 PSM_CA 1.47 % 1.91 % 0.5 px 0.6 px 100.00 % 4.6 s 1 core @ 2.5 Ghz (Python)
40 CMF 1.48 % 2.63 % 0.5 px 0.6 px 100.00 % 0.5 s GPU @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
41 PSMNet code 1.49 % 1.89 % 0.5 px 0.6 px 100.00 % 0.41 s Nvidia Titan Xp
J. Chang and Y. Chen: Pyramid Stereo Matching Network. arXiv preprint arXiv:1803.08669 2018.
42 HSM code 1.53 % 1.99 % 0.5 px 0.6 px 100.00 % 0.15 s Titan X Pascal
G. Yang, J. Manela, M. Happold and D. Ramanan: Hierarchical Deep Stereo Matching on High- Resolution Images. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
43 CAR 1.54 % 1.96 % 0.5 px 0.6 px 100.00 % 0.11 s Nvidia GTX Titan Xp
44 LWSM 1.55 % 2.06 % 0.5 px 0.6 px 100.00 % 0.24 s GPU @ 2.5 Ghz (Matlab + C/C++)
45 CAR 1.59 % 2.01 % 0.5 px 0.6 px 100.00 % 0.11 s GPU @ 3.0 Ghz (Python)
46 GCV 1.61 % 2.08 % 0.5 px 0.6 px 100.00 % 0.78 s 1 cores @ 2.5 Ghz (Python)
47 MSDC-Net 1.63 % 2.09 % 0.5 px 0.6 px 100.00 % 0.6 s 1 core @ 2.5 Ghz (C/C++)
Z. Rao, M. He, Y. Dai, Z. Zhu, B. Li and R. He: MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching. arXiv preprint arXiv:1904.12658 2019.
48 WaveletStereo 1.66 % 2.18 % 0.5 px 0.6 px 100.00 % 0.27 s 1 core @ 2.5 Ghz (C/C++)
. Anonymous: WaveletStereo: Learning wavelet coefficients for stereo matching. arXiv: Computer Vision and Pattern Recognition 2019.
49 SegStereo code 1.68 % 2.03 % 0.5 px 0.6 px 100.00 % 0.6 s Nvidia GTX Titan Xp
G. Yang, H. Zhao, J. Shi, Z. Deng and J. Jia: SegStereo: Exploiting Semantic Information for Disparity Estimation. ECCV 2018.
50 DG-Net 1.68 % 2.25 % 0.5 px 0.5 px 100.00 % 0.03 s 1 core @ 2.5 Ghz (C/C++)
51 AutoDispNet-CSS 1.70 % 2.05 % 0.5 px 0.5 px 100.00 % 0.9 s 1 core @ 2.5 Ghz (C/C++)
T. Saikia, Y. Marrakchi, A. Zela, F. Hutter and T. Brox: AutoDispNet: Improving Disparity Estimation with AutoML. The IEEE International Conference on Computer Vision (ICCV) 2019.
52 CooperativeStereo 1.71 % 2.19 % 0.5 px 0.6 px 100.00 % 0.9 s 1 core @ 2.5 Ghz (Python + C/C++)
53 iResNet-i2 code 1.71 % 2.16 % 0.5 px 0.6 px 100.00 % 0.12 s 1 core @ 2.5 Ghz (C/C++)
Z. Liang, Y. Feng, Y. Guo, H. Liu, W. Chen, L. Qiao, L. Zhou and J. Zhang: Learning for disparity estimation through feature constancy. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018.
54 SF-Net
This method uses optical flow information.
1.73 % 2.16 % 0.5 px 0.5 px 100.00 % 0.06 s GPU, GTX 1080Ti
55 Ours 1.75 % 2.23 % 0.6 px 0.6 px 100.00 % 0.03 s GPU @ 2.5 Ghz (Python)
56 GC-NET 1.77 % 2.30 % 0.6 px 0.7 px 100.00 % 0.9 s Nvidia GTX Titan X
A. Kendall, H. Martirosyan, S. Dasgupta, P. Henry, R. Kennedy, A. Bachrach and A. Bry: End-to-End Learning of Geometry and Context for Deep Stereo Regression. Proceedings of the International Conference on Computer Vision (ICCV) 2017.
57 NWPU-stereo 1.80 % 2.22 % 0.5 px 0.6 px 100.00 % 0.25 s Nvidia GTX Titan X
58 HDRNet code 1.81 % 2.30 % 0.5 px 0.6 px 100.00 % 0.3 s 1 core @ 2.5 Ghz (C/C++)
59 SN 1.83 % 2.43 % 0.5 px 0.6 px 100.00 % 1 s 1 core @ 2.5 Ghz (C/C++)
60 HITNet 1.84 % 2.35 % 0.5 px 0.6 px 100.00 % 0.02 s GPU @ 2.5 Ghz (C/C++)
61 FBW-Net 1.84 % 2.80 % 0.5 px 0.8 px 100.00 % 2 s GPU @ 2.5 Ghz (Python)
62 PDSNet 1.92 % 2.53 % 0.9 px 1.0 px 100.00 % 0.5 s 1 core @ 2.5 Ghz (Python + C/C++)
S. Tulyakov, A. Ivanov and F. Fleuret: Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching. Proceedings of the international conference on Neural Information Processing Systems (NIPS) 2018.
63 ESMNet 2.08 % 2.53 % 0.6 px 0.7 px 100.00 % 0.06 s GPU @ 2.5 Ghz (Python)
64 MC-CNN-acrt+GLR 2.09 % 3.22 % 0.6 px 0.7 px 100.00 % 0.5 s 1 core @ 2.5 Ghz (C/C++)
65 SCBNet 2.13 % 2.68 % 0.6 px 0.7 px 100.00 % 0.19 s 1 core @ 2.5 Ghz (C/C++)
66 LANet 2.13 % 2.57 % 0.6 px 0.7 px 100.00 % 0.08 s GPU @ 2.5 Ghz (Python)
67 CS2D 2.19 % 2.74 % 0.6 px 0.7 px 100.00 % 0.2 s GPU @ 2.5 Ghz (Python)
68 RecResNet code 2.21 % 2.94 % 0.6 px 0.7 px 100.00 % 0.3 s GPU @ NVIDIA TITAN X (Tensorflow)
K. Batsos and P. Mordohai: RecResNet: A Recurrent Residual CNN Architecture for Disparity Map Enhancement. In International Conference on 3D Vision (3DV) 2018.
69 L-ResMatch code 2.27 % 3.40 % 0.7 px 1.0 px 100.00 % 48 s Titan X (Torch7, CUDA)
A. Shaked and L. Wolf: Improved Stereo Matching with Constant Highway Networks and Reflective Loss. arXiv preprint arxiv:1701.00165 2016.
70 CNNF+SGM 2.28 % 3.48 % 0.7 px 0.9 px 100.00 % 71 s TESLA K40C
F. Zhang and B. Wah: Fundamental Principles on Learning New Features for Effective Dense Matching. IEEE Transactions on Image Processing 2018.
71 MBFnet code 2.28 % 3.27 % 0.7 px 0.8 px 100.00 % 0.05 s GPU @ GTX 2070 (Pytorch)
72 SGM-Net 2.29 % 3.50 % 0.7 px 0.9 px 100.00 % 67 s Titan X
A. Seki and M. Pollefeys: SGM-Nets: Semi-Global Matching With Neural Networks. CVPR 2017.
73 SsSMnet 2.30 % 3.00 % 0.7 px 0.8 px 100.00 % 0.8 s Titan Xp
Y. Zhong, Y. Dai and H. Li: Self-Supervised Learning for Stereo Matching with Self-Improving Ability. arXiv:1709.00930 2017.
74 CBMV-GCNet 2.33 % 3.41 % 0.8 px 1.0 px 100.00 % 3 s GTX 1080Ti @3.0 Ghz (Python + C/C++)
75 PBCP 2.36 % 3.45 % 0.7 px 0.9 px 100.00 % 68 s Nvidia GTX Titan X
A. Seki and M. Pollefeys: Patch Based Confidence Prediction for Dense Disparity Map. British Machine Vision Conference (BMVC) 2016.
76 Displets v2 code 2.37 % 3.09 % 0.7 px 0.8 px 100.00 % 265 s >8 cores @ 3.0 Ghz (Matlab + C/C++)
F. Guney and A. Geiger: Displets: Resolving Stereo Ambiguities using Object Knowledge. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
77 CRAR 2.40 % 3.16 % 0.7 px 0.8 px 100.00 % 0.04 s GTX1080Ti (Pytorch)
78 RTSnet code 2.43 % 2.90 % 0.7 px 0.7 px 100.00 % 0.02 s 1 core @ 2.5 Ghz (Python)
H. Lee and Y. Shin: Real-Time Stereo Matching Network with High Accuracy. 2019 IEEE International Conference on Image Processing (ICIP) 2019.
79 MC-CNN-acrt code 2.43 % 3.63 % 0.7 px 0.9 px 100.00 % 67 s Nvidia GTX Titan X (CUDA, Lua/Torch7)
J. Zbontar and Y. LeCun: Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches. Submitted to JMLR .
80 Dense-CNN 2.43 % 3.68 % 0.7 px 0.9 px 100.00 % 53 s 1 core @ 2.5 Ghz (C/C++)
81 cfusion
This method makes use of multiple (>2) views.
code 2.46 % 2.69 % 0.8 px 0.8 px 99.93 % 70 s GPU (Matlab + CUDA)
V. Ntouskos and F. Pirri: Confidence driven TGV fusion. arXiv preprint arXiv:1603.09302 2016.
82 Displets code 2.47 % 3.27 % 0.7 px 0.9 px 100.00 % 265 s >8 cores @ 3.0 Ghz (Matlab + C/C++)
F. Guney and A. Geiger: Displets: Resolving Stereo Ambiguities using Object Knowledge. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
83 MC-CNN 2.61 % 3.84 % 0.8 px 1.0 px 100.00 % 100 s Nvidia GTX Titan (CUDA, Lua/Torch7)
J. Zbontar and Y. LeCun: Computing the Stereo Matching Cost with a Convolutional Neural Network. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
84 Fast DS-CS 2.61 % 3.20 % 0.7 px 0.8 px 100.00 % 0.02 s GPU @ 2.0 Ghz (Python + C/C++)
85 PRSM
This method uses optical flow information.
This method makes use of multiple (>2) views.
code 2.78 % 3.00 % 0.7 px 0.7 px 100.00 % 300 s 1 core @ 2.5 Ghz (C/C++)
C. Vogel, K. Schindler and S. Roth: 3D Scene Flow Estimation with a Piecewise Rigid Scene Model. ijcv 2015.
86 SPS-StFl
This method uses optical flow information.
This method makes use of the epipolar geometry.
2.83 % 3.64 % 0.8 px 0.9 px 100.00 % 35 s 1 core @ 3.5 Ghz (C/C++)
K. Yamaguchi, D. McAllester and R. Urtasun: Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation. ECCV 2014.
87 MC-CNN-WS code 3.02 % 4.45 % 0.8 px 1.0 px 100.00 % 1.35 s 1 core 2.5 Ghz + K40 NVIDIA, Lua-Torch
S. Tulyakov, A. Ivanov and F. Fleuret: Weakly supervised learning of deep metrics for stereo reconstruction. ICCV 2017.
88 VC-SF
This method uses optical flow information.
This method makes use of multiple (>2) views.
3.05 % 3.31 % 0.8 px 0.8 px 100.00 % 300 s 1 core @ 2.5 Ghz (C/C++)
C. Vogel, S. Roth and K. Schindler: View-Consistent 3D Scene Flow Estimation over Multiple Frames. Proceedings of European Conference on Computer Vision. Lecture Notes in, Computer Science 2014.
89 Content-CNN 3.07 % 4.29 % 0.8 px 1.0 px 100.00 % 0.7 s Nvidia GTX Titan X (Torch)
W. Luo, A. Schwing and R. Urtasun: Efficient Deep Learning for Stereo Matching. CVPR 2016.
90 Deep Embed 3.10 % 4.24 % 0.9 px 1.1 px 100.00 % 3 s 1 core @ 2.5 Ghz (C/C++)
Z. Chen, X. Sun, Y. Yu, L. Wang and C. Huang: A Deep Visual Correspondence Embedding Model for Stereo Matching Costs. ICCV 2015.
91 JSOSM 3.15 % 3.94 % 0.8 px 0.9 px 100.00 % 105 s 8 cores @ 2.5 Ghz (C/C++)
X. Li and J. Liu: EFFICIENT STEREO MATCHING USING SEGMENT OPTIMIZATION. ICIP 2016.
92 FD-Fusion - CudaSGM 3.16 % 3.85 % 0.7 px 0.8 px 100.00 % 0.01 s GPU @ 2.5 Ghz (C/C++)
93 DFNet 3.23 % 4.12 % 0.7 px 0.8 px 100.00 % 0.1 s GPU @ 2.5 Ghz (C/C++)
94 OSF
This method uses optical flow information.
code 3.28 % 4.07 % 0.8 px 0.9 px 99.98 % 50 min 1 core @ 3.0 Ghz (Matlab + C/C++)
M. Menze and A. Geiger: Object Scene Flow for Autonomous Vehicles. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
95 CoR code 3.30 % 4.10 % 0.8 px 0.9 px 100.00 % 6 s 6 cores @ 3.3 Ghz (Matlab + C/C++)
A. Chakrabarti, Y. Xiong, S. Gortler and T. Zickler: Low-level Vision by Consensus in a Spatial Hierarchy of Regions. CVPR 2015.
96 TCD-CRF 3.32 % 5.24 % 0.9 px 1.9 px 100.00 % 60 s 4 cores @ 3.5 Ghz (C/C++)
S. Arjomand Bigdeli, G. Budweiser and M. Zwicker: Temporally Coherent Disparity Maps Using CRFs with Fast 4D Filtering. Proc. ACPR 2015.
97 SPS-St code 3.39 % 4.41 % 0.9 px 1.0 px 100.00 % 2 s 1 core @ 3.5 Ghz (C/C++)
K. Yamaguchi, D. McAllester and R. Urtasun: Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation. ECCV 2014.
98 PCBP-SS 3.40 % 4.72 % 0.8 px 1.0 px 100.00 % 5 min 4 cores @ 2.5 Ghz (Matlab + C/C++)
K. Yamaguchi, D. McAllester and R. Urtasun: Robust Monocular Epipolar Flow Estimation. CVPR 2013.
99 OGF 3.46 % 3.93 % 0.8 px 0.8 px 100.00 % 0.1 s GPU @ 2.5 Ghz (Python)
100 CBMV-PSMNet 3.52 % 4.26 % 0.9 px 1.0 px 100.00 % 2.5 s GPU @ 2.5 Ghz (Python)
101 CBMV code 3.56 % 4.73 % 0.9 px 1.1 px 100.00 % 250 s 6 cores@3.0Ghz(Python,C/C++,CUDA TitanX)
K. Batsos, C. Cai and P. Mordohai: CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation. 2018.
102 StereoBit 3.56 % 4.98 % 0.9 px 1.1 px 100.00 % 0.0016 s NVIDAI TITAN Xp(CUDA)
103 CPM2 code 3.58 % 4.41 % 0.9 px 1.1 px 99.99 % 1.8 s 1 core @ 2.5 Ghz (C/C++)
104 WDMC 3.72 % 5.00 % 0.9 px 1.2 px 100.00 % 1 min 8 cores @ 3.5 Ghz (Python)
105 DDS-SS 3.83 % 4.59 % 0.9 px 1.0 px 100.00 % 1 min 1 core @ 2.5 Ghz (Matlab + C/C++)
D. Wei, C. Liu and W. Freeman: A Data-driven Regularization Model for Stereo and Flow. 3DTV-Conference, 2014 International Conference on 2014.
106 StereoSLIC 3.92 % 5.11 % 0.9 px 1.0 px 99.89 % 2.3 s 1 core @ 3.0 Ghz (C/C++)
K. Yamaguchi, D. McAllester and R. Urtasun: Robust Monocular Epipolar Flow Estimation. CVPR 2013.
107 SMCM 3.94 % 5.24 % 0.9 px 1.1 px 100.00 % 1800 s Nvidia GTX 1080 (Caffe)
M. Yang, Y. Liu, Y. Cai and Z. You: Stereo matching based on classification of materials. Neurocomputing 2016.
108 PR-Sf+E
This method uses optical flow information.
4.02 % 4.87 % 0.9 px 1.0 px 100.00 % 200 s 4 cores @ 3.0 Ghz (Matlab + C/C++)
C. Vogel, K. Schindler and S. Roth: Piecewise Rigid Scene Flow. International Conference on Computer Vision (ICCV) 2013.
109 PCBP 4.04 % 5.37 % 0.9 px 1.1 px 100.00 % 5 min 4 cores @ 2.5 Ghz (Matlab + C/C++)
K. Yamaguchi, T. Hazan, D. McAllester and R. Urtasun: Continuous Markov Random Fields for Robust Stereo Estimation. ECCV 2012.
110 DispNetC code 4.11 % 4.65 % 0.9 px 1.0 px 100.00 % 0.06 s Nvidia GTX Titan X (Caffe)
N. Mayer, E. Ilg, P. Häusser, P. Fischer, D. Cremers, A. Dosovitskiy and T. Brox: A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. CVPR 2016.
111 CSPMS 4.13 % 5.92 % 1.2 px 1.6 px 100.00 % 6 s 4 cores @ 2.5 Ghz (C/C++)
J. Cho and M. Humenberger: Fast PatchMatch Stereo Matching Using Multi-Scale Cost Fusion for Automotive Applications. IV 2015.
112 SMV 4.23 % 5.77 % 1.1 px 1.3 px 100.00 % 1.6 min 8 cores @ 3.5 Ghz (Python)
113 SGM-post 4.27 % 5.33 % 1.0 px 1.1 px 100.00 % 5 s 4 cores @ 2.5 Ghz (C/C++)
Z. Zhong: Efficient Learning based Semi-Global Stereo Matching. 2015 submitted.
114 MBM 4.35 % 5.43 % 1.0 px 1.1 px 100.00 % 0.2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: A Multi-Block-Matching Approach for Stereo. IV 2015.
115 PR-Sceneflow
This method uses optical flow information.
4.36 % 5.22 % 0.9 px 1.1 px 100.00 % 150 sec 4 core @ 3.0 Ghz (Matlab - C/C++)
C. Vogel, K. Schindler and S. Roth: Piecewise Rigid Scene Flow. International Conference on Computer Vision (ICCV) 2013.
116 CoR-Conf code 4.49 % 5.26 % 1.0 px 1.2 px 96.37 % 6 s 6 cores @ 3.3 Ghz (Matlab + C/C++)
A. Chakrabarti, Y. Xiong, S. Gortler and T. Zickler: Low-level Vision by Consensus in a Spatial Hierarchy of Regions. CVPR 2015.
117 CVPR #2094
This method uses optical flow information.
4.58 % 5.11 % 1.0 px 1.1 px 100.00 % 0.05 s GPU @ 2.5 Ghz (Python)
118 DispSegNet 4.68 % 5.66 % 0.9 px 1.0 px 100.00 % 0.9 s GPU @ 2.5 Ghz (Python)
J. Zhang, K. Skinner, R. Vasudevan and M. Johnson-Roberson: DispSegNet: Leveraging Semantics for End- to-End Learning of Disparity Estimation From Stereo Imagery. IEEE Robotics and Automation Letters 2019.
119 pSGM 4.68 % 6.13 % 1.0 px 1.4 px 100.00 % 7.92 s 4 cores @ 3.5 Ghz (C/C++)
Y. Lee, M. Park, Y. Hwang, Y. Shin and C. Kyung: Memory-Efficient Parametric Semiglobal Matching. IEEE Signal Processing Letters 2018.
120 OASM-DDS 4.81 % 5.69 % 1.1 px 1.2 px 100.00 % 0.90 s 1 core @ 2.5 Ghz (Python)
121 AARBM 4.86 % 5.94 % 1.0 px 1.2 px 100.00 % 0.25 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
122 wSGM 4.97 % 6.18 % 1.3 px 1.6 px 97.03 % 6s 1 core @ 3.5 Ghz (C/C++)
R. Spangenberg, T. Langner and R. Rojas: Weighted Semi-Global Matching and Center-Symmetric Census Transform for Robust Driver Assistance. CAIP 2013.
123 AABM 4.97 % 6.04 % 1.0 px 1.2 px 100.00 % 0.12 s 1 core @ 3.1 Ghz (C/C++)
N. Einecke and J. Eggert: Stereo Image Warping for Improved Depth Estimation of Road Surfaces. IV 2013.
124 ATGV 5.02 % 6.88 % 1.0 px 1.6 px 100.00 % 6 min >8 cores @ 3.0 Ghz (Matlab + C/C++)
R. Ranftl, T. Pock and H. Bischof: Minimizing TGV-based Variational Models with Non-Convex Data terms. ICSSVM 2013.
125 rSGM code 5.03 % 6.60 % 1.1 px 1.5 px 97.22 % 0.2 s 4 cores @ 2.6 Ghz (C/C++)
R. Spangenberg, T. Langner, S. Adfeldt and R. Rojas: Large Scale Semi-Global Matching on the CPU. IV 2014.
126 iSGM 5.11 % 7.15 % 1.2 px 2.1 px 94.70 % 8 s 2 cores @ 2.5 Ghz (C/C++)
S. Hermann and R. Klette: Iterative Semi-Global Matching for Robust Driver Assistance Systems. ACCV 2012.
127 RBM 5.18 % 6.21 % 1.1 px 1.3 px 100.00 % 0.2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
128 ARW code 5.20 % 6.87 % 1.2 px 1.5 px 99.33 % 4.6s 1 core @ 3.5 Ghz (MATLAB+C/C++)
S. Lee, J. Lee, J. Lim and I. Suh: Robust Stereo Matching using Adaptive Random Walk with Restart Algorithm. Image and vision computing (accepted) 2015.
129 DLP 5.28 % 7.21 % 1.2 px 2.0 px 100.00 % 60 s 8 cores @ >3.5 Ghz (C/C++)
V. Nguyen, H. Nguyen and J. Jeon: Robust Stereo Data Cost With a Learning Strategy. IEEE Transactions on Intelligent Transportation Systems 2017.
130 UnOS(Full) 5.30 % 5.93 % 1.1 px 1.2 px 100.00 % 0.08 s GPU @ 2.5 Ghz (C/C++)
131 CDE Deep Learning 5.33 % 7.60 % 1.3 px 2.3 px 100.00 % 0.04 s 1 core @ 2.5 Ghz (C/C++)
132 CDE 5.33 % 7.60 % 1.3 px 2.3 px 100.00 % 40 s 1 core @ 2.5 Ghz (C/C++)
133 Ensemble 5.34 % 6.91 % 1.5 px 2.0 px 100.00 % 135 s 2 cores @ >3.5 Ghz (Matlab)
A. Spyropoulos and P. Mordohai: Ensemble Classifier for Combining Stereo Matching Algorithms. International Conference on 3D Vision (3DV) 2015.
134 ALTGV 5.36 % 6.49 % 1.1 px 1.2 px 100.00 % 20 s GPU @ 2.5 Ghz (C/C++)
G. Kuschk and D. Cremers: Fast and Accurate Large-scale Stereo Reconstruction using Variational Methods. ICCV Workshop on Big Data in 3D Computer Vision 2013.
135 RADE 5.39 % 7.65 % 1.2 px 2.1 px 100.00 % 0.04 s 8 cores @ 2.5 Ghz (C/C++)
136 SNCC 5.40 % 6.44 % 1.2 px 1.3 px 100.00 % 0.11 s 1 core @ 3.1 Ghz (C/C++)
N. Einecke and J. Eggert: A Two-Stage Correlation Method for Stereoscopic Depth Estimation. DICTA 2010.
137 CAT 5.45 % 6.54 % 1.1 px 1.2 px 100.00 % 10 s 1 core @ 3.5 Ghz (C/C++)
J. Ha, J. Jeon, G. Bae, S. Jo and H. Jeong: Cost Aggregation Table: Cost Aggregation Method Using Summed Area Table Scheme for Dense Stereo Correspondence. Advances in Visual Computing 2014.
138 SGM 5.76 % 7.00 % 1.2 px 1.3 px 85.80 % 3.7 s 1 core @ 3.0 Ghz (C/C++)
H. Hirschmueller: Stereo Processing by Semi-Global Matching and Mutual Information. PAMI 2008.
139 SGMGPU 5.85 % 8.10 % 1.3 px 2.2 px 99.80 % 0.01 s GPU @ >3.5 Ghz (C/C++)
140 mSGM-LDE 6.01 % 8.22 % 1.4 px 2.4 px 100.00 % 55 s 2 cores @ 2.5 Ghz (C/C++)
V. Nguyen, D. Nguyen, S. Lee and J. Jeon: Local Density Encoding for Robust Stereo Matching. TCSVT 2014.
141 Toast2
This method uses stereo information.
6.16 % 7.42 % 1.2 px 1.4 px 95.39 % 0.03 s 4 cores @ 3.5 Ghz (C/C++)
B. Ranft and T. Strau\ss: Modeling Arbitrarily Oriented Slanted Planes for Efficient Stereo Vision based on Block Matching. Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on 2014.
142 ITGV 6.20 % 7.30 % 1.3 px 1.5 px 100.00 % 7 s 1 core @ 3.0 Ghz (Matlab + C/C++)
R. Ranftl, S. Gehrig, T. Pock and H. Bischof: Pushing the Limits of Stereo Using Variational Stereo Estimation. IV 2012.
143 OASM-Net 6.39 % 8.60 % 1.3 px 2.0 px 100.00 % 0.73 s GPU @ 2.5 Ghz (Python)
A. Li and Z. Yuan: Occlusion Aware Stereo Matching via Cooperative Unsupervised Learning. Proceedings of the Asian Conference on Computer Vision, ACCV 2018.
144 nwpuzlnet 7.08 % 8.39 % 1.5 px 1.8 px 100.00 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
145 OCV-SGBM code 7.64 % 9.13 % 1.8 px 2.0 px 86.50 % 1.1 s 1 core @ 2.5 Ghz (C/C++)
H. Hirschmueller: Stereo processing by semiglobal matching and mutual information. PAMI 2008.
146 SSMW 7.83 % 8.95 % 1.6 px 1.8 px 99.99 % 2.5 min 8 cores @ 2.5 Ghz (C/C++)
X. Li, J. Liu, G. Chen and H. Fu: Efficient Methods Using Slanted Support Windows for Slanted Surfaces. IET Computer Vision, http://ietdl.org/t/5QsTxb 2016.
147 GSS 8.00 % 9.15 % 1.6 px 1.8 px 100.00 % 10 min 1 core @ 2.5 Ghz (C/C++)
148 MSMW
This method uses stereo information.
code 8.01 % 9.24 % 1.6 px 1.7 px 72.39 % 3 min 4 cores @ 2.5 Ghz (C/C++)
A. Buades and G. Facciolo: On the performance of local methods for stereovision. 2013 submitted.
149 HSMA 8.15 % 10.33 % 1.9 px 2.9 px 100.00 % 44s 1 core @ 3.0 Ghz (C/C++)
O. Zeglazi, M. Rziza, A. Amine and C. Demonceaux: A hierarchical stereo matching algorithm based on adaptive support region aggregation method. Pattern Recognition Letters 2018.
150 ELAS code 8.24 % 9.96 % 1.4 px 1.6 px 94.55 % 0.3 s 1 core @ 2.5 Ghz (C/C++)
A. Geiger, M. Roser and R. Urtasun: Efficient Large-Scale Stereo Matching. ACCV 2010.
151 linBP 8.56 % 10.70 % 1.7 px 2.7 px 99.89 % 1.6 min 1 core @ 3.0 Ghz (C/C++)
W. Khan, V. Suaste, D. Caudillo and R. Klette: Belief Propagation Stereo Matching Compared to iSGM on Binocular or Trinocular Video Data. IV 2013.
152 ADSM 8.71 % 10.05 % 2.1 px 2.7 px 100.00 % 125 s 1 core @ 2.0 Ghz (C/C++)
O. Zeglazi, M. Rziza, A. Amine and C. Demonceaux: Accurate dense stereo matching for road scenes. 2017 IEEE International Conference on Image Processing, ICIP 2017, Beijing, China, September 17-20, 2017 .
153 DispCC 8.79 % 9.85 % 1.7 px 1.8 px 100.00 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
154 Deep-Raw 8.93 % 11.07 % 3.9 px 4.9 px 100.00 % 1 s 1 core @ 2.5 Ghz (C/C++)
Z. Chen, X. Sun, Y. Yu, L. Wang and C. Huang: A Deep Visual Correspondence Embedding Model for Stereo Matching Costs. ICCV 2015.
155 S+GF (Cen) code 9.03 % 11.21 % 2.1 px 3.4 px 100.00 % 140 s 1 core @ 3.0 Ghz (C/C++)
K. Zhang, Y. Fang, D. Min, L. Sun, S. Yang, S. Yan and Q. Tian: Cross-Scale Cost Aggregation for Stereo Matching. CVPR 2014.
156 CrossCensus 9.46 % 10.86 % 2.3 px 2.7 px 100.00 % 30 s 1 core @ 2.5 Ghz (C/C++)
K. Zhang, J. Lu and G. Lafruit: Cross-Based Local Stereo Matching Using Orthogonal Integral Images. Circuits and Systems for Video Technology, IEEE Transactions on 2009.
157 SymST-GP 9.79 % 11.66 % 2.5 px 3.3 px 100.00 % 0.254 s Dual - Nvidia GTX Titan (CUDA)
R. Ralha, G. Falcao, J. Amaro, V. Mota, M. Antunes, J. Barreto and U. Nunes: Parallel refinement of slanted 3D reconstruction using dense stereo induced from symmetry. Journal of Real-Time Image Processing 2016.
158 SM_GPTM 9.79 % 11.38 % 2.1 px 2.6 px 100.00 % 6.5 s 2 cores @ 2.5 Ghz (C/C++)
C. Cigla and A. Alatan: An Improved Stereo Matching Algorithm with Ground Plane and Temporal Smoothness Constraints. ECCV Workshops 2012.
159 LAMC-DSΜ 9.82 % 11.49 % 2.1 px 2.7 px 99.96 % 10.8 min 2 cores @ 2.5 Ghz (Matlab)
C. Stentoumis, L. Grammatikopoulos, I. Kalisperakis, E. Petsa and G. Karras: A local adaptive approach for dense stereo matching in architectural scene reconstruction. ISPRS 2013.
160 IIW 10.78 % 12.62 % 3.3 px 4.3 px 70.85 % 5.5 s 1 core @ 2.5 Ghz (C/C++)
A. Murarka and N. Einecke: A meta-technique for increasing density of local stereo methods through iterative interpolation and warping. Canadian Conference on Computer and Robot Vision 2014.
161 SDM code 10.95 % 12.14 % 2.0 px 2.3 px 63.58 % 1 min 1 core @ 2.5 Ghz (C/C++)
J. Kostkova: Stratified dense matching for stereopsis in complex scenes. BMVC 2003.
162 HLSC_mesh 11.22 % 12.82 % 2.3 px 2.9 px 100.00 % 800 s 1 core @ 2.5 Ghz (Matlab + C/C++)
S. Hadfield, K. Lebeda and R. Bowden: Stereo reconstruction using top-down cues. Computer Vision and Image Understanding 2016.
163 GF (Census) code 11.65 % 13.76 % 4.5 px 5.6 px 100.00 % 120 s 1 core @ 3.0 Ghz (C/C++)
A. Hosni, C. Rhemann, M. Bleyer, C. Rother and M. Gelautz: Fast Cost-Volume Filtering for Visual Correspondence and Beyond. TPAMI 2013.
K. Zhang, Y. Fang, D. Min, L. Sun, S. Yang, S. Yan and Q. Tian: Cross-Scale Cost Aggregation for Stereo Matching. CVPR 2014.
164 BSM code 11.74 % 13.44 % 2.2 px 2.8 px 97.02 % 2.5 min 1 core @ 3.0 Ghz (C/C++)
K. Zhang, J. Li, Y. Li, W. Hu, L. Sun and S. Yang: Binary stereo matching. Pattern Recognition (ICPR), 2012 21st International Conference on 2012.
165 GCSF
This method uses optical flow information.
code 12.05 % 13.24 % 1.9 px 2.1 px 60.77 % 2.4 s 1 core @ 2.5 Ghz (C/C++)
J. Cech, J. Sanchez-Riera and R. Horaud: Scene Flow Estimation by Growing Correspondence Seeds. CVPR 2011.
166 OCV-BM-post code 12.28 % 13.76 % 2.1 px 2.3 px 47.11 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
G. Bradski: The OpenCV Library. Dr. Dobb's Journal of Software Tools 2000.
167 GCS code 13.38 % 14.54 % 2.1 px 2.3 px 51.06 % 2.2 s 1 core @ 2.5 Ghz (C/C++)
J. Cech and R. Sara: Efficient Sampling of Disparity Space for Fast And Accurate Matching. BenCOS 2007.
168 GLDS code 17.22 % 18.63 % 2.8 px 3.2 px 100.00 % 26 s GPU @ 1.5 Ghz (C/C++)
K. Oguri and Y. Shibata: A new stereo formulation not using pixel and disparity models. 2018.
169 CostFilter code 19.99 % 21.08 % 5.0 px 5.4 px 100.00 % 4 min 1 core @ 2.5 Ghz (Matlab)
C. Rhemann, A. Hosni, M. Bleyer, C. Rother and M. Gelautz: Fast Cost-Volume Filtering for Visual Correspondence and Beyond. CVPR 2011.
170 GC+occ code 33.49 % 34.73 % 8.6 px 9.2 px 87.57 % 6 min 1 core @ 2.5 Ghz (C/C++)
V. Kolmogorov and R. Zabih: Computing Visual Correspondence with Occlusions using Graph Cuts. ICCV 2001.
171 VariableCros 34.84 % 36.11 % 12.4 px 12.9 px 95.66 % 30 s 1 core @ 2.5 Ghz (Matlab)
K. Zhang, J. Lu and G. Lafruit: Cross-Based Local Stereo Matching Using Orthogonal Integral Images. Circuits and Systems for Video Technology, IEEE Transactions on 2009.
172 ALE-Stereo code 50.48 % 51.19 % 13.0 px 13.5 px 100.00 % 50 min 1 core @ 3.0 Ghz (C/C++)
L. Ladicky, P. Sturgess, C. Russell, S. Sengupta, Y. Bastanlar, W. Clocksin and P. Torr: Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction. BMVC 2010.
173 MEDIAN 52.61 % 53.67 % 7.7 px 8.2 px 99.95 % 0.01 s 1 core @ 2.5 Ghz (C/C++)
174 AVERAGE 61.62 % 62.49 % 8.0 px 8.6 px 99.95 % 0.01 s 1 core @ 2.5 Ghz (C/C++)
This table as LaTeX


Related Datasets

  • HCI/Bosch Robust Vision Challenge: Optical flow and stereo vision challenge on high resolution imagery recorded at a high frame rate under diverse weather conditions (e.g., sunny, cloudy, rainy). The Robert Bosch AG provides a prize for the best performing method.
  • Image Sequence Analysis Test Site (EISATS): Synthetic image sequences with ground truth information provided by UoA and Daimler AG. Some of the images come with 3D range sensor information.
  • Middlebury Stereo Evaluation: The classic stereo evaluation benchmark, featuring four test images in version 2 of the benchmark, with very accurate ground truth from a structured light system. 38 image pairs are provided in total.
  • Daimler Stereo Dataset: Stereo bad weather highway scenes with partial ground truth for freespace
  • Make3D Range Image Data: Images with small-resolution ground truth used to learn and evaluate depth from single monocular images.
  • Lubor Ladicky's Stereo Dataset: Stereo Images with manually labeled ground truth based on polygonal areas.

Citation

When using this dataset in your research, we will be happy if you cite us:
@INPROCEEDINGS{Geiger2012CVPR,
  author = {Andreas Geiger and Philip Lenz and Raquel Urtasun},
  title = {Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2012}
}



eXTReMe Tracker