The KITTI Vision Benchmark Suite

The stereo 2015 / flow 2015 / scene flow 2015 benchmark consists of 200 training scenes and 200 test scenes (4 color images per scene, saved in loss less png format). Compared to the stereo 2012 and flow 2012 benchmarks, it comprises dynamic scenes for which the ground truth has been established in a semi-automatic process. Our evaluation server computes the percentage of bad pixels averaged over all ground truth pixels of all 200 test images. For this benchmark, we consider a pixel to be correctly estimated if the disparity or flow end-point error is <3px or <5% (for scene flow this criterion needs to be fulfilled for both disparity maps and the flow map). We require that all methods use the same parameter set for all test pairs. Our development kit provides details about the data format as well as MATLAB / C++ utility functions for reading and writing disparity maps and flow fields. More details can be found in Object Scene Flow for Autonomous Vehicles (CVPR 2015).

• Download stereo 2015/flow 2015/scene flow 2015 data set (2 GB)
• Download calibration files (1 MB)
• Download multi-view extension (20 frames per scene) (14 GB)
• Download development kit (3 MB)

Our evaluation table ranks all methods according to the number of erroneous pixels. All methods providing less than 100 % density have been interpolated using simple background interpolation as explained in the corresponding header file in the development kit. Legend:

• D1: Percentage of stereo disparity outliers in first frame
• D2: Percentage of stereo disparity outliers in second frame
• Fl: Percentage of optical flow outliers
• SF: Percentage of scene flow outliers (=outliers in either D0, D1 or Fl)
• bg: Percentage of outliers averaged only over background regions
• fg: Percentage of outliers averaged only over foreground regions
• all: Percentage of outliers averaged over all ground truth pixels

	Method	Setting	Code	D1-bg	D1-fg	D1-all	Density	Runtime	Environment
1	MS_CSPN			1.56 %	3.78 %	1.93 %	100.00 %	0.5 s	GPU @ 2.5 Ghz (C/C++)
2	Samsung_System_LSI			1.60 %	3.88 %	1.98 %	100.00 %	0.4 s	GPU @ 2.5 Ghz (Python)
3	DSHNet			1.65 %	4.29 %	2.09 %	100.00 %	0.7 s	Nvidia GTX Titan Xp
4	open-depth			1.76 %	3.84 %	2.10 %	100.00 %	0.51 s	NVIDIA TITAN Xp (PyTorch 0.4.0)
5	DM-Net-Pretrained-30			1.74 %	4.22 %	2.15 %	100.00 %	0.40s	Titan XP
6	DN-FN-CSS-I			1.92 %	3.32 %	2.16 %	100.00 %	0.25 s	1 core @ 2.5 Ghz (C/C++)
7	stereo-fusion-SJTU			1.87 %	3.61 %	2.16 %	100.00 %	0.7 s	Nvidia GTX Titan Xp
X. Song, X. Zhao, H. Hu and L. Fang: EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo Matching. arXiv preprint arXiv:1803.05196 2018.
8	TinyStereo_V2			1.93 %	3.76 %	2.24 %	100.00 %	0.4 s	GPU @ 2.5 Ghz (Python)
9	SegStereo			1.88 %	4.07 %	2.25 %	100.00 %	0.6 s	Nvidia GTX Titan Xp
G. Yang, H. Zhao, J. Shi, Z. Deng and J. Jia: SegStereo: Exploiting Semantic Information for Disparity Estimation. arXiv preprint arXiv:1807.11699 2018.
10	HDU-LJJ-Group			1.82 %	4.42 %	2.25 %	100.00 %	0.47 s	GPU @ 1.5 Ghz (Python)
11	PASM			1.78 %	4.64 %	2.26 %	100.00 %	0.52 s	1 core @ 2.5 Ghz (C/C++)
12	MPSMNet			1.78 %	4.63 %	2.26 %	100.00 %	1.0 s	GPU @ 2.5 Ghz (Python)
13	TinyStereo			1.92 %	4.13 %	2.28 %	100.00 %	0.39 s	1 core @ 2.5 Ghz (C/C++)
14	PSMNet_ROB			1.79 %	4.92 %	2.31 %	100.00 %	0.41 s	1 core @ 2.5 Ghz (Python)
15	MeituNet			1.88 %	4.48 %	2.31 %	100.00 %	0.51 s	GPU @ 2.5 Ghz (Python)
16	PSMNet		code	1.86 %	4.62 %	2.32 %	100.00 %	0.41 s	Nvidia GTX Titan Xp
J. Chang and Y. Chen: Pyramid Stereo Matching Network. arXiv preprint arXiv:1803.08669 2018.
17	DenseASPP			1.88 %	4.53 %	2.32 %	100.00 %	0.5 s	GPU @ 2.5 Ghz (Python)
18	MSDC-Net			2.00 %	4.05 %	2.34 %	100.00 %	0.8 s	1 core @ 2.5 Ghz (C/C++)
19	SE-PSM			1.90 %	4.59 %	2.34 %	100.00 %	0.85 s	GPU @ 3.0 Ghz (Python)
20	disparity stereo			1.85 %	4.86 %	2.35 %	100.00 %	0.5 s	GPU @ 1.5 Ghz (Python)
21	Kandao			2.14 %	3.45 %	2.36 %	100.00 %	0.22 s	Nvidia GTX 1080 Ti
22	HiSignal_SM			2.10 %	3.64 %	2.36 %	100.00 %	0.25 s	Nvidia titan x (Python)
23	DeepStereo_V2			2.00 %	4.21 %	2.37 %	100.00 %	0.4 s	1 core @ 2.5 Ghz (C/C++)
24	UMS-GANs			1.95 %	4.57 %	2.38 %	100.00 %	0.7 s	GPU @ 2.5 Ghz (Python)
25	L2-method			1.91 %	4.90 %	2.40 %	100.00 %	0.35 s	GPU @ 2.5 Ghz (Python + C/C++)
26	PSM+NN			1.95 %	4.85 %	2.43 %	100.00 %	1 s	GPU @ 2.5 Ghz (Python + C/C++)
27	iResNet-i2		code	2.25 %	3.40 %	2.44 %	100.00 %	0.12 s	Nvidia Titan X (Pascal)
Z. Liang, Y. Feng, Y. Guo and H. Liu: Learning for Disparity Estimation through Feature Constancy. arXiv preprint arXiv:1712.01039 2017.
28	iResNet			2.35 %	3.23 %	2.50 %	100.00 %	0.1 s	Nvidia Titan X (Pascal)
29	NNet			1.95 %	5.32 %	2.51 %	100.00 %	0.69 s	GPU @ 2.5 Ghz (Python + C/C++)
30	PSMNet-test		code	2.02 %	5.02 %	2.52 %	100.00 %	2 s	GPU @ 2.5 Ghz (Python)
J. Chang and Y. Chen: Pyramid Stereo Matching Network. arXiv preprint arXiv:1803.08669 2018.
31	X_ASPP			2.13 %	4.57 %	2.54 %	100.00 %	0.88 s	GPU @ 2.5 Ghz (Python)
32	FBW-Net			2.08 %	4.98 %	2.56 %	100.00 %	2 s	GPU @ 2.5 Ghz (Python)
33	PDSNet		code	2.29 %	4.05 %	2.58 %	100.00 %	0.5 s	1 core @ 2.5 Ghz (Python)
34	DeepStereo			2.16 %	4.72 %	2.59 %	100.00 %	0.9 s	Titian X
35	EdgeStereo			2.27 %	4.18 %	2.59 %	100.00 %	0.27 s	Nvidia GTX Titan Xp
X. Song, X. Zhao, H. Hu and L. Fang: EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo Matching. arXiv preprint arXiv:1803.05196 2018.
36	SCV			2.22 %	4.53 %	2.61 %	100.00 %	0.36 s	Nvidia GTX 1080 Ti
37	RESC			2.44 %	3.63 %	2.64 %	100.00 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
38	SANet			2.13 %	5.24 %	2.65 %	100.00 %	0.8 s	1 core @ 2.5 Ghz (Python)
39	CRL		code	2.48 %	3.59 %	2.67 %	100.00 %	0.47 s	Nvidia GTX 1080
J. Pang, W. Sun, J. Ren, C. Yang and Q. Yan: Cascade residual learning: A two-stage convolutional neural network for stereo matching. ICCV Workshop on Geometry Meets Deep Learning 2017.
40	CCFP-Net			2.11 %	5.53 %	2.68 %	100.00 %	0.5 s	8 cores @ 2.5 Ghz (Python)
41	NCCL2			2.11 %	5.59 %	2.69 %	100.00 %	0.61 s	GPU @ 2.5 Ghz (Python + C/C++)
42	MFS-NET			2.22 %	5.09 %	2.70 %	100.00 %	0.5 s	GPU @ 2.5 Ghz (Python)
43	iResNet_ROB			2.27 %	4.89 %	2.71 %	100.00 %	0.35 s	Nvidia Titan Xp
44	LALA_ROB			2.22 %	5.30 %	2.73 %	100.00 %	0.5 s	1 core @ 2.5 Ghz (Python)
45	ETE_ROB			2.17 %	5.54 %	2.73 %	100.00 %	0.4 s	GPU @ 2.5 Ghz (Python)
46	NaN_ROB			2.30 %	5.09 %	2.77 %	100.00 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
47	DeepStereo			2.15 %	5.88 %	2.77 %	100.00 %	0.9 s	Titian X
48	DLCB_ROB			2.16 %	5.85 %	2.77 %	100.00 %	140 s	GPU @ >3.5 Ghz (Python)
49	3DResStereo			2.27 %	5.50 %	2.80 %	100.00 %	1.3 s	Titian X
50	ResV2_ASPP			2.32 %	5.47 %	2.84 %	100.00 %	0.56 s	GPU @ 2.5 Ghz (Python)
51	GC-NET			2.21 %	6.16 %	2.87 %	100.00 %	0.9 s	Nvidia GTX Titan X
A. Kendall, H. Martirosyan, S. Dasgupta, P. Henry, R. Kennedy, A. Bachrach and A. Bry: End-to-End Learning of Geometry and Context for Deep Stereo Regression. Proceedings of the International Conference on Computer Vision (ICCV) 2017.
52	DN-CSS_ROB			2.39 %	5.71 %	2.94 %	100.00 %	0.07 s	1 core @ 2.5 Ghz (C/C++)
53	FBW_ROB			2.35 %	6.20 %	2.99 %	100.00 %	2 s	GPU @ 2.5 Ghz (Python)
54	XPNet_ROB			2.41 %	6.06 %	3.01 %	100.00 %	0.42 s	GPU @ 2.5 Ghz (Python)
55	LRCR			2.55 %	5.42 %	3.03 %	100.00 %	49.2 s	Nvidia GTX Titan X
Z. Jie, P. Wang, Y. Ling, B. Zhao, Y. Wei, J. Feng and W. Liu: Left-Right Comparative Recurrent Model for Stereo Matching. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018.
56	CFCNet			2.47 %	5.90 %	3.04 %	100.00 %	0.47 s	GPU @ 3.0 Ghz (Python)
57	RecResNet			2.46 %	6.30 %	3.10 %	100.00 %	.1 s	@ (Nvidia GTX Titan X)
58	NVStereoNet		code	2.62 %	5.69 %	3.13 %	100.00 %	0.6 s	NVIDIA Titan Xp
N. Smolyanskiy, A. Kamenev and S. Birchfield: On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach. arXiv preprint arXiv:1803.09719 2018.
59	NVStereoNet_ROB			2.62 %	5.69 %	3.13 %	100.00 %	0.6 s	NVIDIA Titan Xp
60	MSFNet			2.70 %	5.36 %	3.15 %	100.00 %	0.15 s	Nvidia GTX 1080 Ti
61	DRR			2.58 %	6.04 %	3.16 %	100.00 %	0.4 s	Nvidia GTX Titan X
S. Gidaris and N. Komodakis: Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling. arXiv preprint arXiv:1612.04770 2016.
62	MS-GANs			2.53 %	6.64 %	3.21 %	100.00 %	0.9 s	Nvidia GTX Titan X
63	E2ES-Net			2.61 %	6.43 %	3.25 %	100.00 %	0.5 s	Nvidia GTX 1080 (Python)
64	SSMFCR			3.16 %	4.11 %	3.32 %	100.00 %	0.09 s	1 core @ 2.5 Ghz (C/C++)
65	gcn			2.62 %	6.85 %	3.32 %	100.00 %	0.9 s	1 core @ 2.5 Ghz (C/C++)
66	SsSMnet			2.70 %	6.92 %	3.40 %	100.00 %	0.8 s	P100
Y. Zhong, Y. Dai and H. Li: Self-Supervised Learning for Stereo Matching with Self-Improving Ability. arXiv:1709.00930 2017.
67	L-ResMatch		code	2.72 %	6.95 %	3.42 %	100.00 %	48 s	1 core @ 2.5 Ghz (C/C++)
A. Shaked and L. Wolf: Improved Stereo Matching with Constant Highway Networks and Reflective Loss. arXiv preprint arxiv:1701.00165 2016.
68	Displets v2		code	3.00 %	5.56 %	3.43 %	100.00 %	265 s	>8 cores @ 3.0 Ghz (Matlab + C/C++)
F. Guney and A. Geiger: Displets: Resolving Stereo Ambiguities using Object Knowledge. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
69	D3DNet			2.88 %	6.60 %	3.50 %	100.00 %	0.35 s	Nvidia GTX Titan X
70	VGG-Disp			3.07 %	5.89 %	3.54 %	100.00 %	0.31 s	GPU @ 2.5 Ghz (C/C++)
71	CNNF+SGM			2.78 %	7.69 %	3.60 %	100.00 %	71 s	TESLA K40C
F. Zhang and B. Wah: Fundamental Principles on Learning New Features for Effective Dense Matching. IEEE Transactions on Image Processing 2018.
72	PBCP			2.58 %	8.74 %	3.61 %	100.00 %	68 s	Nvidia GTX Titan X
A. Seki and M. Pollefeys: Patch Based Confidence Prediction for Dense Disparity Map. British Machine Vision Conference (BMVC) 2016.
73	SGM-Net			2.66 %	8.64 %	3.66 %	100.00 %	67 s	Titan X
A. Seki and M. Pollefeys: SGM-Nets: Semi-Global Matching With Neural Networks. CVPR 2017.
74	MRDF-CNN			3.55 %	4.81 %	3.76 %	100.00 %	0.62 s	Nvidia GTX Titan X
75	GPNET			3.91 %	3.67 %	3.87 %	100.00 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
76	MC-CNN-acrt		code	2.89 %	8.88 %	3.89 %	100.00 %	67 s	Nvidia GTX Titan X (CUDA, Lua/Torch7)
J. Zbontar and Y. LeCun: Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches. Submitted to JMLR .
77	ASTCC			2.94 %	8.95 %	3.94 %	100.00 %	130 s	GPU @ 2.5 Ghz (Python + C/C++)
78	MSMD_ROB			3.14 %	9.28 %	4.16 %	100.00 %	1.2 s	GPU @ 2.5 Ghz (Python)
79	RGL			4.22 %	4.02 %	4.19 %	100.00 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
80	PRSM		code	3.02 %	10.52 %	4.27 %	99.99 %	300 s	1 core @ 2.5 Ghz (C/C++)
C. Vogel, K. Schindler and S. Roth: 3D Scene Flow Estimation with a Piecewise Rigid Scene Model. ijcv 2015.
81	PDISCO_ROB			3.79 %	7.00 %	4.33 %	100.00 %	0.2 s	Nvidia GTX Titan X (Python, TensorFlow)
82	PSPO			3.01 %	11.00 %	4.34 %	100.00 %	5 min	1 core @ 2.5 Ghz (Matlab + C/C++)
83	DispNetC		code	4.32 %	4.41 %	4.34 %	100.00 %	0.06 s	Nvidia GTX Titan X (Caffe)
N. Mayer, E. Ilg, P. Häusser, P. Fischer, D. Cremers, A. Dosovitskiy and T. Brox: A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation. CVPR 2016.
84	SGM-Forest			3.11 %	10.74 %	4.38 %	99.92 %	6 seconds	1 core @ 3.0 Ghz (Python/C/C++)
J. Schönberger, S. Sinha and M. Pollefeys: Learning to Fuse Proposals from Multiple Scanline Optimizations in Semi-Global Matching. European Conference on Computer Vision (ECCV) 2018.
85	SSF			3.55 %	8.75 %	4.42 %	100.00 %	5 min	1 core @ 2.5 Ghz (Matlab + C/C++)
Z. Ren, D. Sun, J. Kautz and E. Sudderth: Cascaded Scene Flow Prediction using Semantic Segmentation. International Conference on 3D Vision (3DV) 2017.
86	ISF			4.12 %	6.17 %	4.46 %	100.00 %	10 min	1 core @ 3 Ghz (C/C++)
A. Behl, O. Jafari, S. Mustikovela, H. Alhaija, C. Rother and A. Geiger: Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios?. International Conference on Computer Vision (ICCV) 2017.
87	MSFG-Net			3.62 %	8.90 %	4.50 %	100.00 %	0.6 s	1 core @ 2.5 Ghz (Python)
88	Content-CNN			3.73 %	8.58 %	4.54 %	100.00 %	1 s	Nvidia GTX Titan X (Torch)
W. Luo, A. Schwing and R. Urtasun: Efficient Deep Learning for Stereo Matching. CVPR 2016.
89	MC_TN			3.50 %	10.04 %	4.59 %	100.00 %	1 s	GPU @ 2.5 Ghz (C/C++)
90	FCVF-net			3.96 %	7.74 %	4.59 %	100.00 %	0.5 s	GPU @ 1.5 Ghz (Python)
91	MSFG-Net			3.81 %	9.62 %	4.77 %	100.00 %	0.6 s	GPU @ 2.5 Ghz (C/C++)
92	StereoNet			4.30 %	7.45 %	4.83 %	100.00 %	0.02 s	NVidia Titan Xp (C/C++)
93	MC-CNN-WS		code	3.78 %	10.93 %	4.97 %	100.00 %	1.35 s	1 core 2.5 Ghz + K40 NVIDIA, Lua-Torch
S. Tulyakov, A. Ivanov and F. Fleuret: Weakly supervised learning of deep metrics for stereo reconstruction. ICCV 2017.
94	3DMST			3.36 %	13.03 %	4.97 %	100.00 %	93 s	1 core @ >3.5 Ghz (C/C++)
X. Lincheng Li and L. Zhang: 3D Cost Aggregation with Multiple Minimum Spanning Trees for Stereo Matching. submitted to Applied Optics .
95	CBMV_ROB			3.55 %	12.09 %	4.97 %	100.00 %	300 s	>8 cores @ 3.0 Ghz (Python + C/C++)
96	NN			4.23 %	8.90 %	5.01 %	100.00 %	0.9S	GPU @ 2.5 Ghz
97	EOSF			4.12 %	9.49 %	5.01 %	99.96 %	10 min	1 cores @ 3.5 Ghz (Matlab + C/C++)
98	OSF+TC			4.11 %	9.64 %	5.03 %	100.00 %	50 min	1 core @ 2.5 Ghz (C/C++)
M. Neoral and J. Šochman: Object Scene Flow with Temporal Consistency. 22nd Computer Vision Winter Workshop (CVWW) 2017.
99	SOSF			4.30 %	8.72 %	5.03 %	100.00 %	55 min	1 core @ 2.5 Ghz (Matlab + C/C++)
100	CBMV		code	4.17 %	9.53 %	5.06 %	100.00 %	250 s	6 cores @ 3.0 Ghz (Python,C/C++,CUDA Nvidia TitanX)
K. Batsos, C. Cai and P. Mordohai: CBMV: A Coalesced Bidirectional Matching Volume for Disparity Estimation. 2018.
101	SS-SF			3.59 %	13.11 %	5.18 %	100.00 %	3 min	1 core @ 2.5 Ghz (Matlab + C/C++)
102	OSF 2018		code	4.11 %	11.12 %	5.28 %	100.00 %	390 s	1 core @ 2.5 Ghz (Matlab + C/C++)
M. Menze, C. Heipke and A. Geiger: Object Scene Flow. ISPRS Journal of Photogrammetry and Remote Sensing (JPRS) 2018.
103	SPS-St		code	3.84 %	12.67 %	5.31 %	100.00 %	2 s	1 core @ 3.5 Ghz (C/C++)
K. Yamaguchi, D. McAllester and R. Urtasun: Efficient Joint Segmentation, Occlusion Labeling, Stereo and Flow Estimation. ECCV 2014.
104	MNP			3.92 %	12.37 %	5.33 %	100.00 %	3 min	>8 cores @ 2.5 Ghz (C/C++)
105	MDP			4.19 %	11.25 %	5.36 %	100.00 %	11.4 s	4 cores @ 3.5 Ghz (Matlab + C/C++)
A. Li, D. Chen, Y. Liu and Z. Yuan: Coordinating Multiple Disparity Proposals for Stereo Computation. IEEE Conference on Computer Vision and Pattern Recognition 2016.
106	PWCDC_ROB			5.11 %	7.03 %	5.43 %	100.00 %	0.02 s	1 core @ 2.5 Ghz (Python)
107	CPM2		code	4.13 %	12.03 %	5.44 %	99.95 %	3 s	1 core @ 3.5 Ghz (C/C++)
108	DC-NET			4.31 %	11.52 %	5.51 %	100.00 %	0.53 s	>8 cores @ 3.5 Ghz (C/C++)
109	RIMM-SF			4.31 %	12.23 %	5.63 %	100.00 %	150 s	4 cores @ 3.5 Ghz (C/C++)
110	OSF		code	4.54 %	12.03 %	5.79 %	100.00 %	50 min	1 core @ 2.5 Ghz (C/C++)
M. Menze and A. Geiger: Object Scene Flow for Autonomous Vehicles. Conference on Computer Vision and Pattern Recognition (CVPR) 2015.
111	pSGM			4.84 %	11.64 %	5.97 %	100.00 %	7.77 s	4 cores @ 3.5 Ghz (C/C++)
Y. Lee, M. Park, Y. Hwang, Y. Shin and C. Kyung: Memory-Efficient Parametric Semiglobal Matching. IEEE Signal Processing Letters 2018.
112	CSF			4.57 %	13.04 %	5.98 %	99.99 %	80 s	1 core @ 2.5 Ghz (C/C++)
Z. Lv, C. Beall, P. Alcantarilla, F. Li, Z. Kira and F. Dellaert: A Continuous Optimization Approach for Efficient and Accurate Scene Flow. European Conf. on Computer Vision (ECCV) 2016.
113	DR-Net			4.97 %	11.54 %	6.06 %	100.00 %	18 s	1 core @ 2.5 Ghz (Python)
114	MBM			4.69 %	13.05 %	6.08 %	100.00 %	0.13 s	1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: A Multi-Block-Matching Approach for Stereo. IV 2015.
115	PR-Sceneflow		code	4.74 %	13.74 %	6.24 %	100.00 %	150 s	4 core @ 3.0 Ghz (Matlab + C/C++)
C. Vogel, K. Schindler and S. Roth: Piecewise Rigid Scene Flow. ICCV 2013.
116	SGM+DAISY		code	4.86 %	13.42 %	6.29 %	95.26 %	5 s	1 core @ 2.5 Ghz (C/C++)
117	DeepCostAggr		code	5.34 %	11.35 %	6.34 %	99.98 %	0.03 s	GPU @ 2.5 Ghz (C/C++)
A. Kuzmin, D. Mikushin and V. Lempitsky: End-to-end Learning of Cost-Volume Aggregation for Real-time Dense Stereo. 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) 2017.
118	SGM_ROB			5.06 %	13.00 %	6.38 %	100.00 %	0.11 s	Nvidia GTX 980
H. Hirschm\"uller: Stereo Processing by Semi-Global Matching and Mutual Information. IEEE Transactions on Pattern Analysis and Machine Intelligence 2008.
119	SceneFFields			5.12 %	13.83 %	6.57 %	100.00 %	65 s	4 cores @ 3.7 Ghz (C/C++)
R. Schuster, O. Wasenmüller, G. Kuschk, C. Bailer and D. Stricker: SceneFlowFields: Dense Interpolation of Sparse Scene Flow Correspondences. IEEE Winter Conference on Applications of Computer Vision (WACV) 2018.
120	SPS+FF++			5.47 %	12.19 %	6.59 %	100.00 %	36 s	1 core @ 3.5 Ghz (C/C++)
121	unscene			5.10 %	14.55 %	6.67 %	100.00 %	0.08 s	1 core @ 2.5 Ghz (C/C++)
122	FSF+MS			5.72 %	11.84 %	6.74 %	100.00 %	2.7 s	4 cores @ 3.5 Ghz (C/C++)
T. Taniai, S. Sinha and Y. Sato: Fast Multi-frame Stereo Scene Flow with Motion Segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017) 2017.
123	AABM			4.88 %	16.07 %	6.74 %	100.00 %	0.08 s	1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Stereo Image Warping for Improved Depth Estimation of Road Surfaces. IV 2013.
124	DLM-Net			5.04 %	15.76 %	6.83 %	100.00 %	0.68 s	1 core @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
125	SGM+C+NL		code	5.15 %	15.29 %	6.84 %	100.00 %	4.5 min	1 core @ 2.5 Ghz (C/C++)
H. Hirschmüller: Stereo Processing by Semiglobal Matching and Mutual Information. PAMI 2008. D. Sun, S. Roth and M. Black: A Quantitative Analysis of Current Practices in Optical Flow Estimation and the Principles Behind Them. IJCV 2013.
126	SGM+LDOF		code	5.15 %	15.29 %	6.84 %	100.00 %	86 s	1 core @ 2.5 Ghz (C/C++)
H. Hirschmüller: Stereo Processing by Semiglobal Matching and Mutual Information. PAMI 2008. T. Brox and J. Malik: Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation. PAMI 2011.
127	SGM+SF			5.15 %	15.29 %	6.84 %	100.00 %	45 min	16 core @ 3.2 Ghz (C/C++)
H. Hirschmüller: Stereo Processing by Semiglobal Matching and Mutual Information. PAMI 2008. M. Hornacek, A. Fitzgibbon and C. Rother: SphereFlow: 6 DoF Scene Flow from RGB-D Pairs. CVPR 2014.
128	SSCNNRef			5.87 %	12.75 %	7.01 %	100.00 %	0.05 s	GPU @ 1.0 Ghz (C/C++)
129	SNCC			5.36 %	16.05 %	7.14 %	100.00 %	0.08 s	1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: A Two-Stage Correlation Method for Stereoscopic Depth Estimation. DICTA 2010.
130	DSimNet			6.15 %	13.20 %	7.32 %	100.00 %	0.57 s	GPU @ 2.5 Ghz (Python)
131	WCMA_ROB			5.68 %	16.36 %	7.45 %	100.00 %	40 s	1 core @ 2.5 Ghz (Matlab + C/C++)
132	DMDE			6.89 %	12.92 %	7.90 %	100.00 %	7 s	4 cores @ 3.0 Ghz (C/C++)
133	CSCT+SGM+MF			6.91 %	14.87 %	8.24 %	100.00 %	0.0064 s	Nvidia GTX Titan X @ 1.0 Ghz (CUDA)
D. Hernandez-Juarez, A. Chacon, A. Espinosa, D. Vazquez, J. Moure and A. Lopez: Embedded real-time stereo estimation via Semi-Global Matching on the GPU. Procedia Computer Science 2016.
134	MeshStereo		code	5.82 %	21.21 %	8.38 %	100.00 %	87 s	1 core @ 2.5 Ghz (C/C++)
C. Zhang, Z. Li, Y. Cheng, R. Cai, H. Chao and Y. Rui: MeshStereo: A Global Stereo Model With Mesh Alignment Regularization for View Interpolation. The IEEE International Conference on Computer Vision (ICCV) 2015.
135	PCOF + ACTF			6.31 %	19.24 %	8.46 %	100.00 %	0.08 s	GPU @ 2.0 Ghz (C/C++)
M. Derome, A. Plyer, M. Sanfourche and G. Le Besnerais: A Prediction-Correction Approach for Real-Time Optical Flow Computation Using Stereo. German Conference on Pattern Recognition 2016.
136	PCOF-LDOF			6.31 %	19.24 %	8.46 %	100.00 %	50 s	1 core @ 3.0 Ghz (C/C++)
M. Derome, A. Plyer, M. Sanfourche and G. Le Besnerais: A Prediction-Correction Approach for Real-Time Optical Flow Computation Using Stereo. German Conference on Pattern Recognition 2016.
137	OASM-Net			6.89 %	19.42 %	8.98 %	100.00 %	0.73 s	GPU @ 2.5 Ghz (Python)
138	ELAS_ROB		code	7.38 %	21.15 %	9.67 %	100.00 %	0.19 s	4 cores @ >3.5 Ghz (C/C++)
A. Geiger, M. Roser and R. Urtasun: Efficient Large-Scale Stereo Matching. ACCV 2010.
139	ELAS		code	7.86 %	19.04 %	9.72 %	92.35 %	0.3 s	1 core @ 2.5 Ghz (C/C++)
A. Geiger, M. Roser and R. Urtasun: Efficient Large-Scale Stereo Matching. ACCV 2010.
140	DPSimNet_ROB			8.92 %	15.63 %	10.04 %	100.00 %	0.67 s	GPU @ 2.5 Ghz (Python)
141	REAF		code	8.43 %	18.51 %	10.11 %	100.00 %	1.1 s	1 core @ 2.5 Ghz (C/C++)
C. Cigla: Recursive Edge-Aware Filters for Stereo Matching. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2015.
142	APMBP			8.85 %	20.65 %	10.82 %	100.00 %	5 min	1 core @ 2.5 Ghz (Matlab)
143	iGF			8.64 %	21.85 %	10.84 %	100.00 %	220 s	1 core @ 3.0 Ghz (C/C++)
R. Hamzah, H. Ibrahim and A. Hassan: Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation. Journal of Visual Communication and Image Representation 2016.
144	OCV-SGBM		code	8.92 %	20.59 %	10.86 %	90.41 %	1.1 s	1 core @ 2.5 Ghz (C/C++)
H. Hirschmueller: Stereo processing by semiglobal matching and mutual information. PAMI 2008.
145	SDM			9.41 %	24.75 %	11.96 %	62.56 %	1 min	1 core @ 2.5 Ghz (C/C++)
J. Kostkova: Stratified dense matching for stereopsis in complex scenes. BMVC 2003.
146	DSGCA			10.54 %	20.79 %	12.25 %	100.00 %	144 s	>8 cores @ 3.5 Ghz (C/C++)
147	SGM&FlowFie+			11.93 %	20.57 %	13.37 %	81.24 %	29 s	1 core @ 3.5 Ghz (C/C++)
R. Schuster, C. Bailer, O. Wasenmüller and D. Stricker: Combining Stereo Disparity and Optical Flow for Basic Scene Flow. Commercial Vehicle Technology Symposium (CVTS) 2018.
148	GCSF		code	11.64 %	27.11 %	14.21 %	100.00 %	2.4 s	1 core @ 2.5 Ghz (C/C++)
J. Cech, J. Sanchez-Riera and R. Horaud: Scene Flow Estimation by growing Correspondence Seeds. CVPR 2011.
149	NOSS_ROB			14.81 %	12.36 %	14.40 %	99.59 %	240 s	4 cores @ >3.5 Ghz (C/C++)
150	CostFilter		code	17.53 %	22.88 %	18.42 %	100.00 %	4 min	1 core @ 2.5 Ghz (Matlab)
C. Rhemann, A. Hosni, M. Bleyer, C. Rother and M. Gelautz: Fast Cost-Volume Filtering for Visual Correspondence and Beyond. CVPR 2011.
151	DWBSF			19.61 %	22.69 %	20.12 %	100.00 %	7 min	4 cores @ 3.5 Ghz (C/C++)
C. Richardt, H. Kim, L. Valgaerts and C. Theobalt: Dense Wide-Baseline Scene Flow From Two Handheld Video Cameras. 3DV 2016.
152	SVS			25.18 %	20.77 %	24.44 %	100.00 %	0.13 s	GPU @ 2.5 Ghz (Matlab + C/C++)
153	OCV-BM		code	24.29 %	30.13 %	25.27 %	58.54 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
G. Bradski: The OpenCV Library. Dr. Dobb's Journal of Software Tools 2000.
154	VSF		code	27.31 %	21.72 %	26.38 %	100.00 %	125 min	1 core @ 2.5 Ghz (C/C++)
F. Huguet and F. Devernay: A Variational Method for Scene Flow Estimation from Stereo Sequences. ICCV 2007.
155	SED		code	25.01 %	40.43 %	27.58 %	4.02 %	0.68 s	1 core @ 2.0 Ghz (C/C++)
D. Pe\~{n}a and A. Sutherland: Disparity Estimation by Simultaneous Edge Drawing. Computer Vision -- ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 2017.
156	MST		code	45.83 %	38.22 %	44.57 %	100.00 %	7 s	1 core @ 2.5 Ghz (Matlab + C/C++)
Q. Yang: A Non-Local Cost Aggregation Method for Stereo Matching. CVPR 2012.
157	MEDIAN_ROB			42.67 %	84.03 %	49.55 %	100.00 %	0.01 s	1 core @ 2.5 Ghz (C/C++)
158	seg_stereo			60.71 %	72.46 %	62.67 %	100.00 %	20 min	8 cores @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
159	AVERAGE_ROB			71.19 %	79.59 %	72.59 %	100.00 %	0.01 s	1 core @ 2.5 Ghz (C/C++)
160	DC-NET			89.74 %	96.68 %	90.89 %	100.00 %	0.2 s	1 core @ 2.5 Ghz (Python)

• HCI/Bosch Robust Vision Challenge: Optical flow and stereo vision challenge on high resolution imagery recorded at a high frame rate under diverse weather conditions (e.g., sunny, cloudy, rainy). The Robert Bosch AG provides a prize for the best performing method.
• Image Sequence Analysis Test Site (EISATS): Synthetic image sequences with ground truth information provided by UoA and Daimler AG. Some of the images come with 3D range sensor information.
• Middlebury Stereo Evaluation: The classic stereo evaluation benchmark, featuring four test images in version 2 of the benchmark, with very accurate ground truth from a structured light system. 38 image pairs are provided in total.
• Daimler Stereo Dataset: Stereo bad weather highway scenes with partial ground truth for freespace
• Make3D Range Image Data: Images with small-resolution ground truth used to learn and evaluate depth from single monocular images.
• Lubor Ladicky's Stereo Dataset: Stereo Images with manually labeled ground truth based on polygonal areas.
• Middlebury Optical Flow Evaluation: The classic optical flow evaluation benchmark, featuring eight test images, with very accurate ground truth from a shape from UV light pattern system. 24 image pairs are provided in total.

When using this dataset in your research, we will be happy if you cite us:
@ARTICLE{Menze2018JPRS,
  author = {Moritz Menze and Christian Heipke and Andreas Geiger},
  title = {Object Scene Flow},
  journal = {ISPRS Journal of Photogrammetry and Remote Sensing (JPRS)},
  year = {2018}
}
@INPROCEEDINGS{Menze2015ISA,
  author = {Moritz Menze and Christian Heipke and Andreas Geiger},
  title = {Joint 3D Estimation of Vehicles and Scene Flow},
  booktitle = {ISPRS Workshop on Image Sequence Analysis (ISA)},
  year = {2015}
}