The KITTI Vision Benchmark Suite

The bird's eye view benchmark consists of 7481 training images and 7518 test images as well as the corresponding point clouds, comprising a total of 80.256 labeled objects. For evaluation, we compute precision-recall curves. To rank the methods we compute average precision. We require that all methods use the same parameter set for all test pairs. Our development kit provides details about the data format as well as MATLAB / C++ utility functions for reading and writing the label files.

• Download left color images of object data set (12 GB)
• Download right color images, if you want to use stereo information (12 GB)
• Download the 3 temporally preceding frames (left color) (36 GB)
• Download the 3 temporally preceding frames (right color) (36 GB)
• Download Velodyne point clouds, if you want to use laser information (29 GB)
• Download camera calibration matrices of object data set (16 MB)
• Download training labels of object data set (5 MB)
• Download object development kit (1 MB) (including 3D object detection and bird's eye view evaluation code)
• Download pre-trained LSVM baseline models (5 MB) used in Joint 3D Estimation of Objects and Scene Layout (NIPS 2011). These models are referred to as LSVM-MDPM-sv (supervised version) and LSVM-MDPM-us (unsupervised version) in the tables below.
• Download reference detections (L-SVM) for training and test set (800 MB)
• Qianli Liao (NYU) has put together code to convert from KITTI to PASCAL VOC file format (documentation included, requires Emacs).
• Karl Rosaen (U.Mich) has released code to convert between KITTI, KITTI tracking, Pascal VOC, Udacity, CrowdAI and AUTTI formats.
• We thank David Stutz and Bo Li for developing the 3D object detection benchmark.

We evaluate bird's eye view detection performance using the PASCAL criteria also used for 2D object detection. Far objects are thus filtered based on their bounding box height in the image plane. As only objects also appearing on the image plane are labeled, objects in don't car areas do not count as false positives. We note that the evaluation does not take care of ignoring detections that are not visible on the image plane — these detections might give rise to false positives. For cars we require a bounding box overlap of 70% in bird's eye view, while for pedestrians and cyclists we require an overlap of 50%. Difficulties are defined as follows:

• Easy: Min. bounding box height: 40 Px, Max. occlusion level: Fully visible, Max. truncation: 15 %
• Moderate: Min. bounding box height: 25 Px, Max. occlusion level: Partly occluded, Max. truncation: 30 %
• Hard: Min. bounding box height: 25 Px, Max. occlusion level: Difficult to see, Max. truncation: 50 %

All methods are ranked based on the moderately difficult results.

Car

	Method	Setting	Code	Moderate	Easy	Hard	Runtime	Environment
1	Alibaba-AILabsX			87.49 %	89.10 %	78.98 %	0.05 s	1 core @ >3.5 Ghz (C/C++)
2	UberATG-MMF			87.47 %	89.49 %	79.10 %	0.08 s	GPU @ 2.5 Ghz (Python)
M. Liang*, B. Yang*, Y. Chen, R. Hu and R. Urtasun: Multi-Task Multi-Sensor Fusion for 3D Object Detection. CVPR 2019.
3	EMP			87.45 %	89.96 %	83.63 %	0.5 s	GPU @ 2.5 Ghz (Python)
4	GPOD			87.39 %	89.54 %	81.65 %	0.1 s	GPU @ 2.5 Ghz (Python)
5	A-VoxelNet			86.96 %	88.30 %	78.44 %	0.029 s	GPU @ 2.5 Ghz (Python)
6	MPNet			86.95 %	89.28 %	80.32 %	0.02 s	GPU @ 2.5 Ghz (Python + C/C++)
7	MMV			86.90 %	88.61 %	78.45 %	0.4 s	GPU @ 2.5 Ghz (C/C++)
8	Alibaba-AILabsX			86.71 %	88.84 %	78.62 %	0.2 s	GPU @ >3.5 Ghz (Python)
9	ARPNET			86.67 %	88.10 %	77.94 %	0.08 s	GPU @ 2.5 Ghz (Python + C/C++)
10	SECOND-V1.5		code	86.60 %	88.72 %	78.51 %	0.04 s	GPU @ 2.0 Ghz (Python + C/C++)
11	UberATG-HDNET			86.57 %	89.14 %	78.32 %	0.05 s	GPU @ 2.5 Ghz (Python)
B. Yang, M. Liang and R. Urtasun: HDNET: Exploiting HD Maps for 3D Object Detection. 2nd Conference on Robot Learning (CoRL) 2018.
12	Patches			86.55 %	89.78 %	79.22 %	0.15 s	GPU @ 2.0 Ghz
13	Tencent_ADlab_Lidar			86.29 %	88.67 %	79.27 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
14	NU-optim			86.21 %	88.39 %	78.34 %	0.04 s	GPU @ >3.5 Ghz (Python)
15	PC-CNN-V2			86.10 %	88.49 %	77.26 %	0.5 s	GPU @ 2.5 Ghz (Matlab + C/C++)
X. Du, M. Ang, S. Karaman and D. Rus: A General Pipeline for 3D Detection of Vehicles. 2018 IEEE International Conference on Robotics and Automation (ICRA) 2018.
16	PointPillars		code	86.10 %	88.35 %	79.83 %	16 ms	1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
17	Fast Point R-CNNv1.1			86.10 %	88.03 %	78.17 %	0.06 s	GPU @ 2.5 Ghz (Python + C/C++)
18	DH-ARI			86.08 %	87.93 %	78.22 %	0.2 s	1 core @ >3.5 Ghz (Python + C/C++)
19	RoarNet		code	86.08 %	88.75 %	78.80 %	0.1 s	GPU @ >3.5 Ghz (Python + C/C++)
K. Shin, Y. Kwon and M. Tomizuka: RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement. arXiv preprint arXiv:1811.03818 2018.
20	PointRCNN-deprecated			86.04 %	89.28 %	79.02 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
21	MVX-Net			85.89 %	89.15 %	78.07 %	0.06 s	GPU @ 3.0 Ghz (Python + C/C++)
22	UberATG-ContFuse			85.83 %	88.81 %	77.33 %	0.06 s	GPU @ 2.5 Ghz (Python)
M. Liang, B. Yang, S. Wang and R. Urtasun: Deep Continuous Fusion for Multi-Sensor 3D Object Detection. ECCV 2018.
23	MDC			85.68 %	88.65 %	77.03 %	0.17 s	GPU @ 2.5 Ghz (Python)
24	PointRCNN			85.68 %	89.47 %	79.10 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. CVPR 2019.
25	Fast Point R-CNN			85.53 %	87.98 %	77.68 %	0.06 s	GPU @ 2.5 Ghz (Python + C/C++)
26	AVOD		code	85.44 %	86.80 %	77.73 %	0.08 s	Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
27	SeoulRobotics-HFD			85.22 %	87.17 %	77.71 %	0.035 s	GPU (C++)
28	Roadstar.ai			85.04 %	89.95 %	78.75 %	0.08 s	GPU @ 2.0 Ghz (Python)
29	CONV-BOX			84.56 %	87.54 %	77.79 %	0.2 s	Tesla V100
30	F-PointNet		code	84.00 %	88.70 %	75.33 %	0.17 s	GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
31	IPOD			83.98 %	86.93 %	77.85 %	0.2 s	GPU @ 2.5 Ghz (Python + C/C++)
32	AVOD-FPN		code	83.79 %	88.53 %	77.90 %	0.1 s	Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
33	UberATG-PIXOR++			83.70 %	89.38 %	77.97 %	0.035 s	GPU @ 2.5 Ghz (Python)
B. Yang, M. Liang and R. Urtasun: HDNET: Exploiting HD Maps for 3D Object Detection. 2nd Conference on Robot Learning (CoRL) 2018.
34	F-ConvNet			83.08 %	89.69 %	74.56 %	0.47 s	GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. arXiv preprint arXiv:1903.01864 2019.
35	KP			82.24 %	89.37 %	79.03 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
36	PP_v1.0		code	81.12 %	87.72 %	78.13 %	0.02s	1 core @ 2.5 Ghz (C/C++)
37	Voxel-FPN			80.17 %	88.96 %	79.14 %	0.02 s	GPU @ 2.5 Ghz (Python + C/C++)
38	PFPN			80.14 %	89.05 %	79.02 %	0.02 s	4 cores @ >3.5 Ghz (Python)
39	yl_net			80.06 %	85.64 %	77.52 %	0.03 s	GPU @ 2.5 Ghz (Python)
40	CFR			79.68 %	88.77 %	78.21 %	0.06 s	1 core @ 2.5 Ghz (C/C++)
41	3DBN			79.40 %	88.13 %	77.97 %	0.13s	1080Ti (Python+C/C++)
X. Li, J. Guivant, N. Kwok and Y. Xu: 3D Backbone Network for 3D Object Detection. CoRR 2019.
42	SECOND		code	79.37 %	88.07 %	77.95 %	38 ms	1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
43	ELLIOT			79.30 %	87.06 %	77.28 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
44	AILabs3D			79.09 %	88.32 %	77.40 %	0.6 s	GPU @ >3.5 Ghz (Python)
45	SCANet			79.01 %	87.02 %	77.20 %	0.17 s	>8 cores @ 2.5 Ghz (Python)
46	DFD			78.92 %	86.98 %	77.22 %	0.05 s	GPU @ 2.0 Ghz (Python + C/C++)
47	SCANet			78.64 %	87.36 %	77.37 %	0.09s	GPU @ 2.5 Ghz (Python)
48	SECA			78.54 %	87.52 %	77.25 %	0.09 s	GPU @ 2.5 Ghz (Python)
49	FNV1_RPN			78.51 %	87.40 %	70.23 %	0.12 s	1 core @ 2.5 Ghz (Python + C/C++)
50	SECA			78.50 %	87.65 %	76.99 %	1 s	GPU @ 2.5 Ghz (Python)
51	VSE			78.50 %	87.65 %	76.99 %	0.15 s	GPU @ 2.5 Ghz (Python)
52	NLK			78.28 %	87.65 %	76.20 %	0.02 s	1 core @ 2.5 Ghz (Python)
53	FNV1_Fusion			78.23 %	87.07 %	76.72 %	0.11 s	GPU @ 2.5 Ghz (Python)
54	X_MD			77.80 %	87.02 %	69.66 %	0.2 s	1 core @ 2.5 Ghz (Python + C/C++)
55	RTL3D			77.75 %	86.95 %	76.08 %	0.02 s	GPU @ 2.5 Ghz (Python)
56	FNV1			77.69 %	85.53 %	69.46 %	0.11 s	GPU @ 2.5 Ghz (Python)
57	UberATG-PIXOR			77.05 %	81.70 %	72.95 %	0.035 s	TITAN Xp (Python)
B. Yang, W. Luo and R. Urtasun: PIXOR: Real-time 3D Object Detection from Point Clouds. CVPR 2018.
58	MV3D (LIDAR)			77.00 %	85.82 %	68.94 %	0.24 s	GPU @ 2.5 Ghz (Python + C/C++)
X. Chen, H. Ma, J. Wan, B. Li and T. Xia: Multi-View 3D Object Detection Network for Autonomous Driving. CVPR 2017.
59	MV3D			76.90 %	86.02 %	68.49 %	0.36 s	GPU @ 2.5 Ghz (Python + C/C++)
X. Chen, H. Ma, J. Wan, B. Li and T. Xia: Multi-View 3D Object Detection Network for Autonomous Driving. CVPR 2017.
60	VoxelNet(Unofficial)			75.06 %	83.25 %	67.44 %	0.5 s	GPU @ 2.0 Ghz (Python)
61	FNV2			74.36 %	80.68 %	65.88 %	0.18 s	GPU @ 2.5 Ghz (Python)
62	CLF3D			73.01 %	80.09 %	58.56 %	0.13 s	GPU @ 2.5 Ghz (Python)
63	A3DODWTDA		code	72.86 %	76.65 %	64.51 %	0.08 s	GPU @ 3.0 Ghz (Python)
F. Gustafsson and E. Linder-Norén: Automotive 3D Object Detection Without Target Domain Annotations. 2018.
64	anm			72.73 %	81.79 %	63.98 %	3 s	1 core @ 2.5 Ghz (C/C++)
65	avodC			69.53 %	84.54 %	67.98 %	0.1 s	GPU @ 2.5 Ghz (Python)
66	TopNet-Retina			67.70 %	78.71 %	59.38 %	0.09 s	1 core @ 2.5 Ghz (C/C++)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
67	Complexer-YOLO			66.07 %	74.23 %	65.70 %	0.06 s	GPU @ 3.5 Ghz (C/C++)
68	TopNet-DecayRate			64.12 %	79.76 %	56.48 %	92 ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
69	3D FCN			62.54 %	69.94 %	55.94 %	>5 s	1 core @ 2.5 Ghz (C/C++)
B. Li: 3D Fully Convolutional Network for Vehicle Detection in Point Cloud. IROS 2017.
70	TopNet-UncEst			62.42 %	70.13 %	55.20 %	0.09 s	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
71	TopNet-HighRes			53.71 %	67.53 %	46.54 %	101ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
72	BirdNet			50.81 %	75.52 %	50.00 %	0.11 s	Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
73	Pseudo-LiDAR			47.20 %	66.83 %	40.30 %	0.4 s	GPU @ 2.5 Ghz (C/C++)
Y. Wang, W. Chao, D. Garg, B. Hariharan, M. Campbell and K. Weinberger: Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving. CVPR 2019.
74	SA_3D			47.00 %	58.98 %	39.47 %	0.3 s	GPU @ 2.5 Ghz (Python)
75	PL			45.11 %	65.08 %	38.42 %	0.4 s	GPU @ 2.5 Ghz (Python)
76	Stereo R-CNN			43.87 %	61.67 %	36.44 %	0.4 s	GPU @ 2.5 Ghz (Python)
P. Li, X. Chen and S. Shen: Stereo R-CNN based 3D Object Detection for Autonomous Driving. CVPR 2019.
77	RT3D			42.10 %	54.68 %	44.05 %	0.09 s	GPU @ 1.8Ghz
Y. Zeng, Y. Hu, S. Liu, J. Ye, Y. Han, X. Li and N. Sun: RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving. IEEE Robotics and Automation Letters 2018.
78	Licar			40.40 %	45.81 %	37.09 %	0.09 s	GPU @ 2.0 Ghz (Python)
79	DLMB			37.10 %	45.12 %	32.88 %	0.03 s	8 cores @ 3.5 Ghz (C/C++)
80	monocular			22.24 %	27.91 %	18.62 %	0.4 s	GPU @ 2.5 Ghz (Python + C/C++)
81	CSoR			18.69 %	23.94 %	16.30 %	3.5 s	4 cores @ >3.5 Ghz (Python + C/C++)
L. Plotkin: PyDriver: Entwicklung eines Frameworks für räumliche Detektion und Klassifikation von Objekten in Fahrzeugumgebung. 2015.
82	MonoPSR			17.66 %	20.25 %	15.78 %	0.2 s	GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
83	DT3D			17.19 %	23.38 %	13.86 %	0,21s	GPU @ 2.5 Ghz (Python)
84	MonoGRNet			16.37 %	20.55 %	15.16 %	0.04s	NVIDIA P40
Z. Qin, J. Wang and Y. Lu: MonoGRNet: A Geometric Reasoning Network for 3D Object Localization. The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) 2019.
85	SA_3D			15.99 %	21.09 %	14.56 %	0.3 s	1 core @ 2.5 Ghz (Python)
86	SS3D			12.81 %	17.86 %	12.28 %	48 ms	Tesla V100 (Python)
87	ROI-10D			12.40 %	16.77 %	11.39 %	0.2 s	GPU @ 3.5 Ghz (Python)
F. Manhardt, W. Kehl and A. Gaidon: ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape. Computer Vision and Pattern Recognition (CVPR) 2019.
88	SA_3D			12.17 %	19.13 %	10.56 %	0.3 s	1 core @ 2.5 Ghz (Python)
89	A3DODWTDA (image)		code	10.21 %	10.61 %	8.64 %	0.8 s	GPU @ 3.0 Ghz (Python)
F. Gustafsson and E. Linder-Norén: Automotive 3D Object Detection Without Target Domain Annotations. 2018.
90	Shift R-CNN			8.49 %	13.32 %	6.40 %	0.25 s	GPU @ 1.5 Ghz (Python)
91	OFT-Net			7.99 %	9.50 %	7.51 %	0.5 s	8 cores @ 2.5 Ghz (Python + C/C++)
92	MF3D			5.57 %	7.88 %	5.08 %	0.03 s	GPU @ 2.5 Ghz (C/C++)
93	FQNet			4.62 %	6.51 %	3.99 %	0.5 s	1 core @ 2.5 Ghz (Python)
94	3D-SSMFCNN		code	3.19 %	3.66 %	3.45 %	0.1 s	GPU @ 1.5 Ghz (C/C++)
L. Novak: Vehicle Detection and Pose Estimation for Autonomous Driving. 2017.
95	3DVSSD			2.01 %	2.02 %	1.68 %	0.06 s	1 core @ 2.5 Ghz (C/C++)
96	VeloFCN			0.33 %	0.15 %	0.47 %	1 s	GPU @ 2.5 Ghz (Python + C/C++)
B. Li, T. Zhang and T. Xia: Vehicle Detection from 3D Lidar Using Fully Convolutional Network. RSS 2016 .
97	multi-task CNN			0.00 %	0.00 %	0.00 %	25.1 ms	GPU @ 2.0 Ghz (Python)
M. Oeljeklaus, F. Hoffmann and T. Bertram: A Fast Multi-Task CNN for Spatial Understanding of Traffic Scenes. IEEE Intelligent Transportation Systems Conference 2018.
98	mBoW			0.00 %	0.00 %	0.00 %	10 s	1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.

Pedestrian

	Method	Setting	Code	Moderate	Easy	Hard	Runtime	Environment
1	A-VoxelNet			51.90 %	60.44 %	49.48 %	0.029 s	GPU @ 2.5 Ghz (Python)
2	VMVS			51.73 %	61.46 %	47.69 %	0.25 s	GPU @ 2.5 Ghz (Python)
3	IPOD			51.24 %	60.83 %	45.40 %	0.2 s	GPU @ 2.5 Ghz (Python + C/C++)
4	AVOD-FPN		code	51.05 %	58.75 %	47.54 %	0.1 s	Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
5	F-ConvNet			50.48 %	58.90 %	46.72 %	0.47 s	GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. arXiv preprint arXiv:1903.01864 2019.
6	PointPillars		code	50.23 %	58.66 %	47.19 %	16 ms	1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
7	F-PointNet		code	50.22 %	58.09 %	47.20 %	0.17 s	GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
8	Tencent_ADlab_Lidar			48.93 %	56.89 %	46.28 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
9	PointRCNN			47.53 %	55.92 %	44.67 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. CVPR 2019.
10	PP_v1.0		code	46.48 %	54.26 %	44.14 %	0.02s	1 core @ 2.5 Ghz (C/C++)
11	GPOD			46.39 %	53.09 %	43.62 %	0.1 s	GPU @ 2.5 Ghz (Python)
12	SECOND		code	46.27 %	55.10 %	44.76 %	38 ms	1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
13	Roadstar.ai			46.22 %	49.94 %	44.14 %	0.08 s	GPU @ 2.0 Ghz (Python)
14	CONV-BOX			45.09 %	52.71 %	43.90 %	0.2 s	Tesla V100
15	MDC			45.02 %	53.51 %	43.67 %	0.17 s	GPU @ 2.5 Ghz (Python)
16	SCANet			44.30 %	53.68 %	42.65 %	0.17 s	>8 cores @ 2.5 Ghz (Python)
17	ARPNET			43.35 %	50.80 %	37.79 %	0.08 s	GPU @ 2.5 Ghz (Python + C/C++)
18	ELLIOT			42.66 %	50.68 %	39.67 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
19	anm			40.48 %	50.07 %	35.64 %	3 s	1 core @ 2.5 Ghz (C/C++)
20	CFR			39.24 %	52.47 %	38.07 %	0.06 s	1 core @ 2.5 Ghz (C/C++)
21	SA_3D			35.93 %	47.45 %	34.99 %	0.3 s	GPU @ 2.5 Ghz (Python)
22	anonymous			35.58 %	42.50 %	35.06 %	0.75 s	GPU @ 3.5 Ghz (C/C++)
23	X_MD			35.57 %	43.28 %	34.25 %	0.2 s	1 core @ 2.5 Ghz (C/C++)
24				35.26 %	43.26 %	32.85 %
25	AVOD		code	35.24 %	42.51 %	33.97 %	0.08 s	Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
26	CLF3D			34.25 %	43.11 %	33.01 %	0.13 s	GPU @ 2.5 Ghz (Python)
27	BirdNet			21.35 %	26.07 %	19.96 %	0.11 s	Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
28	Complexer-YOLO			20.88 %	22.00 %	20.81 %	0.06 s	GPU @ 3.5 Ghz (C/C++)
29	TopNet-HighRes			19.08 %	24.30 %	18.46 %	101ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
30	TopNet-DecayRate			12.59 %	15.09 %	12.23 %	92 ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
31	TopNet-UncEst			11.63 %	12.39 %	11.39 %	0.09 s	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
32	TopNet-Retina			11.45 %	10.84 %	11.41 %	0.09 s	1 core @ 2.5 Ghz (C/C++)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
33	Shift R-CNN			11.44 %	13.81 %	10.76 %	0.25 s	GPU @ 1.5 Ghz (Python)
34	MonoPSR			11.22 %	14.27 %	10.54 %	0.2 s	GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
35	SS3D			3.52 %	3.86 %	2.50 %	48 ms	Tesla V100 (Python)
36	SA_3D			3.52 %	4.56 %	3.52 %	0.3 s	1 core @ 2.5 Ghz (Python)
37	SA_3D			2.64 %	3.91 %	2.63 %	0.3 s	1 core @ 2.5 Ghz (Python)
38	OFT-Net			1.55 %	1.93 %	1.65 %	0.5 s	8 cores @ 2.5 Ghz (Python + C/C++)
39	DT3D			1.22 %	1.15 %	1.14 %	0,21s	GPU @ 2.5 Ghz (Python)
40	mBoW			0.01 %	0.01 %	0.01 %	10 s	1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.

Cyclist

	Method	Setting	Code	Moderate	Easy	Hard	Runtime	Environment
1	F-ConvNet			68.62 %	82.59 %	60.62 %	0.47 s	GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. arXiv preprint arXiv:1903.01864 2019.
2	PointRCNN			66.77 %	81.52 %	60.78 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud. CVPR 2019.
3	MDC			64.78 %	77.38 %	56.77 %	0.17 s	GPU @ 2.5 Ghz (Python)
4	ARPNET			62.76 %	77.99 %	55.55 %	0.08 s	GPU @ 2.5 Ghz (Python + C/C++)
5	Tencent_ADlab_Lidar			62.33 %	78.15 %	55.87 %	0.1 s	GPU @ 2.5 Ghz (Python + C/C++)
6	PointPillars		code	62.25 %	79.14 %	56.00 %	16 ms	1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
7	F-PointNet		code	61.96 %	75.38 %	54.68 %	0.17 s	GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
8	CONV-BOX			61.84 %	71.60 %	55.03 %	0.2 s	Tesla V100
9	A-VoxelNet			60.85 %	76.71 %	54.27 %	0.029 s	GPU @ 2.5 Ghz (Python)
10	GPOD			60.73 %	68.84 %	55.05 %	0.1 s	GPU @ 2.5 Ghz (Python)
11	Roadstar.ai			60.26 %	67.31 %	54.62 %	0.08 s	GPU @ 2.0 Ghz (Python)
12	SCANet			59.09 %	70.73 %	52.79 %	0.17 s	>8 cores @ 2.5 Ghz (Python)
13	IPOD			58.92 %	77.10 %	51.01 %	0.2 s	GPU @ 2.5 Ghz (Python + C/C++)
14	ELLIOT			57.63 %	76.27 %	52.36 %	0.1 s	1 core @ 2.5 Ghz (C/C++)
15	AVOD-FPN		code	57.48 %	68.09 %	50.77 %	0.1 s	Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
16	SECOND		code	56.04 %	73.67 %	48.78 %	38 ms	1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
17	PP_v1.0		code	54.59 %	70.83 %	49.11 %	0.02s	1 core @ 2.5 Ghz (C/C++)
18	CFR			53.41 %	69.05 %	46.81 %	0.06 s	1 core @ 2.5 Ghz (C/C++)
19	AVOD		code	47.74 %	63.66 %	46.55 %	0.08 s	Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
20	X_MD			44.49 %	54.22 %	38.48 %	0.2 s	1 core @ 2.5 Ghz (C/C++)
21	CLF3D			41.57 %	53.29 %	35.80 %	0.13 s	GPU @ 2.5 Ghz (Python)
22	anm			40.42 %	56.84 %	35.20 %	3 s	1 core @ 2.5 Ghz (C/C++)
23	Complexer-YOLO			30.16 %	36.12 %	26.01 %	0.06 s	GPU @ 3.5 Ghz (C/C++)
24	BirdNet			27.18 %	38.93 %	25.51 %	0.11 s	Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
25	TopNet-Retina			24.31 %	38.16 %	23.92 %	0.09 s	1 core @ 2.5 Ghz (C/C++)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
26	SA_3D			20.56 %	26.45 %	16.42 %	0.3 s	GPU @ 2.5 Ghz (Python)
27	TopNet-DecayRate			19.92 %	28.06 %	19.13 %	92 ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
28	TopNet-HighRes			12.45 %	15.70 %	12.76 %	101ms	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
29	MonoPSR			12.17 %	14.75 %	11.35 %	0.2 s	GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
30	TopNet-UncEst			10.34 %	14.80 %	10.42 %	0.09 s	NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
31	SS3D			9.65 %	11.52 %	9.09 %	48 ms	Tesla V100 (Python)
32	Shift R-CNN			3.03 %	3.58 %	3.03 %	0.25 s	GPU @ 1.5 Ghz (Python)
33	SA_3D			2.75 %	3.06 %	2.57 %	0.3 s	1 core @ 2.5 Ghz (Python)
34	DT3D			1.26 %	2.54 %	1.47 %	0,21s	GPU @ 2.5 Ghz (Python)
35	SA_3D			1.12 %	2.20 %	1.18 %	0.3 s	1 core @ 2.5 Ghz (Python)
36	OFT-Net			0.43 %	0.79 %	0.43 %	0.5 s	8 cores @ 2.5 Ghz (Python + C/C++)
37	mBoW			0.00 %	0.00 %	0.00 %	10 s	1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.

• CERV Vehicle Lights Dataset: Annotations of vehicle lights for a subset of the object detection benchmark.
• PASCAL3D+: Augments 12 rigid object classes of PASCAL VOC 2012 with 3D annotations.
• The PASCAL Visual Object Classes Challenges: Dataset and benchmarks for object class recognition.
• TME Motorway Dataset: 28 video sequences with vehicle annotations captured from VisLab's BRAiVE vehicle.
• LabelMe: Online annotation tool to build image databases for computer vision research.
• MIT Street Scenes: Street-side images with labels for 9 object categories (including cars, pedestrians, buildings, trees).
• Daimler Pedestrian Datasets: Datasets focusing on pedestrian detection for autonomous driving.
• Caltech Pedestrian Detection Benchmark: 10 hours of video with 350.000 annotated pedestrian bounding boxes.
• Robust Multi-Person Tracking from Mobile Platforms: Videos with annotated pedestrians captured from a stroller.

When using this dataset in your research, we will be happy if you cite us:
@INPROCEEDINGS{Geiger2012CVPR,
  author = {Andreas Geiger and Philip Lenz and Raquel Urtasun},
  title = {Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2012}
}