Bird's Eye View Evaluation 2017


The bird's eye view benchmark consists of 7481 training images and 7518 test images as well as the corresponding point clouds, comprising a total of 80.256 labeled objects. For evaluation, we compute precision-recall curves. To rank the methods we compute average precision. We require that all methods use the same parameter set for all test pairs. Our development kit provides details about the data format as well as MATLAB / C++ utility functions for reading and writing the label files.

We evaluate bird's eye view detection performance using the PASCAL criteria also used for 2D object detection. Far objects are thus filtered based on their bounding box height in the image plane. As only objects also appearing on the image plane are labeled, objects in don't car areas do not count as false positives. We note that the evaluation does not take care of ignoring detections that are not visible on the image plane — these detections might give rise to false positives. For cars we require a bounding box overlap of 70% in bird's eye view, while for pedestrians and cyclists we require an overlap of 50%. Difficulties are defined as follows:

  • Easy: Min. bounding box height: 40 Px, Max. occlusion level: Fully visible, Max. truncation: 15 %
  • Moderate: Min. bounding box height: 25 Px, Max. occlusion level: Partly occluded, Max. truncation: 30 %
  • Hard: Min. bounding box height: 25 Px, Max. occlusion level: Difficult to see, Max. truncation: 50 %

All methods are ranked based on the moderately difficult results.

Note 2: On 08.10.2019, we have followed the suggestions of the Mapillary team in their paper Disentangling Monocular 3D Object Detection and use 40 recall positions instead of the 11 recall positions proposed in the original Pascal VOC benchmark. This results in a more fair comparison of the results, please check their paper. The last leaderboards right before this change can be found here: Object Detection Evaluation, 3D Object Detection Evaluation, Bird's Eye View Evaluation.
Important Policy Update: As more and more non-published work and re-implementations of existing work is submitted to KITTI, we have established a new policy: from now on, only submissions with significant novelty that are leading to a peer-reviewed paper in a conference or journal are allowed. Minor modifications of existing algorithms or student research projects are not allowed. Such work must be evaluated on a split of the training set. To ensure that our policy is adopted, new users must detail their status, describe their work and specify the targeted venue during registration. Furthermore, we will regularly delete all entries that are 6 months old but are still anonymous or do not have a paper associated with them. For conferences, 6 month is enough to determine if a paper has been accepted and to add the bibliography information. For longer review cycles, you need to resubmit your results.
Additional information used by the methods
  • Stereo: Method uses left and right (stereo) images
  • Flow: Method uses optical flow (2 temporally adjacent images)
  • Multiview: Method uses more than 2 temporally adjacent images
  • Laser Points: Method uses point clouds from Velodyne laser scanner
  • Additional training data: Use of additional data sources for training (see details)

Car


Method Setting Code Moderate Easy Hard Runtime Environment
1 SA-SSD 91.03 % 95.03 % 85.96 % 0.04 s 1 core @ 2.5 Ghz (Python)
2 PV-RCNN
This method makes use of Velodyne laser scans.
90.65 % 94.98 % 86.14 % 0.08 s 1 core @ 2.5 Ghz (Python + C/C++)
3 STD 89.19 % 94.74 % 86.42 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: STD: Sparse-to-Dense 3D Object Detector for Point Cloud. ICCV 2019.
4 Point-GNN
This method makes use of Velodyne laser scans.
89.17 % 93.11 % 83.90 % 0.6 s GPU @ 2.5 Ghz (Python)
5 Noah CV Lab - SSL 89.10 % 92.01 % 81.72 % 0.1 s GPU @ 2.5 Ghz (Python)
6 ORP 89.07 % 93.03 % 81.79 % 0.06 s 1 core @ 2.5 Ghz (C/C++)
7 3DSSD 89.02 % 92.66 % 85.86 % 0.04 s GPU @ 2.5 Ghz (Python + C/C++)
8 MLF_PointCas 88.99 % 92.60 % 81.74 % 0.1 s GPU @ 2.5 Ghz (Python)
9 GPOD
This method makes use of Velodyne laser scans.
88.86 % 93.56 % 83.22 % 0.1 s GPU @ 2.5 Ghz (Python)
10 HVNet 88.82 % 92.83 % 83.38 % 0.03 s GPU @ 2.0 Ghz (Python)
11 ELE 88.80 % 94.52 % 85.69 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
12 RGB3D
This method makes use of Velodyne laser scans.
88.69 % 92.84 % 81.76 % 0.39 s GPU @ 2.5 Ghz (Python)
13 FCPP 88.65 % 92.36 % 83.21 % 0.02 s 1 core @ 2.0 Ghz (Python + C/C++)
14 DENFIDet 88.56 % 92.42 % 83.76 % 0.02 s GPU @ 2.5 Ghz (C/C++)
15 EPNet 88.47 % 94.22 % 83.69 % 0.1 s 1 core @ 2.5 Ghz (Python + C/C++)
16 Patches
This method makes use of Velodyne laser scans.
88.39 % 92.72 % 83.19 % 0.15 s GPU @ 2.0 Ghz
J. Lehner, A. Mitterecker, T. Adler, M. Hofmarcher, B. Nessler and S. Hochreiter: Patch Refinement: Localized 3D Object Detection. arXiv preprint arXiv:1910.04093 2019.
17 UberATG-MMF
This method makes use of Velodyne laser scans.
88.21 % 93.67 % 81.99 % 0.08 s GPU @ 2.5 Ghz (Python)
M. Liang*, B. Yang*, Y. Chen, R. Hu and R. Urtasun: Multi-Task Multi-Sensor Fusion for 3D Object Detection. CVPR 2019.
18 Patches - EMP
This method makes use of Velodyne laser scans.
88.17 % 94.49 % 84.75 % 0.5 s GPU @ 2.5 Ghz (Python)
J. Lehner, A. Mitterecker, T. Adler, M. Hofmarcher, B. Nessler and S. Hochreiter: Patch Refinement: Localized 3D Object Detection. arXiv preprint arXiv:1910.04093 2019.
19 OHS 88.11 % 93.73 % 84.98 % 0.03 s 1 core @ 2.5 Ghz (Python/C++)
20 CPRCCNN 88.10 % 94.11 % 83.43 % 0.1 s 1 core @ 2.5 Ghz (Python)
21 DEFT 88.06 % 92.06 % 83.22 % 1 s GPU @ 2.5 Ghz (Python)
22 deprecated 88.05 % 91.96 % 83.21 % 0.05 s GPU @ >3.5 Ghz (Python)
23 3D-CVF
This method makes use of Velodyne laser scans.
88.04 % 91.97 % 83.22 % 0.05 s GPU @ >3.5 Ghz (Python)
24 UberATG-HDNET
This method makes use of Velodyne laser scans.
87.98 % 93.13 % 81.23 % 0.05 s GPU @ 2.5 Ghz (Python)
B. Yang, M. Liang and R. Urtasun: HDNET: Exploiting HD Maps for 3D Object Detection. 2nd Conference on Robot Learning (CoRL) 2018.
25 SPA 87.90 % 91.70 % 83.18 % 0.1 s 1 core @ 2.5 Ghz (Python)
26 Fast Point R-CNN v1
This method makes use of Velodyne laser scans.
87.84 % 90.87 % 80.52 % 0.06 s GPU @ 2.5 Ghz (Python + C/C++)
Y. Chen, S. Liu, X. Shen and J. Jia: Fast Point R-CNN. Proceedings of the IEEE international conference on computer vision (ICCV) 2019.
27 MMLab-PartA^2
This method makes use of Velodyne laser scans.
87.79 % 91.70 % 84.61 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, Z. Wang, X. Wang and H. Li: Part-A^2 Net: 3D Part-Aware and Aggregation Neural Network for Object Detection from Point Cloud. arXiv preprint arXiv:1907.03670 2019.
28 HRI-FusionRCNN 87.77 % 93.18 % 80.20 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
29 MODet
This method makes use of Velodyne laser scans.
87.56 % 90.80 % 82.69 % 0.05 s GTX1080Ti
30 AB3DMOT
This method makes use of Velodyne laser scans.
This is an online method (no batch processing).
code 87.53 % 91.99 % 81.03 % 0.0047s 1 core @ 2.5 Ghz (Python)
X. Weng and K. Kitani: A Baseline for 3D Multi-Object Tracking. arXiv:1907.03961 2019.
31 MLF_SecCas 87.46 % 92.54 % 77.39 % 0.05 s 1 core @ 2.5 Ghz (Python)
32 SAIC-SA-3D
This method makes use of Velodyne laser scans.
87.45 % 92.34 % 83.72 % 0.05 s GPU @ 2.5 Ghz (Python)
33 IE-PointRCNN 87.43 % 92.11 % 81.10 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
34 PC-CNN-V2
This method makes use of Velodyne laser scans.
87.40 % 91.19 % 79.35 % 0.5 s GPU @ 2.5 Ghz (Matlab + C/C++)
X. Du, M. Ang, S. Karaman and D. Rus: A General Pipeline for 3D Detection of Vehicles. 2018 IEEE International Conference on Robotics and Automation (ICRA) 2018.
35 MMLab-PointRCNN
This method makes use of Velodyne laser scans.
code 87.39 % 92.13 % 82.72 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: Pointrcnn: 3d object proposal generation and detection from point cloud. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019.
36 MPNet
This method makes use of Velodyne laser scans.
87.31 % 91.27 % 83.25 % 0.02 s GPU @ 2.5 Ghz (Python + C/C++)
37 deprecated 87.28 % 90.44 % 75.09 % 0.05 s GPU @ 2.0 Ghz (Python)
38 PiP 87.25 % 90.87 % 83.38 % 0.05 s 1 core @ 2.5 Ghz (Python)
39 HRI-VoxelFPN 87.21 % 92.75 % 79.82 % 0.02 s GPU @ 2.5 Ghz (Python + C/C++)
B. Wang, J. An and J. Cao: Voxel-FPN: multi-scale voxel feature aggregation in 3D object detection from point clouds. arXiv preprint arXiv:1907.05286v2 2019.
40 CentrNet-v1
This method makes use of Velodyne laser scans.
87.19 % 90.72 % 83.34 % 0.03 s GPU @ 2.5 Ghz (Python)
41 epBRM
This method makes use of Velodyne laser scans.
code 87.13 % 90.70 % 81.92 % 0.1 s GPU @ >3.5 Ghz (Python + C/C++)
K. Shin: Improving a Quality of 3D Object Detection by Spatial Transformation Mechanism. arXiv preprint arXiv:1910.04853 2019.
42 DH-ARI 87.13 % 90.26 % 80.52 % 0.2 s 1 core @ >3.5 Ghz (Python + C/C++)
43 Roadstar.ai 87.12 % 92.42 % 81.88 % 0.08 s GPU @ 2.0 Ghz (Python)
44 Alibaba-AILabsX
This method makes use of Velodyne laser scans.
87.00 % 92.50 % 79.61 % 0.05 s 1 core @ >3.5 Ghz (C/C++)
45 PCSC-Net 86.94 % 90.44 % 82.32 % 0.04 s 1 core @ 2.5 Ghz (C/C++)
46 SARPNET 86.92 % 92.21 % 81.68 % 0.05 s 1 core @ 2.5 Ghz (Python + C/C++)
Y. Ye, H. Chen, C. Zhang, X. Hao and Z. Zhang: SARPNET: Shape Attention Regional Proposal Network for LiDAR-based 3D Object Detection. Neurocomputing 2019.
47 TBA 86.85 % 90.51 % 83.05 % 0.07 s 1 core @ 2.5 Ghz (Python)
48 ARPNET 86.81 % 90.06 % 79.41 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Y. Ye, C. Zhang and X. Hao: ARPNET: attention region proposal network for 3D object detection. Science China Information Sciences 2019.
49 Tencent_ADlab_Lidar
This method makes use of Velodyne laser scans.
86.72 % 90.27 % 81.35 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
50 SRF 86.60 % 91.90 % 81.43 % 0.05 s GPU @ 2.5 Ghz (Python + C/C++)
51 PointPillars
This method makes use of Velodyne laser scans.
code 86.56 % 90.07 % 82.81 % 16 ms 1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
52 Alibaba-AILabsX
This method makes use of Velodyne laser scans.
86.55 % 91.88 % 79.23 % 0.2 s GPU @ >3.5 Ghz (Python)
53 TANet 86.54 % 91.58 % 81.19 % 0.035s GPU @ 2.5 Ghz (Python + C/C++)
54 A-VoxelNet 86.53 % 89.94 % 79.08 % 0.029 s GPU @ 2.5 Ghz (Python)
55 PointRCNN-deprecated
This method makes use of Velodyne laser scans.
86.52 % 92.51 % 81.39 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
56 SCNet
This method makes use of Velodyne laser scans.
86.48 % 90.07 % 81.30 % 0.04 s GPU @ 3.0 Ghz (Python)
Z. Wang, H. Fu, L. Wang, L. Xiao and B. Dai: SCNet: Subdivision Coding Network for Object Detection Based on 3D Point Cloud. IEEE Access 2019.
57 MMV 86.46 % 90.04 % 79.04 % 0.4 s GPU @ 2.5 Ghz (C/C++)
58 RUC 86.46 % 90.06 % 82.20 % 0.12 s 1 core @ 2.5 Ghz (C/C++)
59 DDB
This method makes use of Velodyne laser scans.
86.45 % 89.91 % 82.21 % 0.05 s GPU @ 2.5 Ghz (Python)
60 PPFNet code 86.44 % 92.35 % 81.48 % 0.1 s 1 core @ 2.5 Ghz (Python + C/C++)
61 HR-SECOND code 86.40 % 91.68 % 81.40 % 0.11 s 1 core @ 2.5 Ghz (Python + C/C++)
62 SECOND-V1.5
This method makes use of Velodyne laser scans.
code 86.37 % 91.81 % 81.04 % 0.04 s GPU @ 2.0 Ghz (Python + C/C++)
63 SegVoxelNet 86.37 % 91.62 % 83.04 % 0.04 s 1 core @ 2.5 Ghz (Python)
64 VOXEL_FPN_HR 86.36 % 90.28 % 81.20 % 0.12 s 8 cores @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
65 CP
This method makes use of Velodyne laser scans.
86.30 % 92.14 % 82.97 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
66 FOFNet
This method makes use of Velodyne laser scans.
86.22 % 90.09 % 78.96 % 0.04 s GPU @ 2.5 Ghz (Python)
67 NU-optim 86.22 % 91.62 % 80.81 % 0.04 s GPU @ >3.5 Ghz (Python)
68 3D IoU Loss
This method makes use of Velodyne laser scans.
86.22 % 91.36 % 81.20 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
D. Zhou, J. Fang, X. Song, C. Guan, J. Yin, Y. Dai and R. Yang: IoU Loss for 2D/3D Object Detection. International Conference on 3D Vision (3DV) 2019.
69 MP 86.16 % 90.24 % 78.86 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
70 MVX-Net
This method makes use of Velodyne laser scans.
86.05 % 92.13 % 78.68 % 0.06 s GPU @ 3.0 Ghz (Python + C/C++)
71 UberATG-PIXOR++
This method makes use of Velodyne laser scans.
86.01 % 93.28 % 80.11 % 0.035 s GPU @ 2.5 Ghz (Python)
B. Yang, M. Liang and R. Urtasun: HDNET: Exploiting HD Maps for 3D Object Detection. 2nd Conference on Robot Learning (CoRL) 2018.
72 PTS
This method makes use of Velodyne laser scans.
code 85.95 % 91.42 % 80.81 % 0.01 s 1 core @ 2.5 Ghz (C/C++)
73 F-ConvNet
This method makes use of Velodyne laser scans.
85.84 % 91.51 % 76.11 % 0.47 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. IROS 2019.
74 PI-RCNN 85.81 % 91.44 % 81.00 % 0.1 s 1 core @ 2.5 Ghz (Python)
75 SAANet 85.69 % 91.72 % 78.77 % 0.10 s 1 core @ 2.5 Ghz (Python)
76 SFB-SECOND 85.63 % 91.38 % 78.60 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
77 Fast Point R-CNN
This method makes use of Velodyne laser scans.
85.61 % 90.76 % 79.99 % 0.06 s GPU @ 2.5 Ghz (Python + C/C++)
78 UberATG-ContFuse
This method makes use of Velodyne laser scans.
85.35 % 94.07 % 75.88 % 0.06 s GPU @ 2.5 Ghz (Python)
M. Liang, B. Yang, S. Wang and R. Urtasun: Deep Continuous Fusion for Multi-Sensor 3D Object Detection. ECCV 2018.
79 MDC
This method makes use of Velodyne laser scans.
85.29 % 91.63 % 75.54 % 0.17 s GPU @ 2.5 Ghz (Python)
80 SeoulRobotics-HFD
This method makes use of Velodyne laser scans.
85.10 % 88.65 % 78.22 % 0.035 s GPU (C++)
81 PFPN 85.02 % 90.68 % 77.47 % 0.02 s 4 cores @ >3.5 Ghz (Python)
82 AVOD
This method makes use of Velodyne laser scans.
code 84.95 % 89.75 % 78.32 % 0.08 s Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
83 CONV-BOX
This method makes use of Velodyne laser scans.
84.91 % 90.58 % 80.24 % 0.2 s Tesla V100
84 AVOD-FPN
This method makes use of Velodyne laser scans.
code 84.82 % 90.99 % 79.62 % 0.1 s Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
85 PP_v1.0 code 84.69 % 88.44 % 80.19 % 0.02s 1 core @ 2.5 Ghz (C/C++)
86 F-PointNet
This method makes use of Velodyne laser scans.
code 84.67 % 91.17 % 74.77 % 0.17 s GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
87 IPOD 84.62 % 89.64 % 79.96 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: IPOD: Intensive Point-based Object Detector for Point Cloud. CoRR 2018.
88 PAD 84.46 % 88.66 % 80.61 % 0.15 s 1 core @ 2.5 Ghz (Python)
89 CFR
This method makes use of Velodyne laser scans.
84.30 % 90.25 % 76.80 % 0.06 s 1 core @ 2.5 Ghz (C/C++)
90 MVSLN 84.26 % 90.30 % 78.94 % 0.1s s 1 core @ 2.5 Ghz (C/C++)
91 3DBN
This method makes use of Velodyne laser scans.
83.94 % 89.66 % 76.50 % 0.13s 1080Ti (Python+C/C++)
X. Li, J. Guivant, N. Kwok and Y. Xu: 3D Backbone Network for 3D Object Detection. CoRR 2019.
92 RADNet-Fusion
This method makes use of Velodyne laser scans.
83.84 % 91.81 % 78.80 % 0.1 s 1 core @ 2.5 Ghz (Python)
93 SECOND code 83.77 % 89.39 % 78.59 % 38 ms 1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
94 RADNet-LIDAR
This method makes use of Velodyne laser scans.
83.74 % 92.43 % 77.00 % 0.1 s 1 core @ 2.5 Ghz (Python)
95 3DNN 83.68 % 88.06 % 77.00 % 0.09 s GPU @ 2.5 Ghz (Python)
96 AILabs3D
This method makes use of Velodyne laser scans.
83.57 % 91.46 % 76.05 % 0.6 s GPU @ >3.5 Ghz (Python)
97 DFD 83.20 % 88.56 % 77.84 % 0.05 s GPU @ 2.0 Ghz (Python + C/C++)
98 SCANet 83.19 % 88.68 % 77.84 % 0.17 s >8 cores @ 2.5 Ghz (Python)
99 ELLIOT
This method makes use of Velodyne laser scans.
83.03 % 88.29 % 78.56 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
100 SCANet 82.85 % 90.33 % 76.06 % 0.09s GPU @ 2.5 Ghz (Python)
101 FailNet-Fusion
This method makes use of Velodyne laser scans.
82.78 % 93.20 % 75.84 % 0.1 s 1 core @ 2.5 Ghz (Python)
102 SECA 82.75 % 90.60 % 75.93 % 0.09 s GPU @ 2.5 Ghz (Python)
103 yl_net 82.70 % 87.27 % 80.23 % 0.03 s GPU @ 2.5 Ghz (Python)
104 MLOD
This method makes use of Velodyne laser scans.
code 82.68 % 90.25 % 77.97 % 0.12 s GPU @ 1.5 Ghz (Python)
J. Deng and K. Czarnecki: MLOD: A multi-view 3D object detection based on robust feature fusion method. arXiv preprint arXiv:1909.04163 2019.
105 SECA 82.58 % 90.37 % 75.75 % 1 s GPU @ 2.5 Ghz (Python)
106 VSE 82.58 % 90.37 % 75.75 % 0.15 s GPU @ 2.5 Ghz (Python)
107 FailNet-LIDAR
This method makes use of Velodyne laser scans.
82.41 % 92.85 % 75.39 % 0.1 s 1 core @ 2.5 Ghz (Python)
108 tiny_rfdet code 81.93 % 86.57 % 75.83 % 0.01 s GPU @ 2.5 Ghz (Python)
109 NLK 81.93 % 89.93 % 76.80 % 0.02 s 1 core @ 2.5 Ghz (Python)
110 RTL3D 81.63 % 89.55 % 76.63 % 0.02 s GPU @ 2.5 Ghz (Python)
111 PointRes
This method makes use of Velodyne laser scans.
This method makes use of GPS/IMU information.
This is an online method (no batch processing).
81.60 % 90.60 % 76.03 % 0.013 s 1 core @ 2.5 Ghz (Python + C/C++)
112 FNV1_RPN 80.85 % 90.39 % 73.88 % 0.12 s 1 core @ 2.5 Ghz (Python + C/C++)
113 FNV1_Fusion 80.41 % 88.48 % 75.33 % 0.11 s GPU @ 2.5 Ghz (Python)
114 X_MD 80.32 % 90.26 % 73.54 % 0.2 s 1 core @ 2.5 Ghz (Python + C/C++)
115 RuiRUC 80.20 % 86.90 % 67.77 % 0.12 s 1 core @ 2.5 Ghz (Python)
116 UberATG-PIXOR
This method makes use of Velodyne laser scans.
80.01 % 83.97 % 74.31 % 0.035 s TITAN Xp (Python)
B. Yang, W. Luo and R. Urtasun: PIXOR: Real-time 3D Object Detection from Point Clouds. CVPR 2018.
117 FNV1 79.62 % 87.37 % 72.57 % 0.11 s GPU @ 2.5 Ghz (Python)
118 MV3D (LIDAR)
This method makes use of Velodyne laser scans.
78.98 % 86.49 % 72.23 % 0.24 s GPU @ 2.5 Ghz (Python + C/C++)
X. Chen, H. Ma, J. Wan, B. Li and T. Xia: Multi-View 3D Object Detection Network for Autonomous Driving. CVPR 2017.
119 MV3D
This method makes use of Velodyne laser scans.
78.93 % 86.62 % 69.80 % 0.36 s GPU @ 2.5 Ghz (Python + C/C++)
X. Chen, H. Ma, J. Wan, B. Li and T. Xia: Multi-View 3D Object Detection Network for Autonomous Driving. CVPR 2017.
120 Multi-3D
This method makes use of Velodyne laser scans.
78.45 % 85.99 % 67.14 % 0.15 s 1 core @ 2.5 Ghz (C/C++)
121 VoxelNet(Unofficial) 78.39 % 87.95 % 71.29 % 0.5 s GPU @ 2.0 Ghz (Python)
122 FNV2 76.69 % 82.57 % 65.60 % 0.18 s GPU @ 2.5 Ghz (Python)
123 ANM 75.40 % 84.78 % 61.28 % 0.12 s 1 core @ 2.5 Ghz (Python)
124 LaserNet 74.52 % 79.19 % 68.45 % 12 ms GPU @ 2.5 Ghz (C/C++)
G. Meyer, A. Laddha, E. Kee, C. Vallespi-Gonzalez and C. Wellington: LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
125 PL V2 (SDN+GDC)
This method uses stereo information.
This method makes use of Velodyne laser scans.
73.80 % 84.61 % 65.59 % 0.6 s GPU @ 2.5 Ghz (C/C++)
126 anm 73.63 % 82.59 % 62.87 % 3 s 1 core @ 2.5 Ghz (C/C++)
127 A3DODWTDA
This method makes use of Velodyne laser scans.
code 73.26 % 79.58 % 62.77 % 0.08 s GPU @ 3.0 Ghz (Python)
F. Gustafsson and E. Linder-Norén: Automotive 3D Object Detection Without Target Domain Annotations. 2018.
128 CLF3D
This method makes use of Velodyne laser scans.
73.13 % 80.84 % 60.64 % 0.13 s GPU @ 2.5 Ghz (Python)
129 avodC 72.78 % 84.61 % 66.02 % 0.1 s GPU @ 2.5 Ghz (Python)
130 E-VoxelNet 69.69 % 81.10 % 60.88 % 0.1 s GPU @ 2.5 Ghz (Python)
131 Complexer-YOLO
This method makes use of Velodyne laser scans.
68.96 % 77.24 % 64.95 % 0.06 s GPU @ 3.5 Ghz (C/C++)
M. Simon, K. Amende, A. Kraus, J. Honer, T. Samann, H. Kaulbersch, S. Milz and H. Michael Gross: Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2019.
132 TopNet-Retina
This method makes use of Velodyne laser scans.
68.16 % 80.16 % 63.43 % 52ms GeForce 1080Ti (tensorflow-gpu, v1.12)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
133 TopNet-DecayRate
This method makes use of Velodyne laser scans.
64.60 % 79.74 % 58.04 % 92 ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
134 3D FCN
This method makes use of Velodyne laser scans.
61.67 % 70.62 % 55.61 % >5 s 1 core @ 2.5 Ghz (C/C++)
B. Li: 3D Fully Convolutional Network for Vehicle Detection in Point Cloud. IROS 2017.
135 TopNet-UncEst
This method makes use of Velodyne laser scans.
59.67 % 72.05 % 51.67 % 0.09 s NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
136 Pseudo-LiDAR V2
This method uses stereo information.
code 58.01 % 78.31 % 51.25 % 0.4 s GPU @ 2.5 Ghz (Python)
137 ZoomNet
This method uses stereo information.
54.91 % 72.94 % 44.14 % 0.3 s 1 core @ 2.5 Ghz (C/C++)
138 VoxelJones code 53.96 % 66.21 % 47.66 % .18 s 1 core @ 2.5 Ghz (Python + C/C++)
M. Motro and J. Ghosh: Vehicular Multi-object Tracking with Persistent Detector Failures. arXiv preprint arXiv:1907.11306 2019.
139 TopNet-HighRes
This method makes use of Velodyne laser scans.
53.05 % 67.84 % 46.99 % 101ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
140 Disp R-CNN (velo)
This method uses stereo information.
52.34 % 74.07 % 43.77 % 0.42 s GPU @ 2.5 Ghz (Python + C/C++)
141 Disp R-CNN
This method uses stereo information.
52.34 % 73.82 % 43.64 % 0.42 s GPU @ 2.5 Ghz (Python + C/C++)
142 BirdNet
This method makes use of Velodyne laser scans.
51.51 % 76.88 % 50.27 % 0.11 s Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
143 OC Stereo
This method uses stereo information.
51.47 % 68.89 % 42.97 % 0.35 s 1 core @ 2.5 Ghz (Python + C/C++)
144 stereo_sa
This method uses stereo information.
49.61 % 71.47 % 42.71 % 0.3 s GPU @ 2.5 Ghz (Python)
145 SA_3D 47.52 % 59.69 % 38.70 % 0.3 s GPU @ 2.5 Ghz (Python)
146 RT3DStereo
This method uses stereo information.
46.82 % 58.81 % 38.38 % 0.08 s GPU @ 2.5 Ghz (C/C++)
H. Königshof, N. Salscheider and C. Stiller: Realtime 3D Object Detection for Automated Driving Using Stereo Vision and Semantic Information. Proc. IEEE Intl. Conf. Intelligent Transportation Systems 2019.
147 Pseudo-LiDAR
This method uses stereo information.
code 45.00 % 67.30 % 38.40 % 0.4 s GPU @ 2.5 Ghz (C/C++)
Y. Wang, W. Chao, D. Garg, B. Hariharan, M. Campbell and K. Weinberger: Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving. CVPR 2019.
148 RT3D
This method makes use of Velodyne laser scans.
44.00 % 56.44 % 42.34 % 0.09 s GPU @ 1.8Ghz
Y. Zeng, Y. Hu, S. Liu, J. Ye, Y. Han, X. Li and N. Sun: RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving. IEEE Robotics and Automation Letters 2018.
149 m-prcnn
This method uses stereo information.
42.81 % 67.82 % 33.63 % 0.43 s 1 core @ 2.5 Ghz (Python)
150 42.22 % 64.44 % 35.61 %
151 Stereo R-CNN
This method uses stereo information.
code 41.31 % 61.92 % 33.42 % 0.3 s GPU @ 2.5 Ghz (Python)
P. Li, X. Chen and S. Shen: Stereo R-CNN based 3D Object Detection for Autonomous Driving. CVPR 2019.
152 Licar
This method makes use of Velodyne laser scans.
38.47 % 46.67 % 35.78 % 0.09 s GPU @ 2.0 Ghz (Python)
153 DLMB
This method makes use of Velodyne laser scans.
This is an online method (no batch processing).
34.53 % 45.90 % 31.83 % 0.03 s 8 cores @ 3.5 Ghz (C/C++)
154 StereoFENet
This method uses stereo information.
32.96 % 49.29 % 25.90 % 0.15 s 1 core @ 3.5 Ghz (Python)
W. Bao, B. Xu and Z. Chen: MonoFENet: Monocular 3D Object Detection with Feature Enhancement Networks. 2019.
155 RefinedMPL 17.60 % 28.08 % 13.95 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
156 AM3D 17.32 % 25.03 % 14.91 % 0.4 s GPU @ 2.5 Ghz (Python + C/C++)
X. Ma, Z. Wang, H. Li, P. Zhang, W. Ouyang and X. Fan: Accurate Monocular Object Detection via Color- Embedded 3D Reconstruction for Autonomous Driving. Proceedings of the IEEE international Conference on Computer Vision (ICCV) 2019.
157 D^4LCN 16.02 % 22.51 % 12.55 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
158 MonoPair 14.83 % 19.28 % 12.89 % 0.06 s GPU @ 2.5 Ghz (Python + C/C++)
159 Decoupled-3D 14.82 % 23.16 % 11.25 % 0.08 s GPU @ 2.5 Ghz (C/C++)
160 Decoupled-3D v2 14.66 % 24.62 % 11.46 % 0.08 s GPU @ 2.5 Ghz (C/C++)
161 DT3D 14.57 % 22.52 % 12.76 % 0,21s GPU @ 2.5 Ghz (Python)
162 MonoSS 14.52 % 20.91 % 12.63 % 0.03 s GPU @ 2.5 Ghz (Python + C/C++)
163 SMOKE 14.49 % 20.83 % 12.75 % 0.03 s GPU @ 2.5 Ghz (Python)
164 Mono3D_PLiDAR code 13.92 % 21.27 % 11.25 % 0.1 s NVIDIA GeForce 1080 (pytorch)
X. Weng and K. Kitani: Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud. arXiv:1903.09847 2019.
165 M3D-RPN code 13.67 % 21.02 % 10.23 % 0.16 s GPU @ 1.5 Ghz (Python)
G. Brazil and X. Liu: M3D-RPN: Monocular 3D Region Proposal Network for Object Detection . ICCV 2019 .
166 RAR-Net 13.55 % 20.70 % 10.13 % 0.5 s 1 core @ 2.5 Ghz (C/C++)
167 MonoDIS 13.19 % 17.23 % 11.12 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
168 CSoR
This method makes use of Velodyne laser scans.
13.07 % 18.67 % 10.34 % 3.5 s 4 cores @ >3.5 Ghz (Python + C/C++)
L. Plotkin: PyDriver: Entwicklung eines Frameworks für räumliche Detektion und Klassifikation von Objekten in Fahrzeugumgebung. 2015.
169 MonoPSR code 12.58 % 18.33 % 9.91 % 0.2 s GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
170 PG-MonoNet 12.45 % 19.79 % 9.68 % 0.19 s GPU @ 2.5 Ghz (Python)
171 SS3D 11.52 % 16.33 % 9.93 % 48 ms Tesla V100 (Python)
E. Jörgensen, C. Zach and F. Kahl: Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss. CoRR 2019.
172 MonoGRNet code 11.17 % 18.19 % 8.73 % 0.04s NVIDIA P40
Z. Qin, J. Wang and Y. Lu: MonoGRNet: A Geometric Reasoning Network for 3D Object Localization. The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) 2019.
173 MonoFENet 11.03 % 17.03 % 9.05 % 0.15 s 1 core @ 3.5 Ghz (Python)
W. Bao, B. Xu and Z. Chen: MonoFENet: Monocular 3D Object Detection with Feature Enhancement Networks. 2019.
174 RADNet-Mono 10.57 % 15.22 % 8.74 % 0.1 s 1 core @ 2.5 Ghz (Python)
175 OACV 10.13 % 16.24 % 8.28 % 0.23 s GPU @ 2.5 Ghz (Python)
176 mylsi-faster-rcnn 9.96 % 15.55 % 8.11 % 0.3 s 1 core @ 2.5 Ghz (Python)
177 FailNet-Mono 9.11 % 14.41 % 7.11 % 0.1 s 1 core @ 2.5 Ghz (Python)
178 A3DODWTDA (image) code 8.66 % 10.37 % 7.06 % 0.8 s GPU @ 3.0 Ghz (Python)
F. Gustafsson and E. Linder-Norén: Automotive 3D Object Detection Without Target Domain Annotations. 2018.
179 mymask-rcnn 8.29 % 15.56 % 6.53 % 0.3 s 1 core @ 2.5 Ghz (Python)
180 Shift R-CNN (mono) code 6.82 % 11.84 % 5.27 % 0.25 s GPU @ 1.5 Ghz (Python)
A. Naiden, V. Paunescu, G. Kim, B. Jeon and M. Leordeanu: Shift R-CNN: Deep Monocular 3D Object Detection With Closed-form Geometric Constraints. ICIP 2019.
181 GS3D 6.08 % 8.41 % 4.94 % 2 s 1 core @ 2.5 Ghz (C/C++)
B. Li, W. Ouyang, L. Sheng, X. Zeng and X. Wang: GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019.
182 MVRA + I-FRCNN+ 5.84 % 9.05 % 4.50 % 0.18 s GPU @ 2.5 Ghz (Python)
183 OFT-Net 5.69 % 7.16 % 4.61 % 0.5 s 8 cores @ 2.5 Ghz (Python + C/C++)
184 ROI-10D 4.91 % 9.78 % 3.74 % 0.2 s GPU @ 3.5 Ghz (Python)
F. Manhardt, W. Kehl and A. Gaidon: ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape. Computer Vision and Pattern Recognition (CVPR) 2019.
185 MF3D 3.49 % 6.39 % 2.69 % 0.03 s GPU @ 2.5 Ghz (C/C++)
186 FQNet 3.23 % 5.40 % 2.46 % 0.5 s 1 core @ 2.5 Ghz (Python)
L. Liu, J. Lu, C. Xu, Q. Tian and J. Zhou: Deep Fitting Degree Scoring Network for Monocular 3D Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019.
187 3D-SSMFCNN code 2.63 % 3.20 % 2.40 % 0.1 s GPU @ 1.5 Ghz (C/C++)
L. Novak: Vehicle Detection and Pose Estimation for Autonomous Driving. 2017.
188 monoref3d 1.70 % 3.22 % 1.21 % 0.1 s 1 core @ 2.5 Ghz (Python)
189 ref3D 1.70 % 3.22 % 1.21 % 0.1 s 1 core @ 2.5 Ghz (Python + C/C++)
190 3DVSSD 1.31 % 1.74 % 1.08 % 0.06 s 1 core @ 2.5 Ghz (C/C++)
191 VeloFCN
This method makes use of Velodyne laser scans.
0.14 % 0.02 % 0.21 % 1 s GPU @ 2.5 Ghz (Python + C/C++)
B. Li, T. Zhang and T. Xia: Vehicle Detection from 3D Lidar Using Fully Convolutional Network. RSS 2016 .
192 ANM 0.00 % 0.00 % 0.00 % 0.12 s 1 core @ 2.5 Ghz (C/C++)
193 ref3D 0.00 % 0.00 % 0.00 % 0.1 s 1 core @ 2.5 Ghz (Python)
194 multi-task CNN 0.00 % 0.00 % 0.00 % 25.1 ms GPU @ 2.0 Ghz (Python)
M. Oeljeklaus, F. Hoffmann and T. Bertram: A Fast Multi-Task CNN for Spatial Understanding of Traffic Scenes. IEEE Intelligent Transportation Systems Conference 2018.
195 mBoW
This method makes use of Velodyne laser scans.
0.00 % 0.00 % 0.00 % 10 s 1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.
Table as LaTeX | Only published Methods

Pedestrian


Method Setting Code Moderate Easy Hard Runtime Environment
1 DENFIDet 51.96 % 61.15 % 49.03 % 0.02 s GPU @ 2.5 Ghz (C/C++)
2 A-VoxelNet 51.79 % 61.34 % 47.93 % 0.029 s GPU @ 2.5 Ghz (Python)
3 TANet 51.38 % 60.85 % 47.54 % 0.035s GPU @ 2.5 Ghz (Python + C/C++)
4 VMVS
This method makes use of Velodyne laser scans.
50.34 % 60.34 % 46.45 % 0.25 s GPU @ 2.5 Ghz (Python)
J. Ku, A. Pon, S. Walsh and S. Waslander: Improving 3D object detection for pedestrians with virtual multi-view synthesis orientation estimation. IROS 2019.
5 AVOD-FPN
This method makes use of Velodyne laser scans.
code 50.32 % 58.49 % 46.98 % 0.1 s Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
6 IPOD 49.79 % 60.88 % 45.43 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: IPOD: Intensive Point-based Object Detector for Point Cloud. CoRR 2018.
7 F-PointNet
This method makes use of Velodyne laser scans.
code 49.57 % 57.13 % 45.48 % 0.17 s GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
8 F-ConvNet
This method makes use of Velodyne laser scans.
48.96 % 57.04 % 44.33 % 0.47 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. IROS 2019.
9 HVNet 48.86 % 54.84 % 46.33 % 0.03 s GPU @ 2.0 Ghz (Python)
10 CentrNet-v1
This method makes use of Velodyne laser scans.
48.78 % 57.58 % 45.94 % 0.03 s GPU @ 2.5 Ghz (Python)
11 STD 48.72 % 60.02 % 44.55 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: STD: Sparse-to-Dense 3D Object Detector for Point Cloud. ICCV 2019.
12 PointPillars
This method makes use of Velodyne laser scans.
code 48.64 % 57.60 % 45.78 % 16 ms 1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
13 DDB
This method makes use of Velodyne laser scans.
48.35 % 57.68 % 45.44 % 0.05 s GPU @ 2.5 Ghz (Python)
14 PiP 48.14 % 56.16 % 45.27 % 0.05 s 1 core @ 2.5 Ghz (Python)
15 Noah CV Lab - SSL 47.94 % 54.74 % 43.78 % 0.1 s GPU @ 2.5 Ghz (Python)
16 PPFNet code 47.92 % 55.04 % 44.95 % 0.1 s 1 core @ 2.5 Ghz (Python + C/C++)
17 LDAM 47.35 % 52.08 % 45.23 % 0.05 s GPU @ 2.5 Ghz (C/C++)
18 Tencent_ADlab_Lidar
This method makes use of Velodyne laser scans.
47.24 % 56.06 % 44.61 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
19 Point-GNN
This method makes use of Velodyne laser scans.
47.07 % 55.36 % 44.61 % 0.6 s GPU @ 2.5 Ghz (Python)
20 SCNet
This method makes use of Velodyne laser scans.
46.73 % 56.87 % 42.74 % 0.04 s GPU @ 3.0 Ghz (Python)
Z. Wang, H. Fu, L. Wang, L. Xiao and B. Dai: SCNet: Subdivision Coding Network for Object Detection Based on 3D Point Cloud. IEEE Access 2019.
21 MMLab-PointRCNN
This method makes use of Velodyne laser scans.
code 46.13 % 54.77 % 42.84 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: Pointrcnn: 3d object proposal generation and detection from point cloud. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019.
22 Multi-3D
This method makes use of Velodyne laser scans.
46.09 % 54.37 % 41.42 % 0.15 s 1 core @ 2.5 Ghz (C/C++)
23 ARPNET 45.92 % 55.48 % 42.54 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Y. Ye, C. Zhang and X. Hao: ARPNET: attention region proposal network for 3D object detection. Science China Information Sciences 2019.
24 PP_v1.0 code 45.73 % 53.93 % 43.05 % 0.02s 1 core @ 2.5 Ghz (C/C++)
25 epBRM
This method makes use of Velodyne laser scans.
code 45.49 % 52.48 % 42.75 % 0.10 s 1 core @ 2.5 Ghz (C/C++)
K. Shin: Improving a Quality of 3D Object Detection by Spatial Transformation Mechanism. arXiv preprint arXiv:1910.04853 2019.
26 MLOD
This method makes use of Velodyne laser scans.
code 45.40 % 55.09 % 41.42 % 0.12 s GPU @ 1.5 Ghz (Python)
J. Deng and K. Czarnecki: MLOD: A multi-view 3D object detection based on robust feature fusion method. arXiv preprint arXiv:1909.04163 2019.
27 MDC
This method makes use of Velodyne laser scans.
45.23 % 54.48 % 41.11 % 0.17 s GPU @ 2.5 Ghz (Python)
28 SECOND code 45.02 % 55.99 % 40.93 % 38 ms 1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
29 CONV-BOX
This method makes use of Velodyne laser scans.
44.84 % 52.98 % 42.30 % 0.2 s Tesla V100
30 OHS 44.59 % 50.87 % 42.14 % 0.03 s 1 core @ 2.5 Ghz (Python/C++)
31 GPOD
This method makes use of Velodyne laser scans.
44.55 % 52.13 % 42.12 % 0.1 s GPU @ 2.5 Ghz (Python)
32 Roadstar.ai 44.35 % 49.63 % 41.39 % 0.08 s GPU @ 2.0 Ghz (Python)
33 SCANet 42.81 % 53.84 % 38.94 % 0.17 s >8 cores @ 2.5 Ghz (Python)
34 FOFNet
This method makes use of Velodyne laser scans.
42.31 % 51.39 % 38.81 % 0.04 s GPU @ 2.5 Ghz (Python)
35 VOXEL_FPN_HR 41.62 % 50.18 % 38.30 % 0.12 s 8 cores @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
36 deprecated 41.32 % 53.09 % 37.16 % 0.05 s GPU @ 2.0 Ghz (Python)
37 ELLIOT
This method makes use of Velodyne laser scans.
40.22 % 49.57 % 37.38 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
38 HR-SECOND code 40.06 % 50.05 % 36.47 % 0.11 s 1 core @ 2.5 Ghz (Python + C/C++)
39 AB3DMOT
This method makes use of Velodyne laser scans.
This is an online method (no batch processing).
code 38.79 % 47.51 % 35.85 % 0.0047s 1 core @ 2.5 Ghz (python)
X. Weng and K. Kitani: A Baseline for 3D Multi-Object Tracking. arXiv:1907.03961 2019.
40 MP 38.77 % 47.59 % 35.50 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
41 CFR
This method makes use of Velodyne laser scans.
38.74 % 50.64 % 36.23 % 0.06 s 1 core @ 2.5 Ghz (C/C++)
42 anm 38.01 % 49.07 % 34.00 % 3 s 1 core @ 2.5 Ghz (C/C++)
43 CSW3D
This method makes use of Velodyne laser scans.
37.96 % 49.27 % 33.83 % 0.03 s 4 cores @ 2.5 Ghz (C/C++)
J. Hu, T. Wu, H. Fu, Z. Wang and K. Ding: Cascaded Sliding Window Based Real-Time 3D Region Proposal for Pedestrian Detection. ROBIO 2019.
44 SA_3D 34.34 % 44.65 % 30.78 % 0.3 s GPU @ 2.5 Ghz (Python)
45 SparsePool code 34.15 % 43.33 % 31.78 % 0.13 s 8 cores @ 2.5 Ghz (Python)
Z. Wang, W. Zhan and M. Tomizuka: Fusing bird view lidar point cloud and front view camera image for deep object detection. arXiv preprint arXiv:1711.06703 2017.
46 SAANet 33.94 % 42.34 % 31.75 % 0.10 s 1 core @ 2.5 Ghz (Python)
47 AVOD
This method makes use of Velodyne laser scans.
code 33.57 % 42.58 % 30.14 % 0.08 s Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
48 anonymous
This method makes use of Velodyne laser scans.
33.47 % 42.38 % 29.97 % 0.75 s GPU @ 3.5 Ghz (C/C++)
49 SparsePool code 33.22 % 41.55 % 29.66 % 0.13 s 8 cores @ 2.5 Ghz (Python)
Z. Wang, W. Zhan and M. Tomizuka: Fusing bird view lidar point cloud and front view camera image for deep object detection. arXiv preprint arXiv:1711.06703 2017.
50 X_MD 32.57 % 42.18 % 30.23 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
51 32.32 % 40.87 % 29.52 %
52 CLF3D
This method makes use of Velodyne laser scans.
31.31 % 40.72 % 27.80 % 0.13 s GPU @ 2.5 Ghz (Python)
53 OC Stereo
This method uses stereo information.
20.80 % 29.79 % 18.62 % 0.35 s 1 core @ 2.5 Ghz (Python + C/C++)
54 Complexer-YOLO
This method makes use of Velodyne laser scans.
18.26 % 21.42 % 17.06 % 0.06 s GPU @ 3.5 Ghz (C/C++)
M. Simon, K. Amende, A. Kraus, J. Honer, T. Samann, H. Kaulbersch, S. Milz and H. Michael Gross: Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2019.
55 BirdNet
This method makes use of Velodyne laser scans.
15.80 % 20.73 % 14.59 % 0.11 s Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
56 TopNet-Retina
This method makes use of Velodyne laser scans.
14.57 % 18.04 % 12.48 % 52ms GeForce 1080Ti (tensorflow-gpu, v1.12)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
57 TopNet-HighRes
This method makes use of Velodyne laser scans.
13.50 % 19.43 % 11.93 % 101ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
58 RefinedMPL 7.75 % 12.64 % 6.88 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
59 MonoPair 7.04 % 10.99 % 6.29 % 0.06 s GPU @ 2.5 Ghz (Python + C/C++)
60 TopNet-DecayRate
This method makes use of Velodyne laser scans.
6.59 % 8.78 % 6.25 % 92 ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
61 Shift R-CNN (mono) code 5.66 % 8.58 % 4.49 % 0.25 s GPU @ 1.5 Ghz (Python)
A. Naiden, V. Paunescu, G. Kim, B. Jeon and M. Leordeanu: Shift R-CNN: Deep Monocular 3D Object Detection With Closed-form Geometric Constraints. ICIP 2019.
62 TopNet-UncEst
This method makes use of Velodyne laser scans.
4.60 % 6.88 % 3.79 % 0.09 s NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
63 MonoPSR code 4.56 % 7.24 % 4.11 % 0.2 s GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
64 M3D-RPN code 4.05 % 5.65 % 3.29 % 0.16 s GPU @ 1.5 Ghz (Python)
G. Brazil and X. Liu: M3D-RPN: Monocular 3D Region Proposal Network for Object Detection . ICCV 2019 .
65 D^4LCN 3.86 % 5.06 % 3.59 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
66 RT3DStereo
This method uses stereo information.
3.65 % 4.72 % 3.00 % 0.08 s GPU @ 2.5 Ghz (C/C++)
H. Königshof, N. Salscheider and C. Stiller: Realtime 3D Object Detection for Automated Driving Using Stereo Vision and Semantic Information. Proc. IEEE Intl. Conf. Intelligent Transportation Systems 2019.
67 PG-MonoNet 3.16 % 4.28 % 2.57 % 0.19 s GPU @ 2.5 Ghz (Python)
68 mylsi-faster-rcnn 2.11 % 3.08 % 1.89 % 0.3 s 1 core @ 2.5 Ghz (Python)
69 SS3D 2.09 % 2.48 % 1.61 % 48 ms Tesla V100 (Python)
E. Jörgensen, C. Zach and F. Kahl: Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss. CoRR 2019.
70 mymask-rcnn 1.60 % 2.60 % 1.48 % 0.3 s 1 core @ 2.5 Ghz (Python)
71 OFT-Net 0.81 % 1.28 % 0.51 % 0.5 s 8 cores @ 2.5 Ghz (Python + C/C++)
72 DT3D 0.51 % 0.75 % 0.48 % 0,21s GPU @ 2.5 Ghz (Python)
73 mBoW
This method makes use of Velodyne laser scans.
0.00 % 0.00 % 0.00 % 10 s 1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.
Table as LaTeX | Only published Methods

Cyclist


Method Setting Code Moderate Easy Hard Runtime Environment
1 Noah CV Lab - SSL 74.27 % 85.74 % 64.03 % 0.1 s GPU @ 2.5 Ghz (Python)
2 HVNet 71.17 % 83.97 % 63.65 % 0.03 s GPU @ 2.0 Ghz (Python)
3 PV-RCNN
This method makes use of Velodyne laser scans.
68.89 % 82.49 % 62.41 % 0.08 s 1 core @ 2.5 Ghz (Python + C/C++)
4 F-ConvNet
This method makes use of Velodyne laser scans.
68.88 % 84.16 % 60.05 % 0.47 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Wang and K. Jia: Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection. IROS 2019.
5 Point-GNN
This method makes use of Velodyne laser scans.
67.28 % 81.17 % 59.67 % 0.6 s GPU @ 2.5 Ghz (Python)
6 MMLab-PointRCNN
This method makes use of Velodyne laser scans.
code 67.24 % 82.56 % 60.28 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
S. Shi, X. Wang and H. Li: Pointrcnn: 3d object proposal generation and detection from point cloud. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2019.
7 STD 67.23 % 81.36 % 59.35 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: STD: Sparse-to-Dense 3D Object Detector for Point Cloud. ICCV 2019.
8 OHS 66.86 % 82.13 % 60.86 % 0.03 s 1 core @ 2.5 Ghz (Python/C++)
9 ARPNET 66.39 % 82.32 % 58.80 % 0.08 s GPU @ 2.5 Ghz (Python + C/C++)
Y. Ye, C. Zhang and X. Hao: ARPNET: attention region proposal network for 3D object detection. Science China Information Sciences 2019.
10 AB3DMOT
This method makes use of Velodyne laser scans.
This is an online method (no batch processing).
code 65.85 % 80.00 % 58.69 % 0.0047s 1 core @ 2.5 Ghz (Python)
X. Weng and K. Kitani: A Baseline for 3D Multi-Object Tracking. arXiv:1907.03961 2019.
11 DENFIDet 65.49 % 82.13 % 57.76 % 0.02 s GPU @ 2.5 Ghz (C/C++)
12 PiP 65.12 % 79.51 % 58.25 % 0.05 s 1 core @ 2.5 Ghz (Python)
13 FOFNet
This method makes use of Velodyne laser scans.
65.06 % 80.44 % 57.55 % 0.04 s GPU @ 2.5 Ghz (Python)
14 VOXEL_FPN_HR 65.02 % 81.07 % 58.44 % 0.12 s 8 cores @ 2.5 Ghz (Python)
ERROR: Wrong syntax in BIBTEX file.
15 HR-SECOND code 64.21 % 78.79 % 57.82 % 0.11 s 1 core @ 2.5 Ghz (Python + C/C++)
16 Multi-3D
This method makes use of Velodyne laser scans.
64.09 % 80.81 % 54.67 % 0.15 s 1 core @ 2.5 Ghz (C/C++)
17 TANet 63.77 % 79.16 % 56.21 % 0.035s GPU @ 2.5 Ghz (Python + C/C++)
18 LDAM 63.17 % 77.22 % 57.34 % 0.05 s GPU @ 2.5 Ghz (C/C++)
19 PointPillars
This method makes use of Velodyne laser scans.
code 62.73 % 79.90 % 55.58 % 16 ms 1080ti GPU and Intel i7 CPU
A. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang and O. Beijbom: PointPillars: Fast Encoders for Object Detection from Point Clouds. CVPR 2019.
20 MDC
This method makes use of Velodyne laser scans.
62.68 % 79.44 % 53.86 % 0.17 s GPU @ 2.5 Ghz (Python)
21 Tencent_ADlab_Lidar
This method makes use of Velodyne laser scans.
62.34 % 78.91 % 55.37 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
22 F-PointNet
This method makes use of Velodyne laser scans.
code 61.37 % 77.26 % 53.78 % 0.17 s GPU @ 3.0 Ghz (Python)
C. Qi, W. Liu, C. Wu, H. Su and L. Guibas: Frustum PointNets for 3D Object Detection from RGB-D Data. arXiv preprint arXiv:1711.08488 2017.
23 CONV-BOX
This method makes use of Velodyne laser scans.
61.01 % 71.70 % 54.69 % 0.2 s Tesla V100
24 A-VoxelNet 60.71 % 76.90 % 53.62 % 0.029 s GPU @ 2.5 Ghz (Python)
25 MP 60.16 % 77.57 % 54.01 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
26 GPOD
This method makes use of Velodyne laser scans.
60.03 % 69.36 % 54.39 % 0.1 s GPU @ 2.5 Ghz (Python)
27 epBRM
This method makes use of Velodyne laser scans.
code 59.79 % 75.13 % 53.36 % 0.10 s 1 core @ 2.5 Ghz (C/C++)
K. Shin: Improving a Quality of 3D Object Detection by Spatial Transformation Mechanism. arXiv preprint arXiv:1910.04853 2019.
28 Roadstar.ai 59.50 % 68.73 % 53.60 % 0.08 s GPU @ 2.0 Ghz (Python)
29 IPOD 59.40 % 78.19 % 51.38 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
Z. Yang, Y. Sun, S. Liu, X. Shen and J. Jia: IPOD: Intensive Point-based Object Detector for Point Cloud. CoRR 2018.
30 CentrNet-v1
This method makes use of Velodyne laser scans.
58.05 % 75.80 % 51.17 % 0.03 s GPU @ 2.5 Ghz (Python)
31 SAANet 57.98 % 74.71 % 51.95 % 0.10 s 1 core @ 2.5 Ghz (Python)
32 ELLIOT
This method makes use of Velodyne laser scans.
57.35 % 76.94 % 51.32 % 0.1 s 1 core @ 2.5 Ghz (C/C++)
33 SCANet 57.20 % 72.86 % 51.16 % 0.17 s >8 cores @ 2.5 Ghz (Python)
34 AVOD-FPN
This method makes use of Velodyne laser scans.
code 57.12 % 69.39 % 51.09 % 0.1 s Titan X (Pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
35 DDB
This method makes use of Velodyne laser scans.
57.01 % 73.70 % 50.71 % 0.05 s GPU @ 2.5 Ghz (Python)
36 deprecated 56.42 % 81.02 % 49.28 % 0.05 s GPU @ 2.0 Ghz (Python)
37 SCNet
This method makes use of Velodyne laser scans.
56.39 % 73.73 % 49.99 % 0.04 s GPU @ 3.0 Ghz (Python)
Z. Wang, H. Fu, L. Wang, L. Xiao and B. Dai: SCNet: Subdivision Coding Network for Object Detection Based on 3D Point Cloud. IEEE Access 2019.
38 SECOND code 56.05 % 76.50 % 49.45 % 38 ms 1080Ti
Y. Yan, Y. Mao and B. Li: SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018.
39 MLOD
This method makes use of Velodyne laser scans.
code 55.06 % 73.03 % 48.21 % 0.12 s GPU @ 1.5 Ghz (Python)
J. Deng and K. Czarnecki: MLOD: A multi-view 3D object detection based on robust feature fusion method. arXiv preprint arXiv:1909.04163 2019.
40 PP_v1.0 code 53.85 % 72.02 % 47.21 % 0.02s 1 core @ 2.5 Ghz (C/C++)
41 CFR
This method makes use of Velodyne laser scans.
52.78 % 68.47 % 45.55 % 0.06 s 1 core @ 2.5 Ghz (C/C++)
42 AVOD
This method makes use of Velodyne laser scans.
code 48.15 % 64.11 % 42.37 % 0.08 s Titan X (pascal)
J. Ku, M. Mozifian, J. Lee, A. Harakeh and S. Waslander: Joint 3D Proposal Generation and Object Detection from View Aggregation. IROS 2018.
43 X_MD 41.47 % 55.29 % 36.10 % 0.2 s 1 core @ 2.5 Ghz (C/C++)
44 SparsePool code 40.74 % 56.52 % 36.68 % 0.13 s 8 cores @ 2.5 Ghz (Python)
Z. Wang, W. Zhan and M. Tomizuka: Fusing bird view lidar point cloud and front view camera image for deep object detection. arXiv preprint arXiv:1711.06703 2017.
45 anm 38.56 % 56.94 % 34.06 % 3 s 1 core @ 2.5 Ghz (C/C++)
46 CLF3D
This method makes use of Velodyne laser scans.
37.40 % 53.98 % 32.06 % 0.13 s GPU @ 2.5 Ghz (Python)
47 TopNet-Retina
This method makes use of Velodyne laser scans.
36.83 % 47.48 % 33.58 % 52ms GeForce 1080Ti (tensorflow-gpu, v1.12)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
48 SparsePool code 35.24 % 43.55 % 30.15 % 0.13 s 8 cores @ 2.5 Ghz (Python)
Z. Wang, W. Zhan and M. Tomizuka: Fusing bird view lidar point cloud and front view camera image for deep object detection. arXiv preprint arXiv:1711.06703 2017.
49 Complexer-YOLO
This method makes use of Velodyne laser scans.
25.43 % 32.00 % 22.88 % 0.06 s GPU @ 3.5 Ghz (C/C++)
M. Simon, K. Amende, A. Kraus, J. Honer, T. Samann, H. Kaulbersch, S. Milz and H. Michael Gross: Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2019.
50 BirdNet
This method makes use of Velodyne laser scans.
23.78 % 36.01 % 21.09 % 0.11 s Titan Xp GPU
J. Beltrán, C. Guindel, F. Moreno, D. Cruzado, F. García and A. Escalera: BirdNet: A 3D Object Detection Framework from LiDAR Information. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
51 OC Stereo
This method uses stereo information.
19.23 % 32.47 % 17.11 % 0.35 s 1 core @ 2.5 Ghz (Python + C/C++)
52 SA_3D 16.91 % 24.56 % 14.06 % 0.3 s GPU @ 2.5 Ghz (Python)
53 TopNet-DecayRate
This method makes use of Velodyne laser scans.
16.00 % 23.02 % 13.24 % 92 ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
54 TopNet-UncEst
This method makes use of Velodyne laser scans.
9.18 % 12.31 % 8.14 % 0.09 s NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, M. Braun, M. Lauer and C. Stiller: Capturing Object Detection Uncertainty in Multi-Layer Grid Maps. 2019.
55 TopNet-HighRes
This method makes use of Velodyne laser scans.
6.48 % 9.99 % 6.76 % 101ms NVIDIA GeForce 1080 Ti (tensorflow-gpu)
S. Wirges, T. Fischer, C. Stiller and J. Frias: Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks. 2018 21st International Conference on Intelligent Transportation Systems (ITSC) 2018.
56 MonoPSR code 5.78 % 9.87 % 4.57 % 0.2 s GPU @ 3.5 Ghz (Python)
J. Ku*, A. Pon* and S. Waslander: Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction. CVPR 2019.
57 RT3DStereo
This method uses stereo information.
4.10 % 7.03 % 3.88 % 0.08 s GPU @ 2.5 Ghz (C/C++)
H. Königshof, N. Salscheider and C. Stiller: Realtime 3D Object Detection for Automated Driving Using Stereo Vision and Semantic Information. Proc. IEEE Intl. Conf. Intelligent Transportation Systems 2019.
58 MonoPair 2.87 % 4.76 % 2.42 % 0.06 s GPU @ 2.5 Ghz (Python + C/C++)
59 RefinedMPL 1.99 % 3.75 % 1.80 % 0.1 s GPU @ 2.5 Ghz (Python + C/C++)
60 SS3D 1.89 % 3.45 % 1.44 % 48 ms Tesla V100 (Python)
E. Jörgensen, C. Zach and F. Kahl: Monocular 3D Object Detection and Box Fitting Trained End-to-End Using Intersection-over-Union Loss. CoRR 2019.
61 D^4LCN 1.82 % 2.72 % 1.79 % 0.2 s GPU @ 2.5 Ghz (Python + C/C++)
62 mylsi-faster-rcnn 1.54 % 2.41 % 1.41 % 0.3 s 1 core @ 2.5 Ghz (Python)
63 PG-MonoNet 1.24 % 1.96 % 1.01 % 0.19 s GPU @ 2.5 Ghz (Python)
64 M3D-RPN code 0.81 % 1.25 % 0.78 % 0.16 s GPU @ 1.5 Ghz (Python)
G. Brazil and X. Liu: M3D-RPN: Monocular 3D Region Proposal Network for Object Detection . ICCV 2019 .
65 DT3D 0.77 % 1.49 % 0.85 % 0,21s GPU @ 2.5 Ghz (Python)
66 mymask-rcnn 0.71 % 1.39 % 0.69 % 0.3 s 1 core @ 2.5 Ghz (Python)
67 Shift R-CNN (mono) code 0.38 % 0.76 % 0.41 % 0.25 s GPU @ 1.5 Ghz (Python)
A. Naiden, V. Paunescu, G. Kim, B. Jeon and M. Leordeanu: Shift R-CNN: Deep Monocular 3D Object Detection With Closed-form Geometric Constraints. ICIP 2019.
68 OFT-Net 0.16 % 0.36 % 0.15 % 0.5 s 8 cores @ 2.5 Ghz (Python + C/C++)
69 mBoW
This method makes use of Velodyne laser scans.
0.00 % 0.00 % 0.00 % 10 s 1 core @ 2.5 Ghz (C/C++)
J. Behley, V. Steinhage and A. Cremers: Laser-based Segment Classification Using a Mixture of Bag-of-Words. Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013.
Table as LaTeX | Only published Methods

Related Datasets

Citation

When using this dataset in your research, we will be happy if you cite us:
@INPROCEEDINGS{Geiger2012CVPR,
  author = {Andreas Geiger and Philip Lenz and Raquel Urtasun},
  title = {Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2012}
}



eXTReMe Tracker