Dataset


This benchmark has been created in collaboration with Jannik Fritsch and Tobias Kuehnl from Honda Research Institute Europe GmbH. The road and lane estimation benchmark consists of 289 training and 290 test images. It contains three different categories of road scenes:

  • uu - urban unmarked (98/100)
  • um - urban marked (95/96)
  • umm - urban multiple marked lanes (96/94)

Ground truth has been generated by manual annotation of the images and is available for two different road terrain types: road - the road area, i.e, the composition of all lanes, and lane - the ego-lane, i.e., the lane the vehicle is currently driving on (only available for category "um"). Ground truth is provided for training images only.

Evaluation

We evaluate road and lane estimation performance in the bird's-eye-view space. For the classical pixel-based evaluation we use established measures as discussed in our ITSC 2013 publication. MaxF: Maximum F1-measure, AP: Average precision as used in PASCAL VOC challenges, PRE: Precision, REC: Recall, FPR: False Positive Rate, FNR: False Negative Rate (the four latter measures are evaluated at the working point MaxF), F1: F1 score, HR: Hit rate. For the novel behavior-based evaluation a corridor with the vehicle width (2.2m) is fitted to the lane estimation processing result and evaluation is performed for 3 different distance values: 20 m, 30 m, and 40 m. We refer to our ITSC 2013 publication for more details.

Additional information used by the methods
  • Stereo: Method uses left and right (stereo) images
  • Laser Points: Method uses point clouds from Velodyne laser scanner
  • GPS: Method uses GPS information

Road Estimation Evaluation

UM_ROAD


Rank Method Setting Code MaxF AP PRE REC FPR FNR Runtime Environment
1 STB 94.36 % 93.31 % 97.23 % 91.66 % 1.21 % 8.34 % 0.2 s 1 core @ 2.5 Ghz (Matlab)
Anonymous submission
2 DDN 93.18 % 88.75 % 94.47 % 91.93 % 2.49 % 8.07 % 2 s GPU @ 2.5 Ghz (Python + C/C++)
R. Mohan: Deep Deconvolutional Networks for Scene Parsing. 2014.
3 FusedCRF
This method makes use of Velodyne laser scans.
89.89 % 80.89 % 85.82 % 94.38 % 7.20 % 5.62 % 2 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
4 SPRAY 88.22 % 91.32 % 88.63 % 87.80 % 5.20 % 12.20 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
5 ProbBoost
This method uses stereo information.
87.60 % 76.04 % 85.92 % 89.36 % 6.76 % 10.64 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: A probabilistic distribution approach for the classification of urban roads in complex environments. Workshop on Modelling, Estimation, Perception and Control of All Terrain Mobile Robots on IEEE International Conference on Robotics and Automation (ICRA) 2014.
6 CN24 86.50 % 89.35 % 87.21 % 85.80 % 5.81 % 14.20 % 30 s >8 cores @ 2.5 Ghz (C/C++)
C. Brust, S. Sickert, M. Simon, E. Rodner and J. Denzler: Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding. VISAPP 2015 - Proceedings of the 10th International Conference on Computer Vision Theory and Applications, Berlin, Germany, 11-14 March, 2015 2015.
7 SPlane + BL
This method uses stereo information.
85.66 % 88.98 % 84.11 % 87.28 % 7.62 % 12.72 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
8 GRES3D+VELO
This method makes use of Velodyne laser scans.
85.49 % 83.39 % 81.20 % 90.27 % 9.65 % 9.74 % 60 ms 4 cores @ 2.8 Ghz (C/C++)
Anonymous submission
9 CB 84.59 % 73.51 % 82.82 % 86.44 % 8.28 % 13.56 % 2 s 1 core @ 2.5 Ghz (Python)
Anonymous submission
10 RES3D-Velo
This method makes use of Velodyne laser scans.
84.25 % 74.95 % 76.07 % 94.41 % 13.71 % 5.59 % 0.36 s 1 core @ 2.5 Ghz (C/C++)
P. Shinzato, D. Wolf and C. Stiller: Road Terrain Detection: Avoiding Common Obstacle Detection Assumptions Using Sensor Fusion. Intelligent Vehicles Symposium (IV) 2014.
11 NNP
This method uses stereo information.
84.22 % 71.99 % 80.96 % 87.74 % 9.53 % 12.26 % 5 s 4 cores @ 2.5 Ghz (Matlab)
Anonymous submission
12 GRES3D+SELAS
This method uses stereo information.
84.08 % 85.23 % 79.34 % 89.42 % 10.75 % 10.58 % 110 ms 4 core @ 2.8 Ghz (C/C++)
Anonymous submission
13 HistonBoost
This method uses stereo information.
83.71 % 73.31 % 82.58 % 84.87 % 8.27 % 15.13 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: Comprehensive Performance Analysis of Road Detection Algorithms Using the Common Urban Kitti-Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application on IEEE Intelligent Vehicles Symposium (IV) 2014.
14 MDE
This method makes use of Velodyne laser scans.
83.40 % 86.61 % 83.45 % 83.35 % 7.63 % 16.65 % 0.30 s 1 core @ 2.5 Ghz (Matlab + C/C++)
Anonymous submission
15 GRES3D+SGBM
This method uses stereo information.
82.56 % 83.69 % 77.20 % 88.73 % 12.10 % 11.27 % 0.72 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
16 BL 82.53 % 85.59 % 79.24 % 86.11 % 10.41 % 13.89 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
17 PGM-ARS 81.20 % 69.82 % 78.32 % 84.30 % 10.78 % 15.70 % 0.05 s i74700MQ @ 2.1Ghz (C/C++)
Anonymous submission
18 BM
This method uses stereo information.
79.19 % 66.78 % 70.29 % 90.66 % 17.69 % 9.34 % 2 s 2 cores @ 2.5 Ghz (Matlab)
B. WANG, V. Fremont and S. Rodriguez Florez: Color-based Road Detection and its Evaluation on the KITTI Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application, IEEE Intelligent Vehicles Symposium 2014.
19 MSCNN 79.17 % 66.82 % 80.03 % 78.34 % 9.03 % 21.66 % 2 s GPU @ 2.5 Ghz (C/C++)
Anonymous submission
20 RES3D-Stereo
This method uses stereo information.
79.01 % 80.21 % 76.64 % 81.54 % 11.47 % 18.46 % 0.7 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
21 SPlane
This method uses stereo information.
78.49 % 76.85 % 72.77 % 85.20 % 14.72 % 14.80 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
22 CN24 76.34 % 79.30 % 72.21 % 80.97 % 14.39 % 19.03 % 20 s >8 cores @ 2.5 Ghz (C/C++)
C. Brust, S. Sickert, M. Simon, E. Rodner and J. Denzler: Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding. VISAPP 2015 - Proceedings of the 10th International Conference on Computer Vision Theory and Applications, Berlin, Germany, 11-14 March, 2015 2015.
23 RF 75.02 % 78.76 % 72.04 % 78.25 % 14.02 % 21.75 % 1 min 1 core @ 3.5 Ghz (Matlab)
Anonymous submission
24 RF 74.29 % 77.78 % 70.04 % 79.10 % 15.63 % 20.90 % 2 min 8 cores @ 3.5 Ghz (Matlab)
Anonymous submission
25 ICF 74.14 % 58.41 % 64.37 % 87.40 % 22.34 % 12.60 % 2 s 4 cores @ 3.0 Ghz (C/C++)
Anonymous submission
26 CN 73.97 % 73.64 % 69.93 % 78.51 % 15.59 % 21.49 % 2 s 1 core @ 2.5 Ghz (C/C++)
J. Alvarez, T. Gevers, Y. LeCun and A. Lopez: Road Scene Segmentation from a Single Image. ECCV 2012 2012.
27 ARSL-AMI 72.05 % 61.67 % 78.87 % 66.32 % 8.21 % 33.68 % 0.05 s 4 cores @ 2.5 Ghz (C/C++)
M. Passani, J. Yebes and L. Bergasa: CRF-based semantic labeling in miniaturized road scenes . Proc. IEEE Intelligent Transportation Systems 2014.
28 ANN
This method uses stereo information.
62.64 % 46.80 % 50.18 % 83.34 % 38.21 % 16.66 % 3 s 1 core @ 3.0 Ghz (C/C++)
G. Vitor, D. Lima, A. Victorino and J. Ferreira: A 2D/3D Vision Based Approach Applied to Road Detection in Urban Environments. Intelligent Vehicles Symposium (IV), 2013 IEEE 2013.
This table as LaTeX

UMM_ROAD


Rank Method Setting Code MaxF AP PRE REC FPR FNR Runtime Environment
1 DDN 93.00 % 91.96 % 95.76 % 90.39 % 4.68 % 9.61 % 2 s GPU @ 2.5 Ghz (Python + C/C++)
R. Mohan: Deep Deconvolutional Networks for Scene Parsing. 2014.
2 PGM-ARS 90.95 % 85.68 % 88.86 % 93.14 % 13.66 % 6.86 % 0.05 s i74700MQ @ 2.1Ghz (C/C++)
Anonymous submission
3 ProbBoost
This method uses stereo information.
90.12 % 85.04 % 88.15 % 92.18 % 14.50 % 7.82 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: A probabilistic distribution approach for the classification of urban roads in complex environments. Workshop on Modelling, Estimation, Perception and Control of All Terrain Mobile Robots on IEEE International Conference on Robotics and Automation (ICRA) 2014.
4 ARSL-AMI 88.75 % 83.66 % 86.63 % 90.98 % 16.43 % 9.02 % 0.05 s 4 cores @ 2.5 Ghz (C/C++)
M. Passani, J. Yebes and L. Bergasa: CRF-based semantic labeling in miniaturized road scenes . Proc. IEEE Intelligent Transportation Systems 2014.
5 SPRAY 88.42 % 93.56 % 88.31 % 88.53 % 13.71 % 11.47 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
6 HistonBoost
This method uses stereo information.
87.70 % 81.59 % 84.36 % 91.32 % 19.81 % 8.68 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: Comprehensive Performance Analysis of Road Detection Algorithms Using the Common Urban Kitti-Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application on IEEE Intelligent Vehicles Symposium (IV) 2014.
7 RES3D-Velo
This method makes use of Velodyne laser scans.
87.64 % 85.81 % 86.70 % 88.60 % 15.91 % 11.40 % 0.36 s 1 core @ 2.5 Ghz (C/C++)
P. Shinzato, D. Wolf and C. Stiller: Road Terrain Detection: Avoiding Common Obstacle Detection Assumptions Using Sensor Fusion. Intelligent Vehicles Symposium (IV) 2014.
8 FusedCRF
This method makes use of Velodyne laser scans.
87.56 % 80.73 % 86.69 % 88.45 % 15.89 % 11.55 % 2 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
9 NNP
This method uses stereo information.
87.43 % 80.00 % 85.80 % 89.13 % 17.26 % 10.87 % 5 s 4 cores @ 2.5 Ghz (Matlab)
Anonymous submission
10 MSCNN 87.26 % 84.89 % 91.78 % 83.16 % 8.72 % 16.84 % 2 s GPU @ 2.5 Ghz (C/C++)
Anonymous submission
11 RF 86.82 % 88.41 % 81.05 % 93.48 % 25.58 % 6.52 % 1 min 1 core @ 3.5 Ghz (Matlab)
Anonymous submission
12 BM
This method uses stereo information.
86.56 % 80.49 % 83.15 % 90.26 % 21.40 % 9.74 % 2 s 2 cores @ 2.5 Ghz (Matlab)
B. WANG, V. Fremont and S. Rodriguez Florez: Color-based Road Detection and its Evaluation on the KITTI Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application, IEEE Intelligent Vehicles Symposium 2014.
13 RF 86.49 % 87.74 % 80.08 % 94.01 % 27.36 % 5.99 % 2 min 8 cores @ 3.5 Ghz (Matlab)
Anonymous submission
14 CN 85.77 % 84.91 % 83.37 % 88.30 % 20.61 % 11.70 % 2 s 1 core @ 2.5 Ghz (C/C++)
J. Alvarez, T. Gevers, Y. LeCun and A. Lopez: Road Scene Segmentation from a Single Image. ECCV 2012 2012.
15 CB 85.70 % 80.94 % 86.94 % 84.49 % 14.85 % 15.51 % 2 s 1 core @ 2.5 Ghz (Python)
Anonymous submission
16 GRES3D+VELO
This method makes use of Velodyne laser scans.
85.56 % 88.39 % 81.49 % 90.06 % 23.94 % 9.94 % 60 ms 4 cores @ 2.8 Ghz (C/C++)
Anonymous submission
17 GRES3D+SELAS
This method uses stereo information.
84.97 % 89.96 % 84.79 % 85.14 % 17.87 % 14.86 % 110 ms 4 core @ 2.8 Ghz (C/C++)
Anonymous submission
18 MDE
This method makes use of Velodyne laser scans.
84.49 % 89.57 % 88.24 % 81.04 % 12.63 % 18.96 % 0.30 s 1 core @ 2.5 Ghz (Matlab + C/C++)
Anonymous submission
19 GRES3D+SGBM
This method uses stereo information.
82.03 % 87.02 % 78.22 % 86.22 % 28.09 % 13.78 % 0.72 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
20 SPlane
This method uses stereo information.
81.95 % 83.09 % 76.77 % 87.88 % 31.11 % 12.12 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
21 SPlane + BL
This method uses stereo information.
81.62 % 85.53 % 75.65 % 88.62 % 33.38 % 11.38 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
22 RES3D-Stereo
This method uses stereo information.
81.31 % 85.43 % 80.04 % 82.62 % 24.11 % 17.38 % 0.7 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
23 ANN
This method uses stereo information.
81.09 % 68.93 % 70.43 % 95.56 % 46.94 % 4.44 % 3 s 1 core @ 3.0 Ghz (C/C++)
G. Vitor, D. Lima, A. Victorino and J. Ferreira: A 2D/3D Vision Based Approach Applied to Road Detection in Urban Environments. Intelligent Vehicles Symposium (IV), 2013 IEEE 2013.
24 BL 76.17 % 78.42 % 65.02 % 91.95 % 57.89 % 8.05 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
This table as LaTeX

UU_ROAD


Rank Method Setting Code MaxF AP PRE REC FPR FNR Runtime Environment
1 DDN 90.89 % 80.24 % 92.53 % 89.30 % 2.40 % 10.70 % 2 s GPU @ 2.5 Ghz (Python + C/C++)
R. Mohan: Deep Deconvolutional Networks for Scene Parsing. 2014.
2 FusedCRF
This method makes use of Velodyne laser scans.
84.26 % 72.65 % 77.42 % 92.42 % 8.96 % 7.58 % 2 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
3 RES3D-Velo
This method makes use of Velodyne laser scans.
83.78 % 73.29 % 78.63 % 89.65 % 8.11 % 10.35 % 0.36 s 1 core @ 2.5 Ghz (C/C++)
P. Shinzato, D. Wolf and C. Stiller: Road Terrain Detection: Avoiding Common Obstacle Detection Assumptions Using Sensor Fusion. Intelligent Vehicles Symposium (IV) 2014.
4 GRES3D+VELO
This method makes use of Velodyne laser scans.
83.78 % 80.48 % 81.39 % 86.31 % 6.57 % 13.69 % 60 ms 4 cores @ 2.8 Ghz (C/C++)
Anonymous submission
5 SPRAY 82.63 % 87.30 % 82.32 % 82.94 % 5.93 % 17.06 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
6 GRES3D+SELAS
This method uses stereo information.
82.31 % 84.08 % 77.06 % 88.32 % 8.75 % 11.68 % 110 ms 4 core @ 2.8 Ghz (C/C++)
Anonymous submission
7 GRES3D+SGBM
This method uses stereo information.
81.73 % 81.69 % 78.92 % 84.76 % 7.53 % 15.24 % 0.72 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
8 CB 81.63 % 71.69 % 82.07 % 81.19 % 5.90 % 18.81 % 2 s 1 core @ 2.5 Ghz (Python)
Anonymous submission
9 NNP
This method uses stereo information.
81.58 % 67.90 % 77.44 % 86.20 % 8.35 % 13.80 % 5 s 4 cores @ 2.5 Ghz (Matlab)
Anonymous submission
10 ProbBoost
This method uses stereo information.
80.29 % 69.05 % 85.58 % 75.61 % 4.24 % 24.39 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: A probabilistic distribution approach for the classification of urban roads in complex environments. Workshop on Modelling, Estimation, Perception and Control of All Terrain Mobile Robots on IEEE International Conference on Robotics and Automation (ICRA) 2014.
11 PGM-ARS 79.82 % 68.33 % 77.97 % 81.76 % 7.68 % 18.24 % 0.05 s i74700MQ @ 2.1Ghz (C/C++)
Anonymous submission
12 MDE
This method makes use of Velodyne laser scans.
79.34 % 80.04 % 82.25 % 76.63 % 5.50 % 23.37 % 0.30 s 1 core @ 2.5 Ghz (Matlab + C/C++)
Anonymous submission
13 BM
This method uses stereo information.
78.15 % 62.68 % 71.06 % 86.82 % 11.76 % 13.18 % 2 s 2 cores @ 2.5 Ghz (Matlab)
B. WANG, V. Fremont and S. Rodriguez Florez: Color-based Road Detection and its Evaluation on the KITTI Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application, IEEE Intelligent Vehicles Symposium 2014.
14 RES3D-Stereo
This method uses stereo information.
78.15 % 73.55 % 77.89 % 78.42 % 7.41 % 21.58 % 0.7 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
15 MSCNN 76.25 % 65.38 % 80.54 % 72.40 % 5.82 % 27.60 % 2 s GPU @ 2.5 Ghz (C/C++)
Anonymous submission
16 SPlane + BL
This method uses stereo information.
74.42 % 80.10 % 66.04 % 85.24 % 14.58 % 14.76 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
17 SPlane
This method uses stereo information.
73.63 % 69.87 % 65.43 % 84.18 % 14.80 % 15.82 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
18 HistonBoost
This method uses stereo information.
73.51 % 63.07 % 77.36 % 70.03 % 6.82 % 29.97 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: Comprehensive Performance Analysis of Road Detection Algorithms Using the Common Urban Kitti-Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application on IEEE Intelligent Vehicles Symposium (IV) 2014.
19 CN 71.48 % 66.30 % 72.09 % 70.88 % 9.13 % 29.12 % 2 s 1 core @ 2.5 Ghz (C/C++)
J. Alvarez, T. Gevers, Y. LeCun and A. Lopez: Road Scene Segmentation from a Single Image. ECCV 2012 2012.
20 RF 70.17 % 67.83 % 62.23 % 80.44 % 16.24 % 19.56 % 1 min 1 core @ 3.5 Ghz (Matlab)
Anonymous submission
21 ARSL-AMI 69.70 % 57.15 % 83.96 % 59.57 % 3.78 % 40.43 % 0.05 s 4 cores @ 2.5 Ghz (C/C++)
M. Passani, J. Yebes and L. Bergasa: CRF-based semantic labeling in miniaturized road scenes . Proc. IEEE Intelligent Transportation Systems 2014.
22 BL 69.49 % 73.84 % 65.73 % 73.70 % 12.78 % 26.30 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
23 RF 68.96 % 66.64 % 61.05 % 79.22 % 16.82 % 20.78 % 2 min 8 cores @ 3.5 Ghz (Matlab)
Anonymous submission
24 ANN
This method uses stereo information.
54.17 % 36.86 % 39.50 % 86.19 % 43.92 % 13.81 % 3 s 1 core @ 3.0 Ghz (C/C++)
G. Vitor, D. Lima, A. Victorino and J. Ferreira: A 2D/3D Vision Based Approach Applied to Road Detection in Urban Environments. Intelligent Vehicles Symposium (IV), 2013 IEEE 2013.
This table as LaTeX

URBAN_ROAD


Rank Method Setting Code MaxF AP PRE REC FPR FNR Runtime Environment
1 DDN 92.55 % 89.34 % 94.62 % 90.58 % 2.97 % 9.42 % 2 s GPU @ 2.5 Ghz (Python + C/C++)
R. Mohan: Deep Deconvolutional Networks for Scene Parsing. 2014.
2 FusedCRF
This method makes use of Velodyne laser scans.
87.42 % 79.71 % 84.03 % 91.08 % 9.96 % 8.92 % 2 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
3 ProbBoost
This method uses stereo information.
87.21 % 77.79 % 86.96 % 87.47 % 7.55 % 12.53 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: A probabilistic distribution approach for the classification of urban roads in complex environments. Workshop on Modelling, Estimation, Perception and Control of All Terrain Mobile Robots on IEEE International Conference on Robotics and Automation (ICRA) 2014.
4 SPRAY 86.33 % 90.91 % 86.78 % 85.89 % 7.53 % 14.11 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
5 PGM-ARS 85.52 % 74.75 % 83.24 % 87.92 % 10.19 % 12.07 % 0.05 s i74700MQ @ 2.1Ghz (C/C++)
Anonymous submission
6 RES3D-Velo
This method makes use of Velodyne laser scans.
85.49 % 79.03 % 79.93 % 91.88 % 13.28 % 8.12 % 0.36 s 1 core @ 2.5 Ghz (C/C++)
P. Shinzato, D. Wolf and C. Stiller: Road Terrain Detection: Avoiding Common Obstacle Detection Assumptions Using Sensor Fusion. Intelligent Vehicles Symposium (IV) 2014.
7 NNP
This method uses stereo information.
85.10 % 74.02 % 82.35 % 88.04 % 10.87 % 11.96 % 5 s 4 cores @ 2.5 Ghz (Matlab)
Anonymous submission
8 GRES3D+VELO
This method makes use of Velodyne laser scans.
84.71 % 84.44 % 82.91 % 86.59 % 10.28 % 13.41 % 60 ms 4 cores @ 2.8 Ghz (C/C++)
Anonymous submission
9 CB 84.42 % 75.84 % 84.57 % 84.27 % 8.85 % 15.73 % 2 s 1 core @ 2.5 Ghz (Python)
Anonymous submission
10 GRES3D+SELAS
This method uses stereo information.
83.94 % 86.44 % 81.15 % 86.93 % 11.62 % 13.07 % 110 ms 4 core @ 2.8 Ghz (C/C++)
Anonymous submission
11 HistonBoost
This method uses stereo information.
83.41 % 74.06 % 82.39 % 84.46 % 10.39 % 15.54 % 2.5 min >8 cores @ 3.0 Ghz (C/C++)
G. Vitor, A. Victorino and J. Ferreira: Comprehensive Performance Analysis of Road Detection Algorithms Using the Common Urban Kitti-Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application on IEEE Intelligent Vehicles Symposium (IV) 2014.
12 MDE
This method makes use of Velodyne laser scans.
82.72 % 87.58 % 85.44 % 80.17 % 7.87 % 19.83 % 0.30 s 1 core @ 2.5 Ghz (Matlab + C/C++)
Anonymous submission
13 MSCNN 82.32 % 72.25 % 85.65 % 79.25 % 7.65 % 20.75 % 2 s GPU @ 2.5 Ghz (C/C++)
Anonymous submission
14 BM
This method uses stereo information.
82.32 % 68.95 % 76.15 % 89.56 % 16.15 % 10.44 % 2 s 2 cores @ 2.5 Ghz (Matlab)
B. WANG, V. Fremont and S. Rodriguez Florez: Color-based Road Detection and its Evaluation on the KITTI Road Benchmark. Workshop on Benchmarking Road Terrain and Lane Detection Algorithms for In-Vehicle Application, IEEE Intelligent Vehicles Symposium 2014.
15 GRES3D+SGBM
This method uses stereo information.
81.77 % 84.33 % 80.30 % 83.29 % 11.77 % 16.71 % 0.72 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
16 ARSL-AMI 80.12 % 71.12 % 84.08 % 76.52 % 8.34 % 23.48 % 0.05 s 4 cores @ 2.5 Ghz (C/C++)
M. Passani, J. Yebes and L. Bergasa: CRF-based semantic labeling in miniaturized road scenes . Proc. IEEE Intelligent Transportation Systems 2014.
17 RES3D-Stereo
This method uses stereo information.
79.91 % 81.56 % 78.55 % 81.32 % 12.79 % 18.68 % 0.7 s 1 core @ 2.5 Ghz (C/C++)
Anonymous submission
18 SPlane + BL
This method uses stereo information.
79.48 % 83.93 % 73.59 % 86.40 % 17.85 % 13.60 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
19 RF 79.23 % 83.08 % 73.92 % 85.36 % 17.34 % 14.64 % 1 min 1 core @ 3.5 Ghz (Matlab)
Anonymous submission
20 CN 78.92 % 79.14 % 76.25 % 81.79 % 14.67 % 18.21 % 2 s 1 core @ 2.5 Ghz (C/C++)
J. Alvarez, T. Gevers, Y. LeCun and A. Lopez: Road Scene Segmentation from a Single Image. ECCV 2012 2012.
21 SPlane
This method uses stereo information.
78.75 % 77.66 % 72.41 % 86.30 % 18.93 % 13.70 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
22 RF 78.56 % 81.93 % 73.23 % 84.72 % 17.83 % 15.28 % 2 min 8 cores @ 3.5 Ghz (Matlab)
Anonymous submission
23 BL 75.61 % 79.72 % 68.93 % 83.73 % 21.73 % 16.27 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
24 ANN
This method uses stereo information.
68.12 % 51.52 % 54.85 % 89.85 % 42.59 % 10.15 % 3 s 1 core @ 3.0 Ghz (C/C++)
G. Vitor, D. Lima, A. Victorino and J. Ferreira: A 2D/3D Vision Based Approach Applied to Road Detection in Urban Environments. Intelligent Vehicles Symposium (IV), 2013 IEEE 2013.
This table as LaTeX

Lane Estimation Evaluation

UM_LANE


Rank Method Setting Code MaxF AP PRE REC FPR FNR Runtime Environment
1 PCA-Lane-S 87.59 % 75.13 % 88.48 % 86.72 % 2.00 % 13.28 % 0.03 s 1 core @ 2.5 Ghz (Python + C/C++)
Anonymous submission
2 SPRAY 83.16 % 86.72 % 84.67 % 81.71 % 2.62 % 18.29 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
3 BL 76.35 % 79.61 % 72.46 % 80.67 % 5.43 % 19.33 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
4 SPlane + BL
This method uses stereo information.
69.81 % 74.06 % 80.89 % 61.39 % 2.57 % 38.61 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
This table as LaTeX

Behaviour Evaluation

UM_LANE


Rank Method Setting Code PRE-20 F1-20 HR-20 PRE-30 F1-30 HR-30 PRE-40 F1-40 HR-40 Runtime Environment
1 PCA-Lane-S 98.05 % 97.31 % 96.56 % 96.58 % 95.96 % 94.25 % 91.21 % 88.37 % 76.00 % 0.03 s 1 core @ 2.5 Ghz (Python + C/C++)
Anonymous submission
2 SPRAY 97.51 % 96.76 % 96.45 % 96.92 % 94.84 % 92.65 % 88.76 % 79.76 % 63.51 % 45 ms NVIDIA GTX 580 (Python + OpenCL)
T. Kuehnl, F. Kummert and J. Fritsch: Spatial Ray Features for Real-Time Ego-Lane Extraction. Proc. IEEE Intelligent Transportation Systems 2012.
3 BL 95.65 % 92.96 % 89.50 % 94.47 % 89.98 % 84.62 % 87.23 % 70.11 % 52.63 % 0.02 s 1 core @ 2.5 Ghz (Python)
J. Fritsch, T. Kuehnl and A. Geiger: A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms. IEEE Int. Conf. on Intelligent Transportation Systems(ITSC) 2013.
4 SPlane + BL
This method uses stereo information.
95.48 % 93.05 % 91.55 % 92.34 % 88.07 % 75.17 % 79.79 % 50.57 % 0.00 % 2 s 1 core @ 3.0 Ghz (C/C++)
N. Einecke and J. Eggert: Block-Matching Stereo with Relaxed Fronto-Parallel Assumption. IV 2014.
This table as LaTeX

Related Datasets

Citation

When using this dataset in your research, we will be happy if you cite us:

@INPROCEEDINGS{Fritsch2013ITSC,
  author = {Jannik Fritsch and Tobias Kuehnl and Andreas Geiger},
  title = {A New Performance Measure and Evaluation Benchmark for Road Detection Algorithms},
  booktitle = {International Conference on Intelligent Transportation Systems (ITSC)},
  year = {2013}
}



eXTReMe Tracker