Novel View Synthesis

Novel View Semantic Synthesis (50% Drop Rate)


Our evaluation table ranks all methods according to the confidence weighted mean intersection-over-union (mIoU). The weighted IoU of one class can be defined as \(\text{IoU} = \frac{\sum_{i\in{\{\text{TP}\}}}c_{i}}{\sum_{i\in{\{\text{TP, FP, FN}\}}}c_{i}}\) where \(\{\text{TP}\}\) and \(\{\text{TP, FP, FN}\}\) are the set of image pixels in the intersection and the union of the class label, respectively. \(c_i \in [0, 1]\) denotes the confidence value at pixel \(i\). In constrast to standard evaluation where \(c_i=1\) for all pixels, we adopt confidence weighted evaluation metrics leveraging the uncertainty to take into account the ambiguity in our automatically generated annotations.

Method Setting Code mIoU Class mIoU Category Runtime Environment
1 PNF 73.06 84.97 15 s GPU @ 2.5 Ghz (Python)
A. Kundu, K. Genova, X. Yin, A. Fathi, C. Pantofaru, L. Guibas, A. Tagliasacchi, F. Dellaert and T. Funkhouser: Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation. CVPR 2022.
2 HUGS 72.65 85.64 0.02 s 1 core @ 2.5 Ghz (C/C++)
3 GT Image + PSPNet 63.82 78.25 0.2 s 1 core @ 2.5 Ghz (C/C++)
Y. Liao, J. Xie and A. Geiger: KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D. ARXIV 2021.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia: Pyramid Scene Parsing Network. CVPR 2017.
4 FVS + PSPNet 60.86 74.61 0.4 s 1 core @ 2.5 Ghz (C/C++)
ERROR: Wrong syntax in BIBTEX file.
5 PBNR + PSPNet 58.43 71.99 1 s 1 core @ 2.5 Ghz (C/C++)
G. Kopanas, J. Philip, T. Leimkühler and G. Drettakis: Point-Based Neural Rendering with Per-View Optimization. Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering) 2021.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia: Pyramid Scene Parsing Network. CVPR 2017.
6 NeRF + PSPNet 49.57 69.14 15 s GPU @ 2.5 Ghz (Python)
B. Mildenhall, P. Srinivasan, M. Tancik, J. Barron, R. Ramamoorthi and R. Ng: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV 2020.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia: Pyramid Scene Parsing Network. CVPR 2017.
7 mip-NeRF + PSPNet 48.25 67.47 15 s GPU @ 2.5 Ghz (Python)
J. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla and P. Srinivasan: Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV 2021.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia: Pyramid Scene Parsing Network. CVPR 2017.
8 PCL + PSPNet code 37.21 44.55 0.4 s 1 core @ 2.5 Ghz (C/C++)
Y. Liao, J. Xie and A. Geiger: KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D. ARXIV 2021.
H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia: Pyramid Scene Parsing Network. CVPR 2017.
Table as LaTeX | Only published Methods





eXTReMe Tracker