KITTI-360

Method

Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation [PNF]

Submitted on 9 Apr. 2022 17:54 by
Abhijit Kundu (Google)

Running time:		15 s
Environment:		GPU @ 2.5 Ghz (Python)

Method Description:

Rendered semantic label images from our Panoptic
Neural Fields (PNF)
model. Unlike other methods like NeRF + <2D
segmentation>, the
semantic images can be directly rendered from
volumetric panoptic -
radiance field created by our model. The same model is
also used to
render color images (see our results on novel view rgb
task).

Parameters:

Stuff MLP: 10 layers, 256 neurons
Object MLPs: 4 layers, 128 neurons

Latex Bibtex:

@inproceedings{pnf2022,
title={Panoptic Neural Fields: A Semantic Object-Aware
Neural Scene Representation},
author={Abhijit Kundu and Kyle Genova and Xiaoqi Yin
and Alireza Fathi and Caroline Pantofaru and Leonidas
Guibas and Andrea Tagliasacchi and Frank Dellaert and
Thomas Funkhouser},
booktitle={CVPR},
year={2022}
}

Detailed Results

This page provides detailed results for the method(s) selected. For the first 10 test images, we display the original image, the color-coded result and an error image. The error image contains 4 colors weighted by the confidence of the pseudo-ground truth:
red: the pixel has the wrong label and the wrong category
yellow: the pixel has the wrong label but the correct category
green: the pixel has the correct label
black: the groundtruth label is not used for evaluation

Test Set Average

Warning: Undefined array key "traffic light" in /home/ageiger/cvlibs.net/datasets/kitti-360/detail_semantic_seg_table.php on line 48

Warning: Undefined array key "rider" in /home/ageiger/cvlibs.net/datasets/kitti-360/detail_semantic_seg_table.php on line 48

Warning: Undefined array key "truck" in /home/ageiger/cvlibs.net/datasets/kitti-360/detail_semantic_seg_table.php on line 48

road	sidewalk	building	wall	fence	pole	traffic light	traffic sign	vegetation	terrain	sky	rider	car	truck	motorcycle	mIoU class
94.76	84.68	69.81	59.79	59.60	58.67	0.00	31.61	92.59	92.98	85.62	0.00	96.14	0.00	50.50	73.06

flat	construction	object	nature	human	vehicle	sky	mIoU category
95.27	84.60	55.41	93.08	0.00	95.83	85.62	84.97

Error