Joint 3D Object and Layout Inference from a single RGB-D Image

2015

Conference Paper

avg

ps


Inferring 3D objects and the layout of indoor scenes from a single RGB-D image captured with a Kinect camera is a challenging task. Towards this goal, we propose a high-order graphical model and jointly reason about the layout, objects and superpixels in the image. In contrast to existing holistic approaches, our model leverages detailed 3D geometry using inverse graphics and explicitly enforces occlusion and visibility constraints for respecting scene properties and projective geometry. We cast the task as MAP inference in a factor graph and solve it efficiently using message passing. We evaluate our method with respect to several baselines on the challenging NYUv2 indoor dataset using 21 object categories. Our experiments demonstrate that the proposed method is able to infer scenes with a large degree of clutter and occlusions.

Award: (Best Paper Award)
Author(s): Andreas Geiger and Chaohui Wang
Book Title: German Conference on Pattern Recognition (GCPR)
Volume: 9358
Pages: 183--195
Year: 2015
Series: Lecture Notes in Computer Science
Publisher: Springer International Publishing

Department(s): Autonomous Vision, Perceiving Systems
Research Project(s): High-level Priors
Scene Understanding
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Award Paper: Best Paper Award
DOI: 10.1007/978-3-319-24947-6_15
Event Place: Aachen
ISBN: 978-3-319-24946-9

Links: pdf
suppmat
video
project
Video:

BibTex

@inproceedings{Geiger2015GCPR,
  title = {Joint 3D Object and Layout Inference from a single RGB-D Image},
  author = {Geiger, Andreas and Wang, Chaohui},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  volume = {9358},
  pages = {183--195},
  series = {Lecture Notes in Computer Science},
  publisher = {Springer International Publishing},
  year = {2015}
}