Patches, Planes and Probabilities: A Non-local Prior for Volumetric 3D Reconstruction

2016

Conference Paper

avg

ps


In this paper, we propose a non-local structured prior for volumetric multi-view 3D reconstruction. Towards this goal, we present a novel Markov random field model based on ray potentials in which assumptions about large 3D surface patches such as planarity or Manhattan world constraints can be efficiently encoded as probabilistic priors. We further derive an inference algorithm that reasons jointly about voxels, pixels and image segments, and estimates marginal distributions of appearance, occupancy, depth, normals and planarity. Key to tractable inference is a novel hybrid representation that spans both voxel and pixel space and that integrates non-local information from 2D image segmentations in a principled way. We compare our non-local prior to commonly employed local smoothness assumptions and a variety of state-of-the-art volumetric reconstruction baselines on challenging outdoor scenes with textureless and reflective surfaces. Our experiments indicate that regularizing over larger distances has the potential to resolve ambiguities where local regularizers fail.

Author(s): Ali Osman Ulusoy and Michael J. Black and Andreas Geiger
Book Title: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2016
Month: June

Department(s): Autonomous Vision, Perceiving Systems
Research Project(s): Volumetric Reconstruction
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2016

Links: YouTube
Video:
Attachments: pdf
poster
suppmat

BibTex

@inproceedings{Ulusoy:CVPR:2016,
  title = {Patches, Planes and Probabilities: A Non-local Prior for Volumetric {3D} Reconstruction},
  author = {Ulusoy, Ali Osman and Black, Michael J. and Geiger, Andreas},
  booktitle = { IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  month = jun,
  year = {2016},
  month_numeric = {6}
}