Header logo is

Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision

2020

Conference Paper

avg


Learning-based 3D reconstruction methods have shown impressive results. However, most methods require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differentiable rendering techniques to train reconstruction models from RGB images. Unfortunately, these approaches are currently restricted to voxel- and mesh-based representations, suffering from discretization or low resolution. In this work, we propose a differentiable rendering formulation for implicit shape and texture representations. Implicit representations have recently gained popularity as they represent shape and texture continuously. Our key insight is that depth gradients can be derived analytically using the concept of implicit differentiation. This allows us to learn implicit shape and texture representations directly from RGB images. We experimentally show that our single-view reconstructions rival those learned with full 3D supervision. Moreover, we find that our method can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.

Author(s): Michael Niemeyer and Lars Mescheder and Michael Oechsle and Andreas Geiger
Book Title: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2020

Department(s): Autonomous Vision
Bibtex Type: Conference Paper (inproceedings)

Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Event Place: Seattle, USA

Links: pdf
suppmat
Video 2
Project Page
Video 1
Video 3
Slides
Poster
Video:
Video:
Video:

BibTex

@inproceedings{Niemeyer2020CVPR,
  title = {Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision},
  author = {Niemeyer, Michael and Mescheder, Lars and Oechsle, Michael and Geiger, Andreas},
  booktitle = { Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  year = {2020}
}