News

[2020.8] Thanks to everyone for attending our workshop! Please check out our YouTube channel for the video recordings.

Introduction

The perception of holistic scene structures, that is, orderly, regular, symmetric, or repetitive patterns and relationships in a scene, plays a critical role in human vision. When walking in a man-made environment, such as office buildings, a human can instantly identify parallel lines, rectangles, cuboids, rotational symmetries, repetitive patterns, and many other types of structure, and exploit them for accurate and robust 3D localization, orientation, and navigation. In computer vision, the use of such holistic structural elements has a long history in 3D modeling of physical environments. Due to their ability and potential in creating high-level, compact, and semantically rich scene representations, which are ideal for modern 3D vision tasks, these methods have drawn increased attention in the research community lately.

However, significantly more efforts are still required to enable real-world complex tasks, such as augmented reality, robot navigation and human-robot interaction, as they often demand human-level understanding of the environments. To fully harness the power of holistic scene structures as humans do, we are interested in the following questions:

1. Representations: What representations are suitable for capturing various types of scene structures?
2. Structured 3D Modeling: How to efficiently recover structures from data acquired by a variety of sensors such as monocular and binocular vision, LiDAR, and RGB-D sensors?
3. Synthesis and Editing: How to leverage structures for photo-realistic image synthesis and editing, as well as 3D scene generation and manipulation?
4. Reasoning, Planning, and Interaction: How to use structures to reason about physical and functional properties, and to anticipate activities in a dynamic environment, in order to enable the agent to act within it?
5. Applications: What are the other pressing issues (e.g.,hardware development) in enabling real-world applications?

As such, this workshop will bring together researchers working on exploring holistic scene structures for accurate, robust, and reliable 3D vision, as well as researchers who use structures in a variety of disciplines (e.g., robotics, VR/AR,interior design, and architectural engineering). In the workshop, both geometrybased and recent learning-based approaches will be discussed. We will examine the challenges, new directions, and the implication of industrial applications on holistic 3D modeling approaches.

Invited Speakers

Marc Pollefeys
ETH Zurich

Niloy J. Mitra
UCL

Daniel G. Aliaga
Purdue U.

Florent Lafarge
Inria

Manolis Savva
Simon Fraser U.

Shubham Tulsiani
Facebook AI Research

Amir R. Zamir
EPFL

Schedule

Resources

We maintain a list of datasets, codes, and papers on holistic 3D reconstruction here.

Organizers

Zihan Zhou
Penn State U.

Yasutaka Furukawa
Simon Fraser U.

Yi Ma
UC Berkeley

Shenghua Gao
ShanghaiTech U.

Chen Liu
Facebook Reality Labs

Yichao Zhou
UC Berkeley

Linjie Luo
ByteDance AI Lab

Jia Zheng
ShanghaiTech U.

Junfei Zhang
Kujiale.com

Rui Tang
Kujiale.com

Senior Advisors

Marc Pollefeys
ETH Zurich

Niloy J. Mitra
UCL

Acknowledgement