News

[2022.04] Our workshop website is online! Stay tuned for more updates.

Introduction

The perception of holistic structures, that is, orderly, regular, symmetric, or repetitive patterns and relationships in a scene, plays a critical role in human vision. When walking in a man-made environment, such as office buildings, a human can instantly identify parallel lines, rectangles, cuboids, rotational symmetries, repetitive patterns, and many other types of structure, and exploit them for accurate and robust 3D localization, orientation, and navigation. In computer vision, the use of such holistic structural elements has a long history in 3D modeling of physical environments. Due to their ability and potential in creating high-level, compact, and semantically rich scene representations, which are ideal for modern 3D vision tasks, these methods have drawn increased attention in the research community lately.

However, significantly more efforts are still required to enable real-world complex tasks, such as augmented reality, robot navigation and human-robot interaction, as they often demand human-level understanding of the environments. To fully harness the power of holistic structures as humans do, we are interested in the following questions:

1. Structured 3D reconstruction: How to efficiently recover structures from data acquired by a variety of sensors such as monocular and binocular vision, LiDAR, and RGB-D sensors?
2. Synthesis and editing: How to leverage structures for photo-realistic image synthesis and editing?
3. 3D scene generation and manipulation: How to enable machines to produce and update CAD-like 3D models?
4. Representation learning: What representations are suitable for capturing various types of scene structures?
5. Reasoning, planning, and interaction: How to use structures to reason about physical and functional properties, and to anticipate activities in a dynamic environment, in order to enable the agent to act within it?
6. Applications: What are the other pressing issues (e.g., hardware development) in enabling real-world applications?

As such, this workshop will bring together researchers working on exploring holistic structures for accurate, robust, and reliable 3D vision, as well as researchers who use structures in a variety of disciplines (e.g., robotics, VR/AR,interior design, and architectural engineering). We will examine the challenges, new directions, and the implication of industrial applications on holistic 3D modeling approaches.

Resources

We maintain a list of datasets, codes, and papers on holistic 3D reconstruction here.

Organizers

Zihan Zhou
Manycore Tech Inc.

Yasutaka Furukawa
Simon Fraser U.

Yi Ma
UC Berkeley

Shenghua Gao
ShanghaiTech U.

Chen Feng
New York U.