ECCV 2020 Workshop

Holistic Scene Structures for 3D Vision

ONLINE - Sunday, August 23, 2020

News

[2020.8] Thanks to everyone for attending our workshop! Please check out our YouTube channel for the video recordings.

Introduction

The perception of holistic scene structures, that is, orderly, regular, symmetric, or repetitive patterns and relationships in a scene, plays a critical role in human vision. When walking in a man-made environment, such as office buildings, a human can instantly identify parallel lines, rectangles, cuboids, rotational symmetries, repetitive patterns, and many other types of structure, and exploit them for accurate and robust 3D localization, orientation, and navigation. In computer vision, the use of such holistic structural elements has a long history in 3D modeling of physical environments. Due to their ability and potential in creating high-level, compact, and semantically rich scene representations, which are ideal for modern 3D vision tasks, these methods have drawn increased attention in the research community lately.

However, significantly more efforts are still required to enable real-world complex tasks, such as augmented reality, robot navigation and human-robot interaction, as they often demand human-level understanding of the environments. To fully harness the power of holistic scene structures as humans do, we are interested in the following questions:

  1. Representations: What representations are suitable for capturing various types of scene structures?
  2. Structured 3D Modeling: How to efficiently recover structures from data acquired by a variety of sensors such as monocular and binocular vision, LiDAR, and RGB-D sensors?
  3. Synthesis and Editing: How to leverage structures for photo-realistic image synthesis and editing, as well as 3D scene generation and manipulation?
  4. Reasoning, Planning, and Interaction: How to use structures to reason about physical and functional properties, and to anticipate activities in a dynamic environment, in order to enable the agent to act within it?
  5. Applications: What are the other pressing issues (e.g.,hardware development) in enabling real-world applications?

As such, this workshop will bring together researchers working on exploring holistic scene structures for accurate, robust, and reliable 3D vision, as well as researchers who use structures in a variety of disciplines (e.g., robotics, VR/AR,interior design, and architectural engineering). In the workshop, both geometrybased and recent learning-based approaches will be discussed. We will examine the challenges, new directions, and the implication of industrial applications on holistic 3D modeling approaches.

Invited Speakers

Schedule

** All times are in UK time zone (GMT+1). **

8:00 - 8:10

[Live] Welcome remarks

8:10 - 9:00

[Live] Keynote talk #1 by Prof. Marc Pollefeys [video]

9:00 - 9:50

[Live] Keynote talk #2 by Prof. Niloy Mitra [video]

10:00 - 10:30

Invited talk #1 by Prof. Amir Zamir [video]

10:30 - 11:00

Invited talk #2 by Prof. Manolis Savva [video]

11:00 - 12:00

Holistic 3D Vision Challenges - track 1

  • Summary remarks by Jia Zheng
  • Winner talk #1 by Cheng Sun, Chi-Wei Hsiao, Min Sun, and Hwann-Tzong Chen [paper]
  • Winner talk #2 by Hao Zhao, Ming Lu, Yangyuxuan Kang, Anbang Yao, Yurong Chen, and Enhua Wu [paper]
  • Winner talk #3 by Dongho Choi [paper]

12:00 - 13:00

Lunch Break

13:00 - 13:30

Holistic 3D Vision Challenges - track 2 summary remarks by Yichao Zhou

13:30 - 14:00

Invited talk #3 by Prof. Florent Lafarge [video]

14:00 - 14:30

Invited talk #4 by Prof. Daniel Aliaga [video]

14:30 - 15:00

Invited talk #5 by Prof. Shubham Tulsiani [video]

15:00 - 16:00

Special session: ECCV 2020 papers on holistic 3D vision

  • Associative3D: Volumetric Reconstruction from Sparse Views by Shengyi Qian, Linyi Jin, and David F. Fouhey [paper]
  • AtlantaNet: Inferring the 3D Indoor Layout from a Single 360 Image Beyond the Manhattan World Assumption by Giovanni Pintore, Marco Agus, and Enrico Gobbetti [paper]
  • Deep Hough-Transform Line Priors by Yancong Lin, Silvia L. Pintea, and Jan C. van Gemert [paper]
  • H3DNet: 3D Object Detection Using Hybrid Geometric Primitives by Zaiwei Zhang, Bo Sun, Haitao Yang, and Qixing Huang [paper]
  • House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation by Nelson Nauata, Kai-Hung Chang, Chin-Yi Cheng, Greg Mori, and Yasutaka Furukawa [paper]
  • Learning 3D Part Assembly from A Single Image by Yichen Li, Kaichun Mo, Lin Shao, Minhyuk Sung, and Leonidas J. Guibas [paper]
  • Modeling 3D Shapes by Reinforcement Learning by Cheng Lin, Tingxiang Fan, Wenping Wang, and Matthias Nießner [paper]
  • SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans by Armen Avetisyan, Tatiana Khanova, Christopher Choy, Denver Dash, Angela Dai, and Matthias Nießner [paper]
  • Structured3d: A large photo-realistic dataset for structured 3d modeling by Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou [paper]

16:00 - 18:00

[Live] Panel discussion with invited speakers [video]

Resources

We maintain a list of datasets, codes, and papers on holistic 3D reconstruction here.

Organizers

Senior Advisors

Acknowledgement