Abstract

Download

Course Notes

• Course notes (PDF)

Slides

• Geometric Reasoning (by Niloy J. Mitra)
• Physical Constraints (by Nobuyuki Umetani)
• Data-Driven Techniques and Machine Learning (by Ariel Shamir)
• Crowdsourcing (by Yuki Koyama)

Course Contents

Introduction [Talk by Ariel Shamir] — 15 min

Geometric Reasoning [Talk by Niloy J. Mitra] — 45 min

An intelligent agent should be able to generate novel variations that are valid and diverse—geometrically, semantically, and topologically. Examples include generating furniture layout, floor plan generation, 3D models, and more generally virtual 3D environments. In this part of the course, we propose to focus on: i) generative 3D models ii) generative layouts incorporating both topological and geometric variations.

Physical Constraints [Talk by Nobuyuki Umetani] — 45 min

Computer controlled fabrication tools such as 3D printers, laser cutters and CNC milling machines have become wildly available to the consumers. However, it is difficult to design original customized shapes while considering its physical functionality. In this talk, we present the studies to incorporate real-time physics simulation into help the design of creative functional shapes. In particular, we focus on the (i) modeling of physics (ii) data-driven approach to facilitate the interactive modeling.

Short Break — 15 min

Data-Driven Techniques and Machine Learning [Talk by Ariel Shamir] — 45 min

Recent years have seen a revolution in the amount and ways data is used in all realms of science in general and computer science specifically. Complex problems are solved using machine learning techniques that rely on data. For example, in computer vision neural networks are now used for almost every task successfully. The question is how to make use of such techniques in the context of an intelligent tool for the creation of graphical content.

To this end, there are several machine learning techniques that enable the tool to make independent decisions without explicit input from the human user. These include:

• Classifiers or clustering algorithms that assist filtering large amounts of data to bring only relevant data to the user.
• Feature extraction from media (images, 3D objects, videos) that allow faster search and retrieval, and faster comparison between objects.
• Cross-modal analysis that allows connecting between different media types.
• Generative models that enable the creation of novel content either for inspiration or for further refinement.

In this part of the course we will present the basics of these technologies and show examples how they are used inside intelligent tools for the creation of various media such as images, video, and 3D objects.

Crowdsourcing [Talk by Yuki Koyama] — 25 min

Tools can be more intelligent if they can adequately model human perception (e.g., visual preference and perceived semantics). Crowdsourcing is an effective way to gather perceptual feedback on visual designs on demand, and researchers in computer graphics have proposed digital content creation tools with crowd intelligence in this decade. In this part of the course, especially focusing on parametric design scenarios, we will describe

• how to model human preference via crowdsourcing (e.g., ways to design crowdsourcing queries and to learn a latent function from gathered data) and
• how to use the preference model to make tools intelligent.

For the latter, we will especially discuss two approaches: embedding crowds into a user interface to facilitate users' exploration and decision making and embedding crowds into an optimization engine to achieve automatic perception-aware parameter adjustment, with case studies on photo color enhancement and material BRDF design.

Discussion — 15 min

Presenters