Events Calendar

CS Colloquium: Joseph Lim (MIT) - Toward Visual Understanding of the Physical World for Interaction

Speaker: Joseph Lim, MIT

Talk Title: Toward Visual Understanding of the Physical World for Interaction

Series: CS Colloquium

Abstract: This lecture satisfies requirements for CSCI 591: Computer Science Research Colloquium

Recently, the computer vision community has made impressive progress on object recognition with deep learning approaches. However, for any visual system to interact with objects, it needs to understand much more than simply recognizing where the objects are. The goal of my research is to explore and solve physical understanding tasks for interaction -- finding an object's pose in 3D, interpreting its physical interactions, and understanding its various states and transformations. Unfortunately, obtaining extensive annotated data for such tasks is often intractable, yet required by recent popular learning techniques.

In this talk, I take a step away from expensive, manually labeled datasets. Instead, I develop learning algorithms that are supervised through physical constraints combined with structured priors. I will first talk about how to build learning algorithms, including a deep learning framework (e.g., convolutional neural networks), that can utilize geometric information from 3D CAD models in combination with real-world statistics from photographs. Then, I will show how to use differentiable physics simulators to learn object properties simply by watching videos.

Biography: Joseph Lim is a postdoctoral researcher at Stanford University. He received a PhD in Electrical Engineering and Computer Science at Massachusetts Institute of Technology, where he was advised by Professor Antonio Torralba. His research interests are in computer vision and machine learning. He is particularly interested in deep learning, structure learning, and multi-domain data. Joseph graduated with BA in Computer Science from UC Berkeley, where he worked under Professor Jitendra Malik.

Host: CS Department

Location: Olin Hall of Engineering (OHE) - 136

Audiences: Everyone Is Invited

Contact: Assistant to CS chair