Events Calendar

CS Colloquium: Gedas Bertasius (Facebook AI) - Designing Video Models for Human Behavior Understanding

Speaker: Gedas Bertasius, Facebook AI

Talk Title: Designing Video Models for Human Behavior Understanding

Series: CS Colloquium

Abstract: Many modern computer vision applications require extracting core attributes of human behavior such as attention, action, or intention. Extracting such behavioral attributes requires powerful video models that can reason about human behavior directly from raw video data. To design such models we need to answer the following three questions: how do we (1) model videos, (2) learn from videos, and lastly, (3) use videos to predict human behavior?

In this talk I will present a series of methods to answer each of these questions. First, I will introduce TimeSformer, the first convolution-free architecture for video modeling built exclusively with self-attention. It achieves the best reported numbers on major action recognition benchmarks while also being more efficient than state-of-the-art 3D CNNs. Afterwards, I will present COBE, a new large-scale framework for learning contextualized object representations in settings involving human-object interactions. Our approach exploits automatically-transcribed speech narrations from instructional YouTube videos, and it does not require manual annotations. Lastly, I will introduce a self-supervised learning approach for predicting a basketball player's future motion trajectory from an unlabeled collection of first-person basketball videos.

This lecture satisfies requirements for CSCI 591: Research Colloquium

Biography: Gedas Bertasius is a postdoctoral researcher at Facebook AI working on computer vision and machine learning problems. His current research focuses on topics of video understanding, first-person vision, and multi-modal deep learning. He received his Bachelors Degree in Computer Science from Dartmouth College, and a Ph.D. in Computer Science from the University of Pennsylvania. His recent work was nominated for the CPVR 2020 best paper award.

Host: Ramakant Nevatia

Audiences: By invitation only.

Contact: Assistant to CS chair