Events Calendar

CS Colloquium: Yisong Yue (Caltech) - A Decision Tree Framework for Data-Driven Speech Animation

Speaker: Yisong Yue, Caltech

Talk Title: A Decision Tree Framework for Data-Driven Speech Animation

Series: CS Colloquium

Abstract: This lecture satisfies requirements for CSCI 591: Computer Science Research Colloquium

In many animation projects, the animation artist typically spends significant time animating the face. This process involves many labor-intensive tasks that offer relatively little potential for creative expression. One particularly tedious task is speech animation: animating the face to match spoken audio. Indeed, the often prohibitive cost of speech animation has limited the types of animations that are feasible, including localization to different languages.

In this talk, I will show how to view speech animation through the lens of data-driven sequence prediction. In contrast to previous sequence prediction settings, visual speech animation is an instance of contextual spatiotemporal sequence prediction, where the output is continuous and high-dimensional (e.g., a configuration of the lower face), and also depends on an input context (e.g., audio or phonetic input).

I will present a decision tree framework for learning to generate context-dependent spatiotemporal sequences given training data. This approach enjoys several attractive properties, including ease of training, fast performance at test time, and the ability to robustly tolerate corrupted training data using a novel latent variable approach. I will showcase this approach in a case study on speech animation, where our approach outperforms several competitive baselines in both quantitative and qualitative evaluations, and also demonstrates strong robustness to corrupted training data.

This is joint work with Taehwan Kim, Sarah Taylor, Barry-John Theobald and Iain Matthews.

The lecture will be available to stream HERE.

Host: Yan Liu

Webcast: https://bluejeans.com/535615811

Location: Henry Salvatori Computer Science Center (SAL) - 101

WebCast Link: https://bluejeans.com/535615811

Audiences: Everyone Is Invited

Contact: Assistant to CS chair