Events Calendar

AI Seminar

Speaker: Yisong Yue, Cal Tech

Talk Title: A Decision Tree Framework for Data-Driven Speech Animation

Abstract: n many animation projects, the animation artist typically spends significant time animating the face, which involves many labor-intensive tasks that offer little potential for creative expression. One particularly tedious task is speech animation: animating the face to match spoken audio. Indeed, the often prohibitive cost of speech animation has limited the types of animations that are feasible, including localization to different languages.

In this talk, I will show how to view speech animation through the lens of data-driven sequence prediction. In contrast to previous sequence prediction settings, speech animation is an instance of contextual spatiotemporal sequence prediction, where the output is continuous and high-dimensional (e.g., a configuration of the lower face), and also depends on an input context (e.g., audio or phonetic input).

I will present a decision tree framework for learning to generate context-dependent spatiotemporal sequences given training data. This approach enjoys several attractive properties, including ease of training, fast performance at test time, and the ability to robustly tolerate corrupted training data using a novel latent variable approach. I will showcase this approach in a case study on speech animation, where our approach outperforms several competitive baselines in both quantitative and qualitative evaluations, and also demonstrates strong robustness to corrupted training data.

This is joint work with Taehwan Kim, Sarah Taylor, Barry-John Theobald, and Iain Matthews.

Biography: Yisong Yue is an assistant professor in the Computing and Mathematical Sciences Department at the California Institute of Technology. He was previously a research scientist at Disney Research. Before that, he was a postdoctoral researcher in the Machine Learning Department and the iLab at Carnegie Mellon University. He received a Ph.D. fromCornell University and a B.S. from the University of Illinois at Urbana-Champaign.

Yisong's research interests lie primarily in the theory and application of statistical machine learning. He is particularly interested in developing novel methods for spatiotemporal reasoning, structured prediction, interactive learning systems, and learning with humans in the loop. In the past, his research has been applied to information retrieval, recommender systems, text classification, learning from rich user interfaces, analyzing implicit human feedback, data-driven animation, sports analytics, policy learning in robotics, and adaptive routing & allocation problems.

Host: Ashish Vaswani

More Info: http://webcasterms1.isi.edu/mediasite/SilverlightPlayer/Default.aspx?peid=6147027e077e4570919a58730193abf91d

Location: Information Science Institute (ISI) - 11th floor large conference room

Audiences: Everyone Is Invited

Contact: Kary LAU

Event Link: http://webcasterms1.isi.edu/mediasite/SilverlightPlayer/Default.aspx?peid=6147027e077e4570919a58730193abf91d