Events Calendar

PhD Dissertation Defense - Zimo Li

PhD Dissertation Defense - Zimo Li

Committee Members: Andrew Nealen, Laurent Itti, Stefanos Nikolaidis, Mike Zyda

Title: Human Appearance and Performance Synthesis Using Deep Learnin

Abstract: Synthesis of human performances is a highly sought after technology in the entertainment industry. In this dissertation, we will go over several new deep learning solutions which tackle the problems of human facial and body performance synthesis.

Facial performance synthesis is a complex multistep graphics problem. First, the target performance to be modified must be tracked and captured accurately. Then, based on the desired modification (whether to change the identity, facial expressions, or both), a modified source performance must be synthesized or captured from a different actor. Finally, the original facial performance must be removed and replaced with the synthesized one. This multistep process poses many unique challenges. Using conventional CG tracking and retargeting of expressions from the source to target using a 3D mesh and static texture will give an undesired rubbery skin effect. Furthermore, inaccuracies in the expression tracking of the source performance using a blendshape model will result in the uncanny valley effect in the output performance. It is often necessary to use costly capture methods, such as a Light Stage, to obtain highly accurate 3D captures and dynamic textures of a source performance in order to avoid these pitfalls. Even then, final modified performances are often uncanny.

When dealing with human body to motion synthesis, creating new motions often requires manual artist animations, tracking new motions on an actor, or stitching together subsequences of previous animations. These methods are limited by cost, or are not able to generate appreciably novel motions.

Over the last several years, the advancement of AI based generation techniques have let us address many of these issues. In this thesis, we will go over several novel techniques which reduce the cost (time, money, ease-of-access), and improve the quality of facial reenactment, as well as body motion synthesis, pipelines. The applications of these techniques allow us to tackle new problem settings in an efficient way.

Audiences: Everyone Is Invited

Contact: Melissa Ochoa

Event Link: https://us05web.zoom.us/j/86385849747?pwd=V2lwR2FXekI5WVpNMGU0bWF5clJIQT09