Logo: University of Southern California

Events Calendar

  • PhD Defense- Koki Nagano

    Wed, Apr 19, 2017 @ 10:30 AM - 12:30 PM

    Thomas Lord Department of Computer Science

    University Calendar

    PhD Candidate: Koki Nagano

    Committee: Paul Debevec (CS, chair), Hao Li (CS), Jernej Barbic (CS), Aiichiro Nakano (CS), Michelle Povinelli (EE)

    Title: Multi-scale Dynamic Capture for High Quality Digital Humans

    Time: April 19 (Wednesday) 10:30-12:30pm

    Room: KAP 164


    Digitally creating a virtual human indistinguishable from a real human has been one of the central goals of Computer Graphics, Human-Computer Interaction, and Artificial Intelligence. Such digital characters are not only the primary creative vessel for immersive storytellers and filmmakers, but also a key technology to understand the process of how humans think, see, and communicate in the social environment. In order for digital character creation techniques to be valuable in simulating and understanding humans, the hardest challenge is for them to appear believably realistic from any point of view in any environment, and to behave and interact in a convincing manner.

    Creating a photorealistic rendering of a digital avatar is increasingly more accessible due to rapid advancement in sensing technologies and rendering techniques. However, generating realistic movement and dynamic details that are compatible with such a photorealistic appearance still relies on manual work from experts, which hinders the potential impact of digital avatar technologies in real world applications. Generating dynamic details is especially important for facial animation, as humans are extremely tuned to sense people's intentions from facial expressions.

    In this dissertation, we propose systems and approaches for capturing the appearance and motion to reproduce high fidelity digital avatars that are rich in subtle motion and appearance details. We aim for a framework which can generate consistent dynamic detail and motion at the resolution of skin pores and fine wrinkles, and can provide extremely high resolution microstructure deformation for use in cinematic storytelling or immersive virtual reality environments.

    This thesis presents three principal techniques for achieving multi-scale dynamic capture for digital humans. The first is a multi-view capture system and a stereo reconstruction technique which directly produces a complete high-fidelity head model with consistent facial mesh topology. Our method jointly solves for stereo constraints and consistent mesh parameterization from static scans or a dynamic performance, producing dense correspondences on an artist quality template. Additionally, we propose a technique to add dynamic per-frame high and middle frequency details from the flat-lit performance video. Second, we propose a technique to estimate high fidelity 3D scene flow from multiview video. The motion estimation fully respects high quality data from multiview input, and can be incorporated to any facial performance capture pipeline to improve the fidelity of the facial motion. Since the motion can be estimated without relying on any domain-specific priors or regularization, our method scales well to modern systems with many high-resolution cameras. Third, we present a technique to synthesize dynamic skin microstructure details to produce convincing facial animation. We measure and quantify how skin microstructure deformation contributes to dynamic skin appearance, and present an efficient way to simulate dynamic skin microstructure. When combined with the state-of-the art performance capture and face scanning techniques, it can significantly improve the realism of animated faces for virtual reality, video games, and visual effects.

    Location: Kaprielian Hall (KAP) - 164

    Audiences: Everyone Is Invited

    Contact: Lizsl De Leon

    Add to Google CalendarDownload ICS File for OutlookDownload iCal File

Return to Calendar