How Hollywood AI Helps Creators Go Global

Speaker: Hao Li , CEO/Co-Founder of Pinscreen

Talk Title: How Hollywood AI Helps Creators Go Global

Abstract: In this talk I will talk about our journey on bringing digital humans to life in Hollywood, and how we use AI to democratize technologies for generating and manipulating photorealisitc human faces. I will showcase some use cases of our technologies developed at my startup Pinscreen, and how top film studios and streaming companies have adopted our approach for AI-driven visual effects, including face swapping, actor de-aging, and facial reenactment. One particular scalable application consists of dubbing movies from foreign language into English, where lips are perfectly matched to the translated content using AI. I will show how we are moving from Hollywood adoption to building a consumer product: pindub.ai for any Internet content creator. I will also showcase some our recent research efforts at MBZUAI with focus on more advanced generative AI capabilities and metaverse application such as telepresence using the currently most advanced one-shot head reenactment system.
 
This lecture satisfies requirements for CSCI 591: Research Colloquium.
 

Biography: Hao Li is the CEO and Co-Founder of Pinscreen, a Los Angeles-based startup renowned for its pioneering generative AI technologies in visual effects, dubbing, and digital humans. He is also Professor at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) and the Director of the MBZUAI Metaverse Center. Previously, Hao was a Distinguished Fellow with the Computer Vision Group at UC Berkeley and an Associate Professor of Computer Science at the University of Southern California (USC), where he directed the USC Institute for Creative Technologies. He has also been a visiting professor at Weta Digital, a research lead at Industrial Light & Magic / Lucasfilm, and a postdoctoral fellow at Columbia and Princeton Universities. Hao’s research lies at the intersection of computer vision, computer graphics, and machine learning with a focus on photorealistic human digitization, real-time performance capture, immersive telepresence, and generative video synthesis and manipulation (deepfakes). He is widely recognized for his groundbreaking work in real-time 3D facial tracking, which has led to the technology behind Apple's Animoji, the digital recreation of Paul Walker in the movie Furious 7, and the first real-time deepfake face-swapping technology based on generative AI. His research on facial reenactment has been widely adopted in Hollywood, driving advancements in photorealistic digital doubles and seamless de-aging effects. Additionally, he is known for pioneering production-grade visual dubbing solutions, which have made possible the first fully AI-lip-synced foreign language films and TV shows dubbed into English. Hao and his company, Pinscreen, have earned credits in numerous high-profile motion pictures, including Fallout, Money Heist – Berlin, Slumberland, Free Guy, Blade Runner 2049, Valerian and the City of a Thousand Planets, Furious 7, and The Hobbit: The Battle of the Five Armies. His algorithms on deformable shape registration have not only advanced the field of data-driven facial animation and human body modeling, but also improved the radiation treatment for cancer patients all over the world through his collaboration with C-RAD on surface guided radiation therapy. Hao has received numerous accolades, including being named a Top 35 Innovator Under 35 by MIT Technology Review in 2013. he has been awarded the Google Faculty Award, the Okawa Foundation Research Grant, and the Andrew and Erna Viterbi Early Career Chair. In 2018, he won the Office of Naval Research (ONR) Young Investigator Award, and in 2019, he was named to the DARPA ISAT Study Group. He received the ACM Symposium on Computer Animation Best Paper Award in 2009, the ACM SIGGRAPH Real-Time Live! “Best in Show” Award in 2020, and an Epic Megagrant in 2021. Hao has been a speaker at the World Economic Forum in Davos (2020) and was featured in the inaugural season of Amazon’s documentary series re:MARS Luminaries in 2022. Hao earned his PhD from ETH Zurich and his MSc from the University of Karlsruhe (TH).

Host: Prof. Stefanos Nikolaidis

Location: Kaprielian Hall (KAP) - 267

Audiences: Everyone Is Invited

Contact: Thomas Lord Department of Computer Science