Andrew and Erna Viterbi Early Career Chair and Assistant Professor of Computer Science
- Doctoral Degree, Computer Science, Eidgenossische Technische Hochschule
- Master's Degree, Computer Science, University of Karlsruhe
I am a tenure-track assistant professor, fashion mödel, and zombie at the Computer Science Department of the University of Southern California (USC) in Los Angeles and CEO of Pinscreen, a well-funded startup bringing AR to mobile communication. Here is our USC website and my official song! Before my faculty appointment, I was a research lead at Industrial Light & Magic (ILM) where I developed the next generation real-time performance capture technology for visual effects. Prior to joining the force, I spent an exciting year as a postdoc at Columbia and Princeton Universities. I worked on geometric capture of human performances with Eitan Grinspun and Szymon Rusinkiewicz who helped me bridge the gap between data capture and physical simulation. Right after obtaining my PhD from ETH Zurich, I spent a few months at EPFL, visiting my PhD advisor Mark Pauly. During grad school, I spent a wonderful summer atILM and another one at Stanford University visiting Leo Guibas. I got my Master’s degree in CS from theUniversity of Karlsruhe in Germany and went to a French high school in a boonie town called Saarbrücken.
My algorithms on dynamic shape reconstruction and non-rigid registration are widely deployed in the industry, ranging from leading visual effects studios to defense projects and manufacturers of state-of-the-art radiation therapy systems. Hey, some of my stuff even works in industry! I received the Google Faculty Research Award, the Okawa Foundation Research Grant, and the Andrew and Erna Viterbi Early Career Chair in 2015. I’ve been named one of the world’s top 35 innovator under 35 by MIT Technology review in 2013 and NextGen 10: Innovators under 40 by C-Suite Quaterly in 2014. I am ranked #1 in 2016, on the Top 10 Leaderboard of Computer Graphics research for the past five years by Microsoft Academic. My research is also featured in the March 2014 issue of Wired magazine. I was awarded the SNF fellowship for prospective researchers in 2011 and Best Paper Award at SCA 2009. Our 3D hair scanning technology is deployed at Weta Digital, where I also worked on the 3D facial tracking for animating the digital Paul Walkerin Furious 7. Oh! That gives me an Erdös-Bacon number of 5. In collaboration with Artec Group, we developed Shapify.me, a free software that allows anyone to create their own 3D miniature from home. I’m also the creator of the dynamic geometry processing tool, BeNTO3D, and the original co-author of the Kinect-based facial performance capture software, faceshift. Together with Oculus & Facebook, we prototyped the world’s first facial performance sensing virtual reality HMD to enable social interactions in cyberspace. Here is my interview on this technology from Oculus Connect 2. I’m member of the New York Academy of Sciences, ACM SIGGRAPH, and Eurographics Association.
With 3D imaging being democratized, I develop algorithms that enable unobtrusive digitization, processing, and understanding of dynamic subjects from the physical world, such as humans and animals.
By the end of this decade, our homes will be equipped with 3D sensors that digitally monitor our actions, habits, and health. In contrast to video sensors, 3D cameras directly capture spatial representations of a scene and facilitate many unsolved computer vision problems. I develop efficient algorithms and computational models for unobtrusive 3D digitization of dynamic shapes. My goal is to capture human body performances and facial expressions at unprecedented resolution and in unconstrained settings. These ad- vances will help machines understand our movements and intentions, revolutionizing the way we interact with computers through natural gestures, and developing new forms of live communication through com- pelling virtual avatars. My research lies at the intersection of computer graphics, vision, and machine learning, with immediate impact in entertainment (film/game production), health (unobtrusive diagnostics), and social science (behavioral analytics).
- 2014 C-Suite Quaterly NextGen 10: Innovators under 40
- 2013 MIT Technology Review TR35, top 35 innovator under 35
- 2011 Swiss National Science Foundation SNF fellowship for prospective researcher
- 2009 ACM SIGGRAPH / EUROGRAPHICS SCA 2009 Best Paper Award