Events Calendar

PhD Defense - Derya Ozkan

Title: Towards Intelligent Virtual Listeners: Computational Models of Social Nonverbal Behaviors

PhD Candidate: Derya Ozkan

Committee:
Louis-Philippe Morency (Chair)
Gerard Medioni
Jonathan Gratch
Stacy Marsella
Shrikanth Narayanan (outside member)


Human nonverbal communication is a highly interactive process, in which the participants dynamically send and respond to nonverbal signals. These signals play a significant role in determining the nature of a social exchange. Although human can naturally recognize, interpret and produce these nonverbal signals in social context, computers are not equipped with such abilities. Therefore, creating computational models for holding fluid interactions with human participants has become an important topic for many research fields including human-computer interaction, robotics, artificial intelligence, and cognitive sciences. Central to the problem of modeling social behaviors is the challenge of understanding the dynamics involved with listener backchannel feedbacks (i.e. the nods and paraverbals such as ``uh-hu'' and ``mm-hmm'' that listeners produce as someone is speaking).

In this thesis, I present a framework for modeling visual backchannels of a listener during a dyadic conversation. I address the four major challenges involved in modeling nonverbal human behaviors, more specifically listener backchannels: (1) high dimensional data, (2) multimodal processing, (3) mutual influence between the participants, and (4) variability in human's behaviors. We address the first challenge by proposing a sparse feature selection method that gives researchers a new tool to analyze human nonverbal communication. To address to second challenge of effective and efficient fusion of multimodal information, we introduce a new model called Latent Mixture of Discriminative Experts (LMDE) that can automatically learn the hidden dynamic between modalities. For the third challenge, we present a context-based prediction framework that models the mutual influence between the participants of a human conversation to improve the final prediction model. Finally, we propose a new approach for modeling wisdom of crowds called wisdom-LMDE, which is able to learn the variations and commonalities among different crowd members.

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

Audiences: Everyone Is Invited

Contact: Lizsl De Leon