Thu, Apr 27, 2023 @ 01:00 PM - 03:00 PM
Thomas Lord Department of Computer Science
PhD Dissertation Defense - Lauren Klein
Committee Members: Maja Mataric (chair), Pat Levitt, Shrikanth Narayanan, Mohammad Soleymani, and Jesse Thomason
Title: Modeling Dyadic Synchrony with Heterogeneous Data: Validation in Infant Mother and Infant Robot Interaction
Abstract: Our health and wellbeing are intricately tied to the dynamics of our social interactions, or social synchrony. The key components of social synchrony during embodied interactions are temporal behavior adaptation, joint attention, and shared affective states. To create comprehensive representations of nuanced social interactions, computational models of social synchrony must account for each of these components.
The goal of this dissertation is to develop and evaluate approaches for modeling social synchrony during embodied dyadic interactions. We present computational models of social synchrony in two contexts. First, we explore human to human social interactions, where attention and affective states must be inferred through behavioral observations. During embodied interactions, social partners communicate using a diverse range of behaviors, therefore, this work develops approaches for modeling temporal behavior adaptation using heterogeneous data, or data representing multiple behavior types. Next, we explore social synchrony in the context of human to robot interaction. Robots must be equipped with perception modules to establish joint attention and shared affective states based on information about their partners behaviors. To address this need, we develop and evaluate models for attention and affective state recognition. Given the central role of communication in cognitive and social development, this dissertation focuses on interactions that occur during infancy and early childhood. Specifically, we develop and evaluate our approaches using recordings of infant to mother, infant to robot, and child to robot interactions.
The work presented in this dissertation for evaluating and supporting social synchrony enables new opportunities to study the relationships between individual behaviors, joint interaction states, and developmental and health outcomes.
Audiences: Everyone Is Invited
Contact: Melissa Ochoa