Events Calendar

Situated Spoken Language Interaction: Challenges and Opportunities

Speaker: Dan Bohus, Microsoft Research

Talk Title: Situated Spoken Language Interaction: Challenges and Opportunities

Abstract: Previous research on spoken language interaction has mostly focused on dialog with single users in limited, predefined contexts. Efforts in this space have led to the development and wide-scale deployment of telephony-based and, more recently, multimodal mobile applications. At the same time, numerous and important challenges in the realm of physically situated, open-world interaction are still largely unaddressed.

In this talk, I will present an overview of the Situated Interaction project at Microsoft Research, which aims to address such challenges: the long term goal is to enable a new generation of interactive systems that reason more deeply about their surroundings and embed spoken interaction into the natural flow of everyday tasks, activities and collaborations.

I will outline a set of core competencies - engagement, turn taking, understanding and interaction planning - that are essential for supporting physically situated spoken language interactions. Developing computational models for supporting these processes in open, relatively unconstrained environments brings to the fore many challenges in representation, multimodal inference and decision making. I will illustrate and discuss some of these challenges by reviewing samples of research work we have conducted in these areas, addressing a diverse set of questions such as: how to represent and model engagement in multiparty open-world settings, how to learn to make inferences about speakers and addressees from raw streams of evidence, how to make fast-paced turn-taking decisions by taking into account uncertainty over the world and over the system’s own computational delays. Throughout the talk, I will discuss and showcase several prototype systems that we have developed to date, and highlight several other on-going research efforts in physically situated spoken language interaction.


Biography: : Dan Bohus is a Researcher in the Adaptive Systems and Interaction Group at Microsoft Research. The central question that drives his long term research agenda is: how do we develop systems that naturally embed interaction and computation deeply into the flow of everyday tasks, activities, and collaborations? Specifically, in the last few years Dan’s work has focused on developing computational models for multiparty engagement, turn taking, interaction planning, and on addressing the challenges in inference and decision making that such models bring to the fore. Prior to joining Microsoft, Dan obtained his Ph.D. degree from Carnegie Mellon University, where he investigated problems of dialog management and error handling in speech interfaces.

Host: Professor Shrikanth Narayanan

Location: Ronald Tutor Hall of Engineering (RTH) - 320

Audiences: Everyone Is Invited

Contact: Talyia Veal