Events Calendar

Inferring Speaker Affect in Spoken Natural Language Communication

Speaker: Heather Pon-Barry, Harvard University

Talk Title: Inferring Speaker Affect in Spoken Natural Language Communication

Abstract: Speech interfaces are now abundant in our everyday lives. Yet, the experience of talking to a dialogue system or posing queries to a smartphone is not as natural or as efficient as conversing with another human. In this talk, I will present my work, at the intersection of spoken language processing and adaptive intelligent systems, that is aimed at making spoken language communication more natural and efficient. Intelligent speech-enabled systems must move beyond recognizing the words that a person is saying; they must also listen to the person’s manner of speaking and detect any emotions or affective signals that are conveyed. As a concrete example, I will address the problem of inferring a speaker’s level of certainty using prosodic information (intonation, rhythm, and energy) from the speech signal. I will present a technique for computing prosodic features from utterance segments that both improves level of certainty classification and can be used to determine the cause of uncertainty. These models are able to accurately classify speaker certainty 75% of the time. I will also discuss a novel method for eliciting affective speech. This method binds the speaker’s uncertainty to a single phrase within the larger utterance, allowing for the comparison of contextually-based feature sets. This work has applications in a wide range of speech-enabled systems, including tutorial dialogue systems, language-learning tools, and voice search applications.

Biography: Heather Pon-Barry is a Ph.D. candidate in Computer Science at Harvard University. She received her B.S. and M.S. in Symbolic Systems at Stanford University. Her research interests include spoken language processing, artificial intelligence, machine learning, and cognitive science. She is the recipient of NSF and NDSEG graduate research fellowships.

Host: Prof. Shrikanth Narayanan

Location: Ronald Tutor Hall of Engineering (RTH) - 320

Audiences: Everyone Is Invited

Contact: Mary Francis