The Science and Technology of a Virtual Tutor: How Baldi Came to Be

Abstract:
Speech and language science and technology evolved under the assumption that speech was a solely auditory event. However, a burgeoning record of research findings reveals that our perception and understanding are
influenced by a speaker's face and accompanying gestures, as well as the actual sound of the speech. Perceivers expertly use these multiple sources of information to identify and interpret the language input. This behavior is
accurately described by our Fuzzy Logical Model of Perception. Extant issues in the processing of auditory and visual speech will be addressed. Given the value of face-to-face interaction and theoretical framework, our persistent goal has been to develop, evaluate, and apply animated agents to produce realistic and accurate speech. Baldi is an accurate three-dimensional animated talking head appropriately aligned with either synthesized or natural speech. Baldi has a realistic tongue and palate, which can be displayed by making his skin transparent.Based on this research and technology, we have implemented computer-assisted speech and language tutors for children with language challenges and persons learning a second language. Our language-training program utilizes Baldi as the conversational agent, who guides students through a variety of exercises designed to teach vocabulary and grammar, to improve speech articulation, and to develop linguistic and phonological awareness. Some of the advantages of the Baldi pedagogy and technology include the popularity and effectiveness of computers and embodied conversational agents, the perpetual availability of the program, and individualized instruction. The science and technology of Baldi holds great promise in language learning, dialog, human-machine interaction, education, and edutainment.Bio:
Dominic W. Massaro is Professor of Psychology and Computer Engineering, director of the Perceptual Science Laboratory, and founding Chair of Digital Arts and New Media M.F.A. program at the University of California, Santa Cruz. He has been a Guggenheim Fellow, a University of Wisconsin Romnes Fellow, a James McKeen Cat tell Fellow, and an NIMH Fellow. He is a past president of the Society for Computers in Psychology, and is currently the book review editor of the American Journal of Psychology and founding co-editor of the journal Interpret ing. He has published numerous academic journal articles, writ ten and edited several books (including Perceiving talking faces: from speech perception to a behavioral principle, Cambridge, Massachusetts: MIT Press; The Science of the Mind: 2001 and Beyond, New York: Oxford University Press; and Experimental Psychology: An information processing approach, Orlando, FL: Harcourt Brace Jovanovich.). His research uses a formal experimental and theoretical approach to the study of speech perception, reading, psycholinguistics, memory, cognition, learning, and decision-making. One focus of his current research is on the development and theoretical and applied use of a completely synthetic and animated head for speech synthesis, language tutoring, and edutainment.Host: Professor Shrikanth Narayanan

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

Audiences: Everyone Is Invited

Contact: Mary Francis