Events Calendar

AI Seminar- Uncovering meaning construction and representation in the reading brain

Speaker: Leila Wehbe , CMU

Talk Title: Uncovering meaning construction and representation in the reading brain

Series: Artificial Intelligence Seminar

Abstract: How is information organized in the brain when it reads? Where and when do the required processes occur, such as perceiving the individual words, combining them with the previous words and maintaining a representation of the overall meaning?

I will present results from a recent experiment in which we align context-based neural network language models and brain activity during reading. When processing a text word by word, both the brain and the neural networks perform the same processes. They both maintain a representation for the previous context. They both represent the properties of the incoming word and then integrate it with context. We study the alignment between the latent vectors used by these neural networks and the brain activity observed via Magnetoencephalography (MEG) when subjects read a chapter from Harry Potter and the Sorcerer’s Stone. For that purpose we apply the neural network to the same chapter the subjects are reading, and explore the ability of these vector representations to predict the observed word-by-word brain activity.

Our novel results include a suggested time-line of how the brain updates its representation of context. They also demonstrate the incremental perception of every new word starting early in the visual cortex, moving next to the temporal lobes and finally to the frontal regions. Furthermore, the results suggest the integration process occurs in the temporal lobes after the new word has been perceived.

This is joint work with Ashish Vaswani, Kevin Knight and Tom Mitchell, and is a part of a larger effort to understand how the brain organizes information in natural reading. I will describe this research direction. I will also mention results from a sister experiment in which we demonstrate how the brain areas involved in reading are processing different types of information (such as syntax, semantics or narrative information). This second experiment is joint work with Brian Murphy, Partha Talukdar, Alona Fyshe, Aaditya Ramdas and Tom Mitchell. Finally I will also mention some of our past and upcoming machine learning projects that aim to improve our methodological pipeline.







Biography: Leila is a PhD student supervised by Professor Tom Mitchell in the Machine Learning Department at Carnegie Mellon University. She is part of the dual-track program in the Center for the Neural Basis of Cognition. She received her BE in Electrical and Computer Engineering from the American University of Beirut.

HomePage Link:
http://www.cs.cmu.edu/~lwehbe/

Host: Kevin Knight

Webcast: http://webcasterms1.isi.edu/mediasite/Viewer/?peid=e1bbe0a29af44690aa0eb5ae9c9f93081d

Location: Information Science Institute (ISI) - 11th Flr Conf Rm # 1135, Marina Del Rey

WebCast Link: http://webcasterms1.isi.edu/mediasite/Viewer/?peid=e1bbe0a29af44690aa0eb5ae9c9f93081d

Audiences: Everyone Is Invited

Contact: Peter Zamar