Thu, Apr 15, 2021 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: R. Thomas McCoy, JHU
Talk Title: Universal linguistic inductive biases via meta learning
Series: NL Seminar
Abstract: Despite their impressive scores on NLP leaderboards, current neural models fall short of humans in two major ways. They require massive amounts of training data, and they generalize poorly to novel types of examples. To address these problems, we propose an approach for giving targeted linguistic inductive biases to a model, where inductive biases are factors that affect how a learner generalizes. Our approach imparts inductive biases using meta learning, a procedure through which the model discovers how to acquire new languages more quickly via exposure to many possible languages. By controlling the properties of the languages used during meta learning, we can control the inductive biases that meta learning imparts. Using a case study from phonology, we show how this approach enables faster learning and more robust generalization.
Biography: Tom McCoy is a PhD student in the Johns Hopkins Cognitive Science department, advised by Tal Linzen and Paul Smolensky. He studies the linguistic abilities of neural networks, focusing on inductive biases the topic of this talk as well as compositional structure: How can neural networks use their continuous vector representations to encode phrases and sentences?
Host: Jon May and Mozhdeh Gheini
More Info: https://nlg.isi.edu/nl-seminar/
WebCast Link: https://www.youtube.com/watch?v=teu6Tr7e7Ns
Audiences: Everyone Is Invited
Contact: Petet Zamar