NL SEMINAR Improving Dialogue Agents with a Social Dimension and Understanding Monolingual Pre-Training for Bilingual Models
Thu, Aug 13, 2020 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Naitian Zhou and Omar Shaikh , ISI interns
Talk Title: Improving Dialogue Agents with a Social Dimension and Understanding Monolingual Pre-Training for Bilingual Models
Abstract: Naitian Zhou
TITLE Improving Dialogue Agents with a Social Dimension
The dialogue problem is challenging because a proper response must be conditioned on many different factors: knowledge about the language, knowledge about the world, knowledge about self, and knowledge about the speaker, to name a few. Prior research has focused on language modeling and "persona" modeling, encoding facts about the dialogue agent. In this project, I try to focus on an alternative dimension to dialogue: how do we condition the way we converse, based on our understanding of ourselves and our social relationship with our dialogue partner?
TITLE Understanding Monolingual Pre Training for Bilingual Models
Monolingual embeddings from models like BERT are known to help on a variety of downstream tasks in a straightforward way. Usually, these embeddings are plug and play initializing models with BERT embeddings or using them as input representations result in increased model performance. However, supervised NMT tasks do not appear to benefit equally from traditional pretraining methods. We explore what makes NMT bilingual and BERT LM monolingual representations different on several probing tasks, and why certain training methods succeed in extracting performance from BERT embeddings from NMT tasks.
Biography: 1. Naitian Zhou is a rising junior at the University of Michigan studying computer science and data science. His interests include computational social science and natural language processing.
2. Omar is a Summer 2020 intern with Dr. Jon May. He is also a rising junior at Georgia Institute of Technology.
Host: Jon May and Emily Sheng
More Info: https://nlg.isi.edu/nl-seminar/
WebCast Link: https://youtu.be/QS3NVSbunt8
Audiences: Everyone Is Invited
Contact: Petet Zamar