Conferences, Lectures, & Seminars
Events for August
-
NL Seminar-Fair Comparisons and Fundamental Ideas for Open Vocabulary Generative Language and Translation Models
Thu, Aug 12, 2021 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Sabrina Mielke, Johns Hopkins Univ
Talk Title: Fair Comparisons and Fundamental Ideas for Open-Vocabulary Generative Language and Translation Models
Series: NL Seminar
Abstract: REMINDER Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you're highly encouraged to use your USC account to sign into Zoom. If you're an outside visitor, please inform nlg DASH seminar DASH admin2 AT isi.edu beforehand so we'll be aware of your attendance and let you in.
ABSTRACT:
How can we fairly compare the performance of generative language and translation models on multiple languages? We will see how to use probabilistic and information theory based measures, first to evaluate monolingual open vocabulary language models by total bits and then, considering the case of Translationese, pondering the meaning of information and how to use it to compare machine translation models. In both cases, we get a little glimpse at what linguistic and non-linguistic factors might make languages easier or harder for models. The last part of the talk will if time permits propose some somewhat opinionated guidelines for open-vocabulary language modeling, and show work in progress in taxonomizing tokenization methods and the literature around open vocabulary modeling.
Biography: Sabrina is a PhD student at the Johns Hopkins University and a part-time research intern at HuggingFace, currently researching open vocabulary language modeling for unit discovery in a variety of typologically varying languages. While her pre PhD work focused on formal language theory applied to parsing and translation, during her PhD she published on morphology, fair language model comparison, stochastic romanization at Google AI, and metacognition and calibration for chatbots at Facebook AI Research, co organized workshops and shared tasks around morphology and typology, and is currently involved in the BigScience summer of large language models workshop.
Host: Jon May and Mozhdeh Gheini
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=zIP8XMCtHuMLocation: Information Science Institute (ISI) - Virtual Only
WebCast Link: https://www.youtube.com/watch?v=zIP8XMCtHuM
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar-1.Leveraging Abstract Meaning Representations to Amplify the Semantic Information Captured in Transformer Models 2. Improving Multilingual Encoder With Contrastive Objective and Luna
Thu, Aug 19, 2021 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: 1.Shira Wein 2. Leo Zeyu Liu, ISI Interns
Talk Title: 1.Leveraging Abstract Meaning Representations to Amplify the Semantic Information Captured in Transformer Models 2. Improving Multilingual Encoder With Contrastive Objective and Luna
Series: NL Seminar
Abstract: REMINDER Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you're highly encouraged to use your USC account to sign into Zoom. If you're an outside visitor, please inform nlg DASH seminar DASH admin2 AT isi.edu beforehand so we'll be aware of your attendance and let you in.
SHIRA WEIN
Though state of the art language models perform well on a variety of natural language processing tasks, these models are not exposed to explicit semantic information. We propose that language models ability to capture semantic information can be improved through the inclusion of explicit semantic information in the form of meaning representations, thus improving performance on select downstream tasks. We discuss potential ways to incorporate meaning representations and present our preliminary results.
LEO ZEYU LIU
Transformers has been successfully adapted to multilingual pretraining. With only token level losses like masked language model, transformer encoder could produce good token and sentence representations. We propose to explicitly impose sentence level objectives using contrastive learning to further improve multilingual encoder. Furthermore, we also propose to merge this modification with what a new transformer architecture, Luna, could offer disentanglement between token and sentence representations. We will also discuss ways to evaluate the models and present our experimental progress.
Biography: Shira Wein is an intern at ISI and a third year Ph.D. student at Georgetown University, working on semantic representations and multilingual cross lingual applications. Her previous work centers around L2 corpora, Abstract Meaning Representations, and information extraction from design documents, which she published on while interning at the Jet Propulsion Lab. Prior to starting her Ph.D., Shira was an undergraduate at Lafayette College, where she received a B.S. in Computer Science and B.A. in Spanish.
Leo Zeyu Liu is a Master student in Computer Science at the University of Washington, advised by Noah A. Smith and Shane Steinert Threlkeld. His research aims at interpretability, pretraining, and intersection between NLP and Linguistics. He completed his bachelor in Computer Science at the University of Washington.
Host: Jon May and Mozhdeh Gheini
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://usc.zoom.us/j/93331739032Location: Information Science Institute (ISI) - Virtual
WebCast Link: https://usc.zoom.us/j/93331739032
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar-FROM CONSTRAINED EVENT SEQUENCES GENERATION TO TEXT GENERATION
Thu, Aug 26, 2021 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Shanxiu He, ISI Intern
Talk Title: FROM CONSTRAINED EVENT SEQUENCES GENERATION TO TEXT GENERATION
Abstract: REMINDER: Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you're highly encouraged to use your USC account to sign into Zoom. If you're an outside visitor, please inform nlg DASH seminar DASH admin2 AT isi.edu beforehand so we'll be aware of your attendance and let you in.
Understanding events is a critical component of natural language understanding NLU. A key challenge lies in the fact that events can be described in different granularities. A coarse grained event e.g., publishing a paper can often be disseminated into a fine grained process of events e.g., writing the paper, passing the peer review, and presenting at the conference. In this work, we tackle the problem of goal oriented event process generation, where a task goal event, a process that completes this goal is automatically generated. We tackle this task with a constrained generation approach, inferring unobserved event chains based on existing sequences. To leverage prior knowledge to facilitate commonsense reasoning, we employ pre trained LMs to generate event sequences and to retrieve original stories.
Biography: Shanxiu He is an undergraduate at UCLA and a member of UCLANLP lab. Prior to the internship, her research interest focuses on pre rained Vision and Language models such as VisualBERT and ClipBert and their applications to various structural learning tasks. During this internship, she researches on event centric knowledge representation and specifically event sequences generations.
Host: Jon May and Mozhdeh Gheini
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=0ZG9d51emsILocation: Information Science Institute (ISI) - Virtual Only
WebCast Link: https://www.youtube.com/watch?v=0ZG9d51emsI
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor.
