Events Calendar

PhD Thesis Proposal - Mozhdeh Gheini

PhD Thesis Proposal - Mozhdeh Gheini
 
Committee Members: Jonathan May (Chair), Xiang Ren, Xuezhe Ma, Swabha Swayamdipta, Khalil Iskarous
 
Title: Inductive Biases for Data- and Parameter-Efficient Transfer Learning
 
Abstract: The widespread success of natural language processing (NLP) models, such as Large Language Models, and the subsequent attention from the public often conceal and distract from the sheer amount of data and computational resources they have relied on to reach this point. The very same models often fail to perform as well in the absence of sufficient data and computational resources. However, how to adjust methods under such constraints remains under-discussed. In this talk, I present work incorporating inductive biases during both pretraining and downstream transfer learning and showcase the boosted performance for machine translation and named entity recognition under resource limitations. Following that, I discuss our work on creating a pretrained model using MEGA, a novel architecture with extensions to Transformers, and our ongoing efforts to investigate MEGA's inductive biases that significantly set it apart from Transformer in low-resource scenarios

Location: Ronald Tutor Hall of Engineering (RTH) - 306

Audiences: Everyone Is Invited

Contact: Melissa Ochoa

Event Link: https://usc.zoom.us/j/6564802162