Logo: University of Southern California

Events Calendar

  • PhD Thesis Proposal - Mozhdeh Gheini

    Tue, Nov 07, 2023 @ 09:00 AM - 11:00 AM

    Thomas Lord Department of Computer Science

    University Calendar

    PhD Thesis Proposal - Mozhdeh Gheini
    Committee Members: Jonathan May (Chair), Xiang Ren, Xuezhe Ma, Swabha Swayamdipta, Khalil Iskarous
    Title: Inductive Biases for Data- and Parameter-Efficient Transfer Learning
    Abstract: The widespread success of natural language processing (NLP) models, such as Large Language Models, and the subsequent attention from the public often conceal and distract from the sheer amount of data and computational resources they have relied on to reach this point. The very same models often fail to perform as well in the absence of sufficient data and computational resources. However, how to adjust methods under such constraints remains under-discussed. In this talk, I present work incorporating inductive biases during both pretraining and downstream transfer learning and showcase the boosted performance for machine translation and named entity recognition under resource limitations. Following that, I discuss our work on creating a pretrained model using MEGA, a novel architecture with extensions to Transformers, and our ongoing efforts to investigate MEGA's inductive biases that significantly set it apart from Transformer in low-resource scenarios

    Location: Ronald Tutor Hall of Engineering (RTH) - 306

    Audiences: Everyone Is Invited

    Contact: Melissa Ochoa

    Event Link: https://usc.zoom.us/j/6564802162

    Add to Google CalendarDownload ICS File for OutlookDownload iCal File

Return to Calendar