Logo: University of Southern California

Events Calendar

  • PhD Thesis Defense - Meryem M'Hamdi

    Tue, Jan 16, 2024 @ 10:00 AM - 12:00 PM

    Thomas Lord Department of Computer Science

    University Calendar

    PhD Thesis Defense - Meryem MHamdi       
    Title: Towards More Human-Like Cross-lingual Transfer Learning      
    Committee Members: Jonathan May (Chair), Aiichiro Nakano, Khalil Iskarous.    
    Abstract: Cross-lingual transfer learning comprises a set of techniques used to adapt a model trained on (a) source language(s), enabling it to generalize to new target languages. With the emergence of Transformer-based contextualized encoders, there has been a surge in multilingual representations that adapt these encoders to various cross-lingual downstream applications. The surprising zero-shot capabilities of these encoders make them promising substitutes for other fully supervised techniques, bypassing the need for large-scale annotation. However, these representations are still far from solving the long-tail of NLP phenomenon, where models are biased more towards high-resource and typologically similar languages to the source language. This bias can be attributed to the over-reliance of current transfer learning pipelines on what we define as the 'Data-Intensive Identically-Distributed Minimally-Evaluated' paradigm.  In this thesis, we analyze and propose techniques to advance the capabilities of multilingual language models beyond the traditional paradigm and more toward human-like cross-lingual transfer learning. We achieve that through 1) human-inspired input requirements by using few-shot meta-learning techniques, 2) human-inspired outcomes by defining a cross-lingual continual learning evaluation paradigm, and 3) human-inspired approaches through devising cognitive strategies to consolidate retention of knowledge learned across languages. Our contributions towards advancing the current transfer learning paradigm towards human-like learning are four-fold: 1) We explore cross-lingual fine-tuning on low-resource multilingual applications such as event trigger extraction and semantic search, shedding light on the strengths and limitations of existing cross-lingual transfer learning techniques. 2) We propose language-agnostic meta-learning approaches that can further bridge the gap between source and target typologically diverse languages. We show the merits of our approaches in reaching quicker and smoother generalization compared to naive fine-tuning, especially under low-resource scenarios. 3) We are the first to define a lifelong learning paradigm that analyzes language shifts. We show the merits and challenges of a multi-hop analysis where the system continually learns over several languages one at a time. 4) We are the first to adapt a cognitively inspired technique based on Leitner-queues to choose what to repeat in a cross-lingual continual learning setup and investigate its impact on reducing the forgetting of previously learned languages.  

    Location: Henry Salvatori Computer Science Center (SAL) - 213

    Audiences: Everyone Is Invited

    Contact: CS Events

    Event Link: https://usc.zoom.us/j/94477374759?pwd=ajZrd1o0QktVVXZsRk9UL1J6NGdtdz09#success


Return to Calendar