Events Calendar

PHD Thesis Proposal (Meryem M'Hamdi)

Presentation Title: Towards More Human-Like Cross-lingual Transfer Learning

Abstract: Since their release, multilingual off-the-shelf representations such as M-BERT and other Transformer-based variants has gained tremendous popularity. Despite exhibiting surprisingly good zero-shot performance, they are often pre-trained and fine-tuned in a data-intensive manner and are less robust against data distribution shifts which is orthogonal to how humans learn. In this thesis proposal, we analyze and propose techniques to advance the capabilities of multilingual language models beyond this data-intensive identically distributed paradigm and more towards human-like cross-lingual transfer learning. We achieve that through human-inspired input requirements by adapting few-shot meta-learning approaches, human-inspired outcomes by understanding what it means to learn continually over a stream of languages, and cognitive human-learning strategies like spaced repetition to consolidate retention of knowledge learned across languages. We apply our techniques to information extraction, natural language understanding, question answering, and semantic search downstream tasks and analyze on typologically diverse benchmarks.


Committee Members: Jonathan May (Chair), Kallirroi Georgila, Xuezhe Ma, Shrikanth Narayanan, Aiichiro Nakano

Location: https://usc.zoom.us/j/94778821094?pwd=cFZISUdZZ0trUlpMNFdGSEE0TDExdz09

Audiences: Everyone Is Invited

Contact: Asiroh Cham