Tue, Apr 25, 2023 @ 03:00 PM - 04:30 PM
Thomas Lord Department of Computer Science
PHD Thesis Proposal - Peifeng Wang
Title: Building Small-Scale but Advanced Language Reasoners
The entanglement of multiple language capabilities within large language models requires expensive scaling to work effectively. I argue that a disassociation of these capabilities from core language skills can enable the creation of smaller, more accessible language models. Additionally, this disassociation will facilitate the development of language models with enhanced reasoning abilities.
This thesis proposal presents three techniques to build small language models with advanced reasoning capabilities. First, I introduce an Imagine&Verbalize framework for generative commonsense reasoning, which decomposes a complex generation task into easier sub-tasks and learns from a diverse set of indirect supervision from multiple domains. Second, I present a knowledge-transferring pipeline which prompts large language models to rationalize for an open-domain question and then trains small language models to answer consistently. Third, I discuss augmenting small LMs with a working memory for coherent language reasoning by tracking the states of the described world.
Venue: zoom at https://usc.zoom.us/j/97850702935?pwd=ekJ0K1RMM045Tk1EQUV1OUEvOE5iQT09
Date and time: 3:00pm-4:30pm on April 25th
Committee Members: Xiang Ren (chair), Filip Ilievski, Swabha Swayamdipta, Ram Nevatia, Emilio Ferrara
Audiences: Everyone Is Invited
Contact: Asiroh Cham