Events Calendar

PHD Thesis Proposal - Peifeng Wang

PHD Thesis Proposal - Peifeng Wang

Title: Building Small-Scale but Advanced Language Reasoners

Abstract:
The entanglement of multiple language capabilities within large language models requires expensive scaling to work effectively. I argue that a disassociation of these capabilities from core language skills can enable the creation of smaller, more accessible language models. Additionally, this disassociation will facilitate the development of language models with enhanced reasoning abilities.

This thesis proposal presents three techniques to build small language models with advanced reasoning capabilities. First, I introduce an Imagine&Verbalize framework for generative commonsense reasoning, which decomposes a complex generation task into easier sub-tasks and learns from a diverse set of indirect supervision from multiple domains. Second, I present a knowledge-transferring pipeline which prompts large language models to rationalize for an open-domain question and then trains small language models to answer consistently. Third, I discuss augmenting small LMs with a working memory for coherent language reasoning by tracking the states of the described world.



Venue: zoom at https://usc.zoom.us/j/97850702935?pwd=ekJ0K1RMM045Tk1EQUV1OUEvOE5iQT09

Date and time: 3:00pm-4:30pm on April 25th

Committee Members: Xiang Ren (chair), Filip Ilievski, Swabha Swayamdipta, Ram Nevatia, Emilio Ferrara

Location: https://usc.zoom.us/j/97850702935?pwd=ekJ0K1RMM045Tk1EQUV1OUEvOE5iQT09

Audiences: Everyone Is Invited

Contact: Asiroh Cham

Event Link: https://usc.zoom.us/j/97850702935?pwd=ekJ0K1RMM045Tk1EQUV1OUEvOE5iQT09