Fri, Oct 06, 2023 @ 02:00 PM - 04:00 PM
Thomas Lord Department of Computer Science
PhD Dissertation Defense - Peifeng Wang
Committee Members:Xiang Ren (chair), Muhao Chen, Emilio Ferrara, Dan O Leary
Title: Externalized Reasoning in Language Models for Scalable, Trustworthy AI
Abstract: Language models LMs have traditionally intertwined language skills and reasoning abilities within a single architecture, relying on reckless scaling for effectiveness, but sacrificing interpretability due to their opaque nature. This presentation advocates for externalizing reasoning from the core language proficiency, which enables the development of compact, accessible LMs with enhanced capabilities. The process of externalized reasoning also offers a pathway to explaining model behavior.
I would present three techniques to build LMs with externalized reasoning. First, I introduce an Imagine and Verbalize framework for generative commonsense reasoning, which decomposes a complex generation task into easier subtasks and learns from a diverse set of indirect supervision from multiple domains. Second, I present a knowledge distillation pipeline which prompts a large LM to rationalize for an open domain question and then trains a small LM to reason consistently. Third, I discuss augmenting an LM with working memory for coherent language reasoning by tracking the states of the described world.
Audiences: Everyone Is Invited
Contact: Melissa Ochoa