PhD Defense - Yuchen Lin
Mon, Dec 05, 2022 @ 10:00 AM - 12:00 PM
PhD Candidate: Yuchen Lin
Title: Evaluating and Improving the Commonsense Reasoning Ability of Language Models
Committee: Xiang Ren (chair), Ram Nevatia, Yan Liu, Toby Mintz
Date & Time: Dec 5th (Monday) from 10:00 AM to 12:00 PM.
Large pre-trained language models have become the foundation models for natural language processing. Some LMs (e.g., GPT-3) show the potential to acquire general language intelligence. However, we find that they can still make mistakes because they lack commonsense knowledge and reasoning ability, which are of vital significance in developing human-level general AI systems. In this talk, I will introduce how we can better evaluate and improve the commonsense reasoning (CSR) ability of LMs. Prior works mainly use mask-based probing and multiple-choice QA for evaluation. Their limitations prevent us from comprehensively measuring the CSR ability of LMs. To this end, I will present several benchmarks that aim to measure CSR ability in terms of open-endedness, generalization, and robustness, which are three key dimensions that are missing from the prior evaluation protocols. Then, I will introduce CSR methods that improve LMs by incorporating external knowledge. The external knowledge can be either structured graphs (e.g., ConceptNet) or unstructured text (e.g., GenericsKB), or even implicit as input-output pairs. Finally, I will briefly introduce a few interesting future directions for CSR.
WebCast Link: https://usc.zoom.us/j/91622202680?pwd=cUFNbzY2OXYyTFpuaFVIZHlTWEtLUT09
Audiences: Everyone Is Invited
Contact: Lizsl De Leon