Events Calendar

NL Seminar-Exposing Brittleness in Reading Comprehension Systems

Speaker: Robin Jia, Stanford University

Talk Title: Exposing Brittleness in Reading Comprehension Systems

Series: Natural Language Seminar

Abstract: Reading comprehension systems that answer questions over a context passage can often achieve high test accuracy, but they are frustratingly brittle: they often rely heavily on superficial cues, and therefore struggle on out-of-domain inputs. In this talk, I will describe our work on understanding and challenging these systems. First, I will show how to craft adversarial reading comprehension examples by adding irrelevant distracting text to the context passage. Next, I will present the newest version of the SQuAD dataset, SQuAD 2.0, which tests whether models can distinguish answerable questions from similar but unanswerable ones. Finally, I will share some observations from our recent attempts to use reading comprehension systems as a natural language interface for building other NLP systems.


Biography: Robin Jia is a fifth-year PhD student advised by Percy Liang at Stanford University. He is an NSF Graduate Fellow, and has received Outstanding Paper and Best Short Paper Awards from EMNLP and ACL, respectively.

Host: Xusen Yin

More Info: http://nlg.isi.edu/nl-seminar/

Location: Information Science Institute (ISI) - 6th Floor Conf Rm-CR# 689

Audiences: Everyone Is Invited

Contact: Peter Zamar

Event Link: http://nlg.isi.edu/nl-seminar/