Thu, Nov 01, 2018 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Robin Jia, Stanford University
Talk Title: Exposing Brittleness in Reading Comprehension Systems
Series: Natural Language Seminar
Abstract: Reading comprehension systems that answer questions over a context passage can often achieve high test accuracy, but they are frustratingly brittle: they often rely heavily on superficial cues, and therefore struggle on out-of-domain inputs. In this talk, I will describe our work on understanding and challenging these systems. First, I will show how to craft adversarial reading comprehension examples by adding irrelevant distracting text to the context passage. Next, I will present the newest version of the SQuAD dataset, SQuAD 2.0, which tests whether models can distinguish answerable questions from similar but unanswerable ones. Finally, I will share some observations from our recent attempts to use reading comprehension systems as a natural language interface for building other NLP systems.
Biography: Robin Jia is a fifth-year PhD student advised by Percy Liang at Stanford University. He is an NSF Graduate Fellow, and has received Outstanding Paper and Best Short Paper Awards from EMNLP and ACL, respectively.
Host: Xusen Yin
More Info: http://nlg.isi.edu/nl-seminar/
Audiences: Everyone Is Invited
Posted By: Peter Zamar