Events Calendar

NL Seminar-HOW I LEARNED TO STOP WORRYING AND LOVE EVALUATIONS (AND KEEP WORRYING)

Speaker: Jon May, USC/ISI

Talk Title: HOW I LEARNED TO STOP WORRYING AND LOVE EVALUATIONS (AND KEEP WORRYING)

Series: Natural Language Seminar

Abstract: Bake-offs, shared tasks, evaluations: these are names for short, high-stress periods in many CS researchers' lives where their algorithms and models are exposed to unseen data, often with reputations and funding on the line. Evaluations are sometimes perceived to be the bane of much of our work lives. We grouse about metrics, procedures, glitches, and all the time "wasted" chasing scores, rather than doing Real Science (TM). In this talk I will argue that despite valid criticisms of the approach, coordinated evaluation is a net benefit to NLP research and has led to accomplishments that might not have otherwise arisen. This argument will frame a more in-depth discussion of several pieces of recent evaluation-grounded work: rapid generation of translation and information extraction for low-resource surprise languages (DARPA LORELEI) and organization of SemEval shared tasks in semantic parsing and generation.




Biography: Jonathan May is a Research Assistant Professor at the University of Southern California's Information Sciences Institute (USC/ISI). Previously, he was a research scientist at SDL Research (formerly Language Weaver) and a scientist at Raytheon BBN Technologies. He received a Ph.D. in Computer Science from the University of Southern California in 2010 and a BSE and MSE in Computer Science Engineering and Computer and Information Science, respectively, from the University of Pennsylvania in 2001. Jon's research interests include automata theory, natural language processing, machine translation, and machine learning.

Host: Marjan Ghazvininejad and Kevin Knight

More Info: http://nlg.isi.edu/nl-seminar/

Location: Information Science Institute (ISI) - 6th Flr -CR#689 (ISI/Marina Del Rey)

Audiences: Everyone Is Invited

Contact: Peter Zamar

Event Link: http://nlg.isi.edu/nl-seminar/