NL Seminar

Speaker: Kawin Ethayarajh, Stanford University

Talk Title: Machine Learning with Human Fault-Tolerance

Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded.  Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/ In machine learning, we have long recognized the need to build systems that can tolerate hardware faults and software faults. In this talk, I propose the need for a third kind of fault-tolerance: human fault-tolerance. The methods used to develop, evaluate, and deploy machine learning systems today assume that the humans that build and use them are rational actors making highly-informed decisions based on consistent preferences—this is far from true in practice. We can address the failures of these assumptions by drawing from economics, a field that has long been aware of how unfounded beliefs about human behavior can go wrong. Specifically, I will cover how we can develop theoretically grounded tools that discover human mistakes, design algorithms and methods for robustly eliciting and incorporating human feedback, and implement end-to-end platforms that make ML and NLP more transparent and reproducible. This line of work has led to the creation of datasets, models, and platforms that have been widely adopted by industry giants like Amazon, Google, and Meta.

Biography: Kawin Ethayarajh is a 5th year PhD student at Stanford University, where he works on bringing human fault-tolerance to machine learning. His research draws from economics to make machine learning and NLP more robust to the irrational, inconsistent, and uninformed human decisions made at every step. His work has been supported by a Facebook Fellowship and an NSERC PGS-D, and he has received an Outstanding Paper Award at ICML 2022. He co-created the Stanford Human Preferences dataset and the Dynaboard platform (behind Dynabench).

Host: Jon May and Justin Cho

More Info: https://nlg.isi.edu/nl-seminar/

Webcast: https://usc.zoom.us/j/99484520082

Location: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689

WebCast Link: https://usc.zoom.us/j/99484520082

Audiences: Everyone Is Invited

Contact: Pete Zamar

Event Link: https://nlg.isi.edu/nl-seminar/