Events Calendar

Design Criteria for Human-Centered Natural Language Generation

Speaker: Kaitlyn Zhou, Stanford University

Talk Title: Design Criteria for Human-Centered Natural Language Generation

Series: NL Seminar

Abstract: Abstract: REMINDER: Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you are highly encouraged to use your USC account to sign into Zoom. If you are an outside visitor, please inform us at nlg DASH seminar DASH host AT isi DOT edu beforehand so we will be aware of your attendance and let you in. In-person attendance will be permitted for USC/ISI faculty, staff, students only. Open to the public virtually via the zoom link. Large language models have made substantial steps towards generating human-like language. However, this endeavor to mimic human language comes with potential drawbacks. By mimicking and appropriating human language, the systems produce language that inherits the harms and cognitive biases of humans while failing to ensure features like clarity and transparency. My research asks: how can generated language avoid the harms of natural language while supporting safe and collaborative human-AI collaboration? Starting with the researchers, I study the quality criteria of natural language generation, using mixed methods approaches to reveal design decisions made consciously and subconsciously by natural language generation by practitioners. Looking through datasets of natural language, I identify the origins of language appropriation and illustrate the safety risks mimicry has via the linguistic miscalibration of language models. Lastly, I study how humans perceive the appropriation of social behaviors such as politeness and refusal and the risks they may pose in chat settings. What I find throughout my research is that language models inappropriately appropriate the style, the use of linguistic cues, and the prosocial language of the human text they are trained on. My future work seeks to develop design criteria for generated language, centered on user-needs, to build training methods to achieve this goal.

Biography: Kaitlyn Zhou is currently pursuing her PhD in computer science at Stanford University, advised by Dan Jurafsky. Her research focuses on investigating the unintended consequences that stem from the appropriation of natural language by language models. Her work delves into various aspects, including the fairness implications associated with the evaluation of natural language generation, the linguistic miscalibration displayed by language models, and the misplaced overconfidence of publicly deployed chatbots. Kaitlyn has previously spent summers at Microsoft Research and the Allen Institute for Artificial Intelligence. She is funded by the Stanford Graduate Fellowship and her visualization techniques have gained recognition in prominent publications like The New York Times and the Wall Street Journal. In 2018, Kaitlyn was appointed by Washington State Governor Jay Inslee to the University of Washington Board of Regents.

Host: Jon May and Justin Cho

More Info: https://nlg.isi.edu/nl-seminar/

Webcast: https://youtu.be/bJC6PFxU99s

Location: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689

WebCast Link: https://youtu.be/bJC6PFxU99s

Audiences: Everyone Is Invited

Contact: Pete Zamar

Event Link: https://nlg.isi.edu/nl-seminar/