Logo: University of Southern California

Events Calendar

  • NL Seminar -On formulating and evaluating language agents

    Thu, Oct 05, 2023 @ 11:00 AM - 12:00 PM

    Information Sciences Institute

    Conferences, Lectures, & Seminars

    Speaker: Shunyu Yao, Princeton University

    Talk Title: On formulating and evaluating language agents

    Abstract: REMINDER:

    Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you are highly encouraged to use your USC account to sign into Zoom.

    If you are an outside visitor, please inform us at nlg DASH seminar DASH host AT isi DOT edu beforehand so we will be aware of your attendance and let you in.

    Language agents are AI systems that use large language models LLMs to interact with the world. While various methods have been developed, it is often hard to systematically understand or evaluate them. In this talk, we present Cognitive Architectures for Language Agents CoALA, a theoretical framework grounded in the classical research of cognitive architectures to make sense of existing agents and shed light into future directions. We also present three benchmarks WebShop, InterCode, Collie to develop and evaluate language agents using web, code, and grammar respectively. Notably, all three are scalable and practical, with simple and faithful evaluation metrics that do not rely on human preference labeling or LLM scoring.

    Biography: Shunyu Yao is a final year Phd student with Karthik Narasimhan at Princeton NLP Group. His research focuses on language agents, and is supported by the Harold W. Dodds Fellowship from Princeton.

    Host: Jon May and Justin Cho

    More Info: https://nlg.isi.edu/nl-seminar/

    Webcast: https://youtu.be/p6wSLDZat1w

    Location: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689

    WebCast Link: https://youtu.be/p6wSLDZat1w

    Audiences: Everyone Is Invited

    Contact: Pete Zamar

    Event Link: https://nlg.isi.edu/nl-seminar/


Return to Calendar