Events Calendar

NL Seminar -LLMs Do Not Have Human-Like Working Memories

Speaker: Jen-Tse (Jay) Huang, Johns Hopkins University

Talk Title: LLMs Do Not Have Human-Like Working Memories

Abstract: Meeting hosts only admit on-line guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) to make us aware of your attendance so we can admit you. Specify if you will attend remotely or in person at least one business day prior to the event. Provide your: full name, job title and professional affiliation and arrive at least 10 minutes before the seminar begins. If you do not have access to the 6th Floor for in-person attendance, please check in at the 10th floor main reception desk to register as a visitor and someone will escort you to the conference room location. Join Zoom Meetinghttps://usc.zoom.us/j/93233836050?pwd=cCKn1GHZ6XeCK1sZa9ZL1h3ahyyf5h.1 Meeting ID: 932 3383 6050 Passcode: 804448  
Human working memory is an active cognitive system that enables not only the temporary storage of information but also its processing and utilization. Without working memory, individuals may produce unreal conversations, struggle with tasks requiring mental reasoning, and exhibit self-contradiction. In this presentation, we demonstrate that Large Language Models (LLMs) lack this human-like cognitive ability, posing a significant challenge to achieving artificial general intelligence. We validate this claim through three experiments: (1) Number Guessing Game, (2) Yes or No Game, and (3) Math Magic. Experimental results on several model families indicate that current LLMs fail to exhibit human-like cognitive behaviors in these scenarios. By highlighting this limitation, we aim to encourage further research in developing LLMs with improved working memory capabilities.   

Biography: Jen-Tse (Jay) Huang is a postdoctoral researcher at the Center for Language and Speech Processing (CLSP) at Johns Hopkins University, working with Mark Dredze. He received his Ph.D. in Computer Science and Engineering from the Chinese University of Hong Kong and his B.Sc. from Peking University. His research explores the evaluation of large language models (LLMs), both as individual agents and as collectives in multi-agent systems, through the lens of social science. His work has been published in top-tier AI venues, including an oral presentation at ICLR 2024. He actively serves as a reviewer for major conferences and journals such as ICML, NeurIPS, ICLR, ACL, CVPR, TMLR and Nature Human Behaviour, and has been recognized as an Outstanding Reviewer at NeurIPS 2024 and EMNLP 2024.
 

Host: Jonathan May and Katy Felkner

More Info: https://usc.zoom.us/j/93233836050?pwd=cCKn1GHZ6XeCK1sZa9ZL1h3ahyyf5h.1

Webcast: https://www.isi.edu/research-groups-nlg/nlg-seminars/

Location: Information Science Institute (ISI) - Conf Rm#689

WebCast Link: https://www.isi.edu/research-groups-nlg/nlg-seminars/

Audiences: Everyone Is Invited

Contact: Pete Zamar

Event Link: https://usc.zoom.us/j/93233836050?pwd=cCKn1GHZ6XeCK1sZa9ZL1h3ahyyf5h.1

This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor.

Add to Google Calendar