NL Seminar - DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models

Speaker: Ollie Liu, USC, USC

Talk Title: DeLLMa: A Framework for Decision Making Under Uncertainty with Large Language Models

Abstract: REMINDER: Meeting hosts only admit on-line guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) to make us aware of your attendance so we can admit you. Specify if you will attend remotely or in person at least one business day prior to the event  Provide your: full name, job title and professional affiliation and arrive at least 10 minutes before the seminar begins. If you do not have access to the 6th Floor for in-person attendance, please check in at the 10th floor main reception desk to register as a visitor and someone will escort you to the conference room location.   Abstract: Large language models (LLMs) are increasingly used across society, including in domains like business, engineering, and medicine. These fields often grapple with decision-making under uncertainty, a critical yet challenging task. In this paper, we show that directly prompting LLMs on these types of decision-making problems yields poor results, especially as the problem complexity increases. To overcome this limitation, we propose DeLLMa (Decision-making Large Language Model assistant), a framework designed to enhance decision-making accuracy in uncertain environments. DeLLMa involves a multi-step scaffolding procedure, drawing upon principles from decision theory and utility theory, to provide an optimal and human-auditable decision-making process. We validate our framework on decision-making environments involving real agriculture and finance data. Our results show that DeLLMa can significantly improve LLM decision-making performance, achieving up to a 40% increase in accuracy over competing methods.

Biography: Ollie Liu is second-year Ph.D student in Computer Science at University of Southern California, co-advised by Prof. Dani Yogatama and Prof. Willie Neiswanger. In life, I usually go by Oliver. My current research interests lie in (multimodal) foundation models, especially their algorithmic reasoning capabilities and applications in sciences.  

Host: Jonathan May and Justin Cho

More Info: https://www.isi.edu/research-groups-nlg/nlg-seminars/

Webcast: https://www.youtube.com/watch?v=XSTIFr9J0ko

Location: Information Science Institute (ISI) - Conf Rm#689

WebCast Link: https://www.youtube.com/watch?v=XSTIFr9J0ko

Audiences: Everyone Is Invited

Contact: Pete Zamar

Event Link: https://www.isi.edu/research-groups-nlg/nlg-seminars/