Select a calendar:
Filter October Events by Event Type:
SUNMONTUEWEDTHUFRISAT
Events for October 31, 2024
-
NL Seminar-InterIntent Investigating Social Intelligence of LLMs via Intention Understanding in a Game context
Thu, Oct 31, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Ziyi Liu, USC
Talk Title: InterIntent Investigating Social Intelligence of LLMs via Intention Understanding in a Game Context
Abstract: REMINDER: Meeting hosts only admit on-line guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) to make us aware of your attendance so we can admit you. Specify if you will attend remotely or in person at least one business day prior to the event Provide your: full name, job title and professional affiliation and arrive at least 10 minutes before the seminar begins. If you do not have access to the 6th Floor for in-person attendance, please check in at the 10th floor main reception desk to register as a visitor and someone will escort you to the conference room location. Zoom Info: https://usc.zoom.us/j/95325436571?pwd=NMJIFIQNQ01esvL9UffxxIp4dnSCmF.1Meeting ID: 953 2543 6571/Passcode: 985321 Abstract: Large language models (LLMs) have demonstrated the potential to mimic human social intelligence. However, most studies focus on simplistic and static self-report or performance-based tests, which limits the depth and validity of the analysis. In this paper, we developed a novel framework, INTERINTENT, to assess LLMs’ social intelligence by mapping their ability to understand and manage intentions in a game setting. We focus on four dimensions of social intelligence: situational awareness, self-regulation, self-awareness, and theory of mind. Each dimension is linked to a specific game task: intention selection, intention following, intention summarization, and intention guessing. Our findings indicate that while LLMs exhibit high proficiency in selecting intentions, achieving an accuracy of 88%, their ability to infer the intentions of others is significantly weaker, trailing human performance by 20%. Additionally, game performance correlates with intention understanding, highlighting the importance of the four components towards success in this game. These findings underline the crucial role of intention understanding in evaluating LLMs’ social intelligence and highlight the potential of using social deduction games as a complex testbed to enhance LLM evaluation. INTERINTENT contributes a structured approach to bridging the evaluation gap in social intelligence within multiplayer games.
Biography: Ziyi Liu is a second-year PhD student at the University of Southern California, advised by Professor Jieyu Zhao in LIME Lab. Previously, she earned her master’s degree at USC and was a Research Assistant in USC ISI’s Ink Lab for two years under the guidance of Professor Xiang Ren. Her research focuses on social intelligence and hallucination detection in human-LLM interactions, particularly in evaluating LLM behaviors and aligning LLM values with those of humans. Her work is driven by two key questions: (1) How can we make interactions between models and humans more seamless? (2) How can we ensure the faithfulness of LLMs and avoid hallucinations during interactions?
Host: Jonathan May and Katy Felkner
More Info: https://www.isi.edu/research-groups-nlg/nlg-seminars/
Webcast: https://www.youtube.com/watch?v=yHfeHKahMoILocation: Information Science Institute (ISI) - Conf Rm#689
WebCast Link: https://www.youtube.com/watch?v=yHfeHKahMoI
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://www.isi.edu/research-groups-nlg/nlg-seminars/
-
AIF4S Seminar: Value of Pretraining Data: Scaling Laws for Downstream Task Performance of Large Language Models
Thu, Oct 31, 2024 @ 02:00 PM - 03:00 PM
Ming Hsieh Department of Electrical and Computer Engineering
Conferences, Lectures, & Seminars
Speaker: Dr. Berivan Isik, Research Scientist, Google, Inc.
Talk Title: Value of Pretraining Data: Scaling Laws for Downstream Task Performance of Large Language Models
Abstract: This talk explores the challenges and open questions surrounding the value of pretraining data for large language models (LLMs) in transfer learning settings. While scaling laws have provided valuable insights for LLM design, existing work has predominantly focused on pretraining loss. In contrast, this work investigates scaling behavior in a transfer learning setting where LLMs are finetuned for downstream tasks. Specifically, we examine how the choice and size of pretraining data impact downstream performance, as measured by cross-entropy and translation quality metrics such as BLEU and COMET. Our experiments reveal that the size of the finetuning dataset and the alignment between pretraining and downstream data significantly influence scaling behavior. With sufficient alignment, both cross-entropy and translation quality improve with increased pretraining data, and we demonstrate the ability to predict translation quality using a new log-law. However, in cases of moderate misalignment, we observe that translation quality can fluctuate or even deteriorate with more pretraining data, despite consistent improvements in cross-entropy. Through analysis of these findings, we provide insights for selecting appropriate pretraining data. The talk will conclude with a discussion of future research directions and remaining open questions in this area.
Biography: Berivan Isik is a research scientist at Google, working on efficient and trustworthy AI. Her current interests are efficient training/finetuning of large models, pretraining data valuation and scaling laws for LLMs, differential privacy, and unlearning. She earned her PhD from Stanford University in 2024, where she was affiliated with the SAIL and StatsML groups. Her research was supported by Stanford Graduate Fellowship (2019-2023), Google Ph.D. Fellowship (2023-2026), and a Meta research grant.
Host: Dr. Mahdi Soltanolkotbi, soltanol@usc.edu
Webcast: https://usc.zoom.us/j/98648507063?pwd=kORhNLFVMLol7FYlHv6TsAmqcKqD7t.1Location: Hughes Aircraft Electrical Engineering Center (EEB) - 132
WebCast Link: https://usc.zoom.us/j/98648507063?pwd=kORhNLFVMLol7FYlHv6TsAmqcKqD7t.1
Audiences: Everyone Is Invited
Contact: Mayumi Thrasher
-
PhD Dissertation Defense - Fei Wang
Thu, Oct 31, 2024 @ 03:00 PM - 05:00 PM
Thomas Lord Department of Computer Science
University Calendar
Title: Developing Robust and Controllable (Multimodal) Large Language Models
Date and Time: Thur, Oct 31, 2024 @ 3:00 PM - 05:00 PM
Location: RTH 211
Zoom Link: https://usc.zoom.us/my/feiwangnlp
Committee members: Aram Galstyan (Chair), Muhao Chen, Laurent Itti, Dan O’Leary
Abstract: As (multimodal) large language models (LLMs) become integral to intelligent systems, they are increasingly used in scenarios ranging from everyday applications to high-stakes domains such as healthcare, finance, and law. Consequently, there is a growing urgency to enhance the robustness and controllability of these models and mitigate critical risks in their development and deployment. This dissertation talk will introduce methods to ensure responsible outcomes from (multimodal) LLMs through three key perspectives: (1) dynamic integration of up-to-date and domain-specific knowledge, (2) robust alignment with human intents, preferences, and values, and (3) precise control over model behavior to ensure compliance with task constraints, authorization protocols, and safety requirements.Location: Ronald Tutor Hall of Engineering (RTH) - 211
Audiences: Everyone Is Invited
Contact: Ellecia Williams
