SUNMONTUEWEDTHUFRISAT
Conferences, Lectures, & Seminars
Events for March
-
NL Seminar - Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Thu, Mar 07, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Zixiang Chen, UCLA
Talk Title: Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
Series: NL Seminar
Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded. Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/. Harnessing the power of human-annotated data through Supervised Fine-Tuning (SFT) is pivotal for advancing Large Language Models (LLMs). In this talk, I will introduce our newest fine-tuning method, Self-Play Fine-Tuning (SPIN), which improves LLMs without the need for additional human-annotated data. SPIN utilizes a self-play mechanism, where the LLM enhances its capabilities by generating its own training data through interactions with instances of itself. Specifically, the LLM generates its own training data from its previous iterations, refining its policy by discerning these self-generated responses from those obtained from human-annotated data. As a result, SPIN unlocks the full potential of human-annotated data for SFT. Our empirical results show that SPIN can improve the LLM’s performance across a variety of benchmarks and even outperform models trained through direct preference optimization (DPO) supplemented with extra GPT-4 preference data. Additionally, I will outline the theoretical guarantees of our method. For more details and access to our codes, visit our GitHub repository (https://github.com/uclaml/SPIN).
Biography: Zixiang Chen is currently a Ph.D. student in computer science at the Department of Computer Science, University of California, Los Angeles (UCLA), advised by Prof. Quanquan Gu. He obtained his bachelor’s degree in mathematics from Tsinghua University. He is broadly interested in the theory and applications of deep learning, optimization, and control, with a focus on generative models, representation learning, and multi-agent reinforcement learning. Recently, he has been utilizing AI to enhance scientific discovery in the domain of public health. He was a visiting graduate student in the theory of reinforcement learning program at the Simons Institute for the Theory of Computing. If speaker approves to be recorded for this NL Seminar talk, it will be posted on our USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI. Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://youtu.be/Fg4C6YZcqQ4Location: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://youtu.be/Fg4C6YZcqQ4
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
AI Seminar
Fri, Mar 08, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Yu-Ru Lin, Univ. of Pitt., Univ of Pitt
Talk Title: A Gateway to Trustworthy AI: Using Visual Analytics to Unmask Coincidental Correlations
Abstract: Join Zoom Meeting https://usc.zoom.us/s/99782858348?pwd=MnlSdGlTVWNETGFFbDQ4OWRmakdEQT09 Meeting ID: 997 8285 8348 Passcode: 580559 Register in advance for this webinar: https://usc.zoom.us/webinar/register/WN_xxYy3NkSQpidFYRY3fg_Ew In the realm of machine learning and data-driven decision-making, the risk of spurious and biased associations poses significant challenges to the integrity and reliability of AI systems. In this talk, I will introduce how visual analytic designs can empower data practitioners in navigating these complex issues. First, through a human-in-the-loop workflow, we tackle the problem of AI blindspots in classification models, where key patterns are often missed or misleading. Our design offers visually interpretable statistical methods to quantify and understand concept associations. It also includes debiasing techniques to address misleading patterns in data. Second, we tackle Simpson’s Paradox, a phenomenon where associations in data appear contradictory at different levels of aggregation, leading to cognitive confusion and incorrect interpretations. Our design offers an intuitive causal analysis framework and a human-centric workflow, enabling users to identify, understand, and prevent spurious associations, leading to more accountable causal decision-making. Together, these design frameworks contribute to making AI more trustworthy, offering robust tools for overcoming the challenges of spurious and biased associations in machine learning through advanced visual analytics.
Biography: Website: http://www.yurulin.com/ Yu-Ru Lin is an Associate Professor in the School of Computing and Information and the Research Director of the Institute for Cyber Law, Policy, and Security (Pitt Cyber) at the University of Pittsburgh, where she directs the PITT Computational Social Dynamics Lab (PICSO LAB). Her research lies at the intersection of Computational Social Science, Data Mining, and Visualization. She specializes in using social network and text data along with statistical learning tools and social theories to study phenomena spanning societal events and policy, anomalous behaviors, and other crucially important complex patterns concerning collective attention and actions, as well as human and social dynamics in response to societal risks. Her work has appeared in prestigious scientific venues and has been featured in the press, including WSJ, The Boston Globe, The Atlantic, MIT News, and NPR. She has authored or co-authored more than 100 refereed journal and conference papers and served on more than 50 conference program committees in the areas of big data, network science, and computational social science. She has served as a chair/co-chair of leading computational social science, web mining, and social media conferences such as AAAI ICWSM and TheWebConference/WWW (Web & Society Track). She currently serves as an Editor-in-Chief of AAAI ICWSM and an Associate Editor for multiple journals, including PLOS ONE, Springer EPJ Data Science, Nature's Scientific Reports, and Frontiers in Big Data. She was selected as a Fellow of Kavli Frontiers of Science, National Academy of Sciences (NAS).
Host: Fred Morstatter and Zhuoyu Shi
More Info: https://www.isi.edu/events/4389/ai-seminar-a-gateway-to-trustworthy-ai-using-visual-analytics-to-unmask-coincidental-correlations/
Webcast: https://www.youtube.com/watch?v=2uZOOM6-nooLocation: Information Science Institute (ISI) - Virtual Only
WebCast Link: https://www.youtube.com/watch?v=2uZOOM6-noo
Audiences: Everyone Is Invited
Contact: Pete Zamar
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar-Do Androids Know They're Only Dreaming of Electric Sheep?
Mon, Mar 18, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Sky Wang, Columbia University
Talk Title: Do Androids Know They're Only Dreaming of Electric Sheep?
Series: NL Seminar
Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded. Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/ We design probes trained on the internal representations of a transformer language model that are predictive of its hallucinatory behavior on in-context generation tasks. To facilitate this detection, we create a span-annotated dataset of organic and synthetic hallucinations over several tasks. We find that probes trained on the force-decoded states of synthetic hallucinations are generally ecologically invalid in organic hallucination detection. Furthermore, hidden state information about hallucination appears to be task and distribution-dependent. Intrinsic and extrinsic hallucination saliency varies across layers, hidden state types, and tasks; notably, extrinsic hallucinations tend to be more salient in a transformer's internal representations. Outperforming multiple contemporary baselines, we show that probing is a feasible and efficient alternative to language model hallucination evaluation when model states are available.
Biography: If speaker approves to be recorded for this NL Seminar talk, it will be posted on our USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI. Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/ Sky is a Ph.D. candidate in Computer Science at Columbia University advised by Zhou Yu and Smaranda Muresan. His research primarily revolves around Natural Language Processing (NLP), with broad interests in the area where NLP meets Computational Social Science (CSS). Here, his research primarily revolves around three major areas: (1) revealing and designing for social difference and inequality, (2) cross-cultural NLP, and (3) mechanistic interpretability. His research is supported by a NSF Graduate Research Fellowship and has received two outstanding paper awards at EMNLP. He has previously been an intern at Microsoft Semantic Machines, Google Research, and Amazon AWS AI.
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=Pm0ljFMg0cwLocation: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://www.youtube.com/watch?v=Pm0ljFMg0cw
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar -The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI
Thu, Mar 21, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Anthony Chen and Shayne Longpre, MIT
Talk Title: The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI
Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded. Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/ The arms race to train language models on vast, diverse, and inconsistently documented datasets has raised pressing concerns about the legal and ethical risks for practitioners. To remedy these practices threatening data transparency and understanding, we introduce the Data Provenance Initiative, a multi-disciplinary effort between legal and machine learning experts to systematically audit and trace 1800+ text datasets. We develop tools and standards to trace the lineage of these datasets, from their source, creators, series of license conditions, properties, and subsequent use. Our landscape analysis highlights the sharp divides in composition and focus of commercially open vs closed datasets, with closed datasets monopolizing important categories: lower resource languages, more creative tasks, richer topic variety, newer and more synthetic training data.
Biography: Bio 1:Anthony Chen is an engineer at Google DeepMind doing research on factuality and long-context language models. He received his PhD from UC Irvine last year where he focused on generative evaluation and factuality in language models. Bio 2: Shayne Longpre is a PhD candidate at MIT with a focus on data-centric AI, language models, and their societal impact. If speakers approve to be recorded for this NL Seminar talk, it will be posted on our USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI. Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=np9HeJN6miwLocation: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://www.youtube.com/watch?v=np9HeJN6miw
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar-Informative Example Selection for In-Context Learning
Thu, Mar 28, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Shivanshu Gupta, UCI
Talk Title: Informative Example Selection for In-Context Learning
Series: NL Seminar
Abstract: Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please inform us at (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance and let you in. In-person attendance will be permitted for USC/ISI faculty, staff, students only. Open to the public virtually via the zoom link. For more information on the NL Seminar series and upcoming talks, please visit: https://nlg.isi.edu/nl-seminar/ In-context Learning (ICL) uses large language models (LLMs) for new tasks by conditioning them on prompts comprising a few task examples. With the rise of LLMs that are intractable to train or hidden behind APIs, the importance of such a training-free interface cannot be overstated. However, ICL is known to be critically sensitive to the choice of in-context examples. Despite this, the standard approach for selecting in-context examples remains to use general-purpose retrievers due to the limited effectiveness and training requirements of prior approaches. In this talk, I'll posit that good in-context examples demonstrate the salient information necessary to solve a given test input. I'll present efficient approaches for selecting such examples, with a special focus on preserving the training-free ICL pipeline. Through results with a wide range of tasks and LLMs, I'll demonstrate that selecting informative examples can indeed yield superior ICL performance.
Biography: Shivanshu Gupta is a Computer Science Ph.D. Candidate at the University of California Irvine, advised by Sameer Singh. Prior to this, he was a Research Fellow at LinkedIn and Microsoft Research India, and completed his B.Tech. and M.Tech. in Computer Science at IIT Delhi. His primary research interests are systematic generalization, in-context learning, and multi-step reasoning capabilities of large language models. If speaker approves to be recorded for this NL Seminar talk, it will be posted on the USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI. Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=Vqvy4XIOtcELocation: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://www.youtube.com/watch?v=Vqvy4XIOtcE
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor.
