Conferences, Lectures, & Seminars
Events for February
-
NL Seminar -Harnessing Black-Box Control to Boost Commonsense in LM's Generation
Thu, Feb 01, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Yufei Tian, UCLA
Talk Title: Harnessing Black-Box Control to Boost Commonsense in LM's Generation
Series: NL Seminar
Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded. Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/ Large language models like Alpaca and GPT-3 generate coherent texts but sometimes lack commonsense, yet improving their commonsense via fine-tuning is resource expensive in terms of both data and computation. In this talk, I'll present BOOST, a resource-efficient framework that steers a frozen Pre-Trained Language Model (PTLM) towards more reasonable outputs. This involves creating an interpretable and reference-free evaluator that assigns a sentence with a commonsensical score which grounds the sentence to a dynamic commonsense knowledge base. Using this evaluator as a guide, we extend the NADO controllable generation method to train an auxiliary head that improves the PTLM's output. Our framework was tested on various language models, including GPT-2, Flan-T5, and Alpaca-based models. On two constrained concept-to-sentence benchmarks, human evaluation results show that BOOST consistently generates the most commonsensical content. Finally, I will demonstrate how ChatGPT outputs are different from and sometimes less favored than our outputs.
Biography: Yufei Tian is a CS PhD student at UCLA advised by Prof. Nanyun (Violet) Peng. Her research is centered around creative and controllable text generation, machine reasoning and its interaction with cognitive science, as well as designing evaluation metrics for open-ended NLG tasks. She is supported by the UCLA-Amazon fellowship program.
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://www.youtube.com/watch?v=WTIKszPDzDkLocation: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://www.youtube.com/watch?v=WTIKszPDzDk
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor. -
NL Seminar -Red Teaming Language Model Detectors with Language Models
Thu, Feb 22, 2024 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Yihan Wang, UCLA
Talk Title: Red Teaming Language Model Detectors with Language Models
Series: NL Seminar
Abstract: REMINDER: This talk will be a live presentation only, it will not be recorded. Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you’re highly encouraged to use your USC account to sign into Zoom. If you’re an outside visitor, please provide your: Full Name, Title and Name of Workplace to (nlg-seminar-host(at)isi.edu) beforehand so we’ll be aware of your attendance. Also, let us know if you plan to attend in-person or virtually. More Info for NL Seminars can be found at: https://nlg.isi.edu/nl-seminar/ The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to detect LLM-generated text and protect LLMs. In this paper, we investigate the robustness and reliability of these LLM detectors under adversarial attacks. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation. In both strategies, we leverage an auxiliary LLM to generate the word replacements or the instructional prompt. Different from previous works, we consider a challenging setting where the auxiliary LLM can also be protected by a detector. Experiments reveal that our attacks effectively compromise the performance of all detectors in the study with plausible generations, underscoring the urgent need to improve the robustness of LLM-generated text detection systems. This talk may also introduce some of our other recent works on trustworthy and ethical LLMs.
Biography: Yihan is Ph.D. student at UCLA in Computer Science. She received her B.Eng. degree in Computer Science and Technology from Tsinghua University in June 2020. Ms. Wang's research interest is machine learning, especially improving trustworthiness and generalization of machine learning models. Yihan is currently working with Prof. Cho-Jui Hsieh at UCLA. If speaker approves to be recorded for this NL Seminar talk, it will be posted on our USC/ISI YouTube page within 1-2 business days: https://www.youtube.com/user/USCISI. Subscribe here to learn more about upcoming seminars: https://www.isi.edu/events/
Host: Jon May and Justin Cho
More Info: https://nlg.isi.edu/nl-seminar/
Webcast: https://youtu.be/Fx1T9lyNDh0?si=qEL0QipveladKDwPLocation: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689
WebCast Link: https://youtu.be/Fx1T9lyNDh0?si=qEL0QipveladKDwP
Audiences: Everyone Is Invited
Contact: Pete Zamar
Event Link: https://nlg.isi.edu/nl-seminar/
This event is open to all eligible individuals. USC Viterbi operates all of its activities consistent with the University's Notice of Non-Discrimination. Eligibility is not determined based on race, sex, ethnicity, sexual orientation, or any other prohibited factor.
