Logo: University of Southern California

Events Calendar



Select a calendar:



Filter June Events by Event Type:



Events for the 3rd week of June

  • NL Seminar Sources of Variance in Pretraining and Finetuning LLMs

    Mon, Jun 13, 2022 @ 02:00 PM - 03:00 PM

    Information Sciences Institute

    Conferences, Lectures, & Seminars


    Speaker: Naomi Saphra, NYU

    Talk Title: Sources of Variance in Pretraining and Finetuning LLMs

    Series: NL Seminar

    Abstract: REMINDER
    Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you are highly encouraged to use your USC account to sign into Zoom.

    If you are an outside visitor, please inform us at (nlg DASH seminar DASH host AT isi DOR edu beforehand so we will be aware of your attendance and let you in.

    You have engaged in the very modern practice of transfer learning. You pretrained a model on a self supervised objective, then you finetuned it on a downstream task, and you find excellent performance on the test set. Aha, you say. I found a good pretraining procedure. Did you? You try finetuning again. The results are terrible! Aha, you say. I found a bad finetuning procedure. Did you?

    The random seeds for both pretraining and finetuning stages have a substantial influence on outcome. However, it is computationally expensive to pretrain new models, so measuring the robustness of a procedure across different seeds can be prohibitive. This talk will address, first, the influence that a pretraining seed has on both in domain and OOD performance. Then we will address the role of the finetuning seed. Much variation in OOD generalization can be ascribed to where the finetuning seeds direct SGD trajectories. In particular, we discuss how to predict generalization behavior in a finetuned model, based on topographic properties of its region of the loss surface. By understanding the degree of influence that random seeds have on performance, we can fairly evaluate a robust training procedure, rather than a single set of parameters. By understanding the mechanism of that influence, we can go further by developing improved training methods.


    Biography: Naomi has interests relating to NLP learning dynamics how models learn to encode linguistic structure, and how we can encode useful inductive biases into the training process. Having earned a PhD from University of Edinburgh, they are now a postdoc at NYU. In their spare time, they play roller derby under the name Gaussian Retribution, do standup comedy, and shepherd programmers who cannot type into the world of code dictation.

    Host: Jon May and Thamme Gowda

    More Info: https://nlg.isi.edu/nl-seminar/

    Webcast: https://www.youtube.com/watch?v=Lni4PIlbJjI

    Location: Information Science Institute (ISI) - Virtual

    WebCast Link: https://www.youtube.com/watch?v=Lni4PIlbJjI

    Audiences: Everyone Is Invited

    Contact: Pete Zamar

    Event Link: https://nlg.isi.edu/nl-seminar/

    OutlookiCal
  • Repeating Event[Virtual] First-Year Admission Information Session

    Tue, Jun 14, 2022 @ 04:00 PM - 05:00 PM

    Viterbi School of Engineering Undergraduate Admission

    Workshops & Infosessions


    Our virtual information session is a live presentation from a USC Viterbi admission counselor designed for high school students and their family members to learn more about the USC Viterbi undergraduate experience. Our session will cover an overview of our undergraduate engineering programs, the application process, and more on student life. Guests will be able to ask questions and engage in further discussion toward the end of the session.

    Register Here!

    Audiences: Everyone Is Invited

    View All Dates

    Contact: Viterbi Admission

    OutlookiCal
  • PhD Defense - Jiaoyang Li

    Thu, Jun 16, 2022 @ 04:00 PM - 06:00 PM

    Thomas Lord Department of Computer Science

    University Calendar


    PhD Candidate: Jiaoyng Li

    Title:
    Efficient and Effective Techniques for Large-Scale Multi-Agent Path Finding

    Committee:
    Sven Koenig, T. K. Satish Kumar, Satyandra K. Gupta, Nora Ayanian , and Brian C. Williams.

    Abstract:
    There is no doubt that robots will play a crucial role in the future and need to work as a team in increasingly more complex applications. Advances in robotics have laid the hardware foundations for building large-scale multi-robot systems, such as for mobile robots, vehicles, and drones. But how to coordinate robots intelligently is a difficult problem. In this dissertation, I introduce planning algorithms for solving this challenge with a focus on one fundamental problem: letting a large team of agents navigate without collisions in congested environments while minimizing their travel times. I present techniques based on heuristic search, symmetry breaking, and stochastic local search that can efficiently and effectively coordinate hundreds of agents with rigorous guarantees of completeness and even optimality and thousands of agents with good empirical performance (although no theoretical guarantees). These techniques speed up optimal and bounded-suboptimal algorithms by up to four orders of magnitude without sacrificing their theoretical guarantees and improve the solution quality of non-optimal algorithms by up to thirty-six times.

    Location: Henry Salvatori Computer Science Center (SAL) - 322

    WebCast Link: https://usc.zoom.us/j/93790809266?pwd=SDVIMWFtYTVtaEZpeVNGM0MxSWM2dz09

    Audiences: Everyone Is Invited

    Contact: Lizsl De Leon

    OutlookiCal
  • Repeating Event[Virtual] First-Year Admission Information Session

    Thu, Jun 16, 2022 @ 04:00 PM - 05:00 PM

    Viterbi School of Engineering Undergraduate Admission

    Workshops & Infosessions


    Our virtual information session is a live presentation from a USC Viterbi admission counselor designed for high school students and their family members to learn more about the USC Viterbi undergraduate experience. Our session will cover an overview of our undergraduate engineering programs, the application process, and more on student life. Guests will be able to ask questions and engage in further discussion toward the end of the session.

    Register Here!

    Audiences: Everyone Is Invited

    View All Dates

    Contact: Viterbi Admission

    OutlookiCal