Logo: University of Southern California

Events Calendar


  • PhD Defense - Michiel De Jong

    Mon, May 08, 2023 @ 03:00 PM - 05:00 PM

    Thomas Lord Department of Computer Science

    University Calendar



    Title: EXPANDING THE QUALITY-COMPUTE FRONTIER FOR RETRIEVAL-AUGMENTED LANGUAGE MODELS

    Abstract: Retrieval-augmented language models set the state-of-the-art on a broad spectrum of knowledge-intensive tasks, outperforming orders of magnitude larger models. However, such models can also be expensive for training and inference. Model performance and computational cost represent two sides of the coin: we can generally improve performance through scale at the expense of an increased computational burden. Therefore, we are really interested in pushing out the quality-compute frontier, improving performance at any given level of computational resources.

    In this dissertation, I analyze the factors that determine the computational burden of retrieval-augmented language models and propose strategies to extract a better performance-compute trade-off. The dissertation consists of three sections. The first section contains a detailed analysis of components of retrieval-augmented models and introduces methods to improve generation efficiency. The second section explores the use of dense memory to reduce the cost of encoding retrievals. Finally, the third section proposes a hybrid between dense memory and text retrieval, combining lessons from previous chapters.

    Names of the Dissertation defense committee members:
    Chair: Leana Golubchik
    Members:
    Fei Sha
    Dani Yogatama
    Jacob Bien

    Venue: Zoom, https://usc.zoom.us/my/lgzoomeeting



    Location: https://usc.zoom.us/my/lgzoomeeting

    Audiences: Everyone Is Invited

    Contact: Asiroh Cham

    Add to Google CalendarDownload ICS File for OutlookDownload iCal File

Return to Calendar