Logo: University of Southern California

Events Calendar

  • PhD Defense - Michiel De Jong

    Mon, May 08, 2023 @ 03:00 PM - 05:00 PM

    Thomas Lord Department of Computer Science

    University Calendar


    Abstract: Retrieval-augmented language models set the state-of-the-art on a broad spectrum of knowledge-intensive tasks, outperforming orders of magnitude larger models. However, such models can also be expensive for training and inference. Model performance and computational cost represent two sides of the coin: we can generally improve performance through scale at the expense of an increased computational burden. Therefore, we are really interested in pushing out the quality-compute frontier, improving performance at any given level of computational resources.

    In this dissertation, I analyze the factors that determine the computational burden of retrieval-augmented language models and propose strategies to extract a better performance-compute trade-off. The dissertation consists of three sections. The first section contains a detailed analysis of components of retrieval-augmented models and introduces methods to improve generation efficiency. The second section explores the use of dense memory to reduce the cost of encoding retrievals. Finally, the third section proposes a hybrid between dense memory and text retrieval, combining lessons from previous chapters.

    Names of the Dissertation defense committee members:
    Chair: Leana Golubchik
    Fei Sha
    Dani Yogatama
    Jacob Bien

    Venue: Zoom, https://usc.zoom.us/my/lgzoomeeting

    Location: https://usc.zoom.us/my/lgzoomeeting

    Audiences: Everyone Is Invited

    Contact: Asiroh Cham


Return to Calendar