Events Calendar

PhD Defense - Michiel De Jong

Title: EXPANDING THE QUALITY-COMPUTE FRONTIER FOR RETRIEVAL-AUGMENTED LANGUAGE MODELS

Abstract: Retrieval-augmented language models set the state-of-the-art on a broad spectrum of knowledge-intensive tasks, outperforming orders of magnitude larger models. However, such models can also be expensive for training and inference. Model performance and computational cost represent two sides of the coin: we can generally improve performance through scale at the expense of an increased computational burden. Therefore, we are really interested in pushing out the quality-compute frontier, improving performance at any given level of computational resources.

In this dissertation, I analyze the factors that determine the computational burden of retrieval-augmented language models and propose strategies to extract a better performance-compute trade-off. The dissertation consists of three sections. The first section contains a detailed analysis of components of retrieval-augmented models and introduces methods to improve generation efficiency. The second section explores the use of dense memory to reduce the cost of encoding retrievals. Finally, the third section proposes a hybrid between dense memory and text retrieval, combining lessons from previous chapters.

Names of the Dissertation defense committee members:
Chair: Leana Golubchik
Members:
Fei Sha
Dani Yogatama
Jacob Bien

Venue: Zoom, https://usc.zoom.us/my/lgzoomeeting

Location: https://usc.zoom.us/my/lgzoomeeting

Audiences: Everyone Is Invited

Contact: Asiroh Cham