Events Calendar

NL Seminar-Should PTLMs go to school?

Speaker: Manuel Ciosici, ISI Boston Waltham

Talk Title: Should PTLMs go to school?

Series: NL Seminar

Abstract: REMINDER Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you're highly encouraged to use your USC account to sign into Zoom. If you're an outside visitor, please inform nlg DASH seminar DASH admin2 AT isi.edu beforehand so we'll be aware of your attendance and let you in.

I will present a couple of research vignettes of the working knowledge inside large pre trained language models. I will use the vignettes to argue for a new task to measure modern language models knowledge and ability to learn from textbooks. Unlike machines, humans do not need to read, for example, all of Wikipedia, to learn.

For humans, reading a textbook or a manual is often enough to provide working knowledge on the books topic. We propose LEFT, a new task to measure a machines capacity to learn from the same textbooks that college graduates use to learn about society and history. The task reveals surprising results for current state of the art language models like T5 and GPT Neo.


Biography: Manuel Ciosici is a postdoc at ISI Boston (Waltham), working with Ralph Weischedel and Marjorie Friedman on understanding the knowledge inside large language models and putting it to use, for example, in filling in sequences of events. He is also interested in Natural Language Processing for languages other than English and has recently released a large corpus of Danish to support training large language models. Before joining ISI, Manuel received his Ph.D. from Aarhus University in Denmark and was a postdoc at the IT University in Copenhagen.

Host: Jon May and Mozhdeh Gheini

More Info: https://nlg.isi.edu/nl-seminar/

Webcast: https://youtu.be/ZtM7b0ggfvs

Location: Information Science Institute (ISI) - Virtual Only

WebCast Link: https://youtu.be/ZtM7b0ggfvs

Audiences: Everyone Is Invited

Contact: Pete Zamar

Event Link: https://nlg.isi.edu/nl-seminar/