BEGIN:VCALENDAR BEGIN:VEVENT SUMMARY:NL Seminar-The Unreasonable Syntactic Expressivity of RNNs DESCRIPTION:Speaker: John Hewitt, Stanford University Talk Title: The Unreasonable Syntactic Expressivity of RNNs Series: NL Seminar Abstract: In 2015, Andrej Karpathy posted a now famous blog post on The Unreasonable Effectiveness of Recurrent Neural Networks. To summarize this sense of wonder, Karpathy emphasized We will train RNNs to generate text character by character and ponder the question how is that even possible? RNNs empirically generate natural language with high syntactic fidelity, but their success is not well understood theoretically. In this talk, I will provide theoretical insight into this success, proving in a finite precision setting that RNNs can efficiently generate bounded hierarchical languages that reflect the scaffolding of natural language syntax. I will introduce Dyck k,m, the language of well nested brackets of k types and m bounded nesting depth, reflecting the bounded memory needs and long distance dependencies of natural language syntax. The best previously known results use Ok m 2 memory hidden units to generate these languages. I will prove that an RNN with O m log k hidden units suffices, an exponential reduction in memory, by an explicit construction. Finally, I will show that no algorithm, even with unbounded computation, can suffice with o m log k hidden units. Biography: John is a 3rd year PhD student in computer science at Stanford University, advised by Chris Manning and Percy Liang. He works on understanding and improving how unsupervised neural networks learn and process human languages. He is supported by a National Science Foundation Graduate Research Fellowship, and is the recipient of an EMNLP Runner Up Best Paper award. \n Host: Jon May and Mozhdeh Gheini More Info: https://nlg.isi.edu/nl-seminar/ Webcast: https://usc.zoom.us/j/95584315616 DTSTART:20201112T110000 LOCATION:ISI Virtual Only URL;VALUE=URI:https://nlg.isi.edu/nl-seminar/ DTEND:20201112T120000 END:VEVENT END:VCALENDAR