Events Calendar

Taylor Berg-Kirkpatrick (Carnegie Mellon University) – Balancing Constraint and Flexibility in Unsupervised Models for Language Analysis

Speaker: Taylor Berg-Kirkpatrick, Carnegie Mellon University

Talk Title: Balancing Constraint and Flexibility in Unsupervised Models for Language Analysis

Series: Computer Science Colloquium

Abstract: Without careful consideration of the relationship between input and output, unsupervised learning problems can be under-constrained. This talk will discuss approaches for making unsupervised problems feasible by incorporating different types of inductive bias. First, we focus on a set of raw data analysis tasks related to the digital humanities, including historical document recognition, music transcription, and compositor attribution. For each of these tasks, strong prior knowledge about the causal process behind the data can be encoded into the model. We show how to leverage this casual knowledge as a helpful source of constraint, yielding systems that in some cases outperform their supervised counterparts. Next, we investigate several linguistic analysis tasks where causal structure is more difficult to encode. Here, we develop a new unsupervised model class that combines structured and continuous representations by leveraging the flexibility of neural networks. We show that incorporating a volume-preserving constraint on the neural component of our model makes learning well-behaved. Using this approach, we demonstrate start-of-the-art results on two standard unsupervised NLP tasks: part-of-speech induction and unsupervised dependency parsing.

This lecture satisfies requirements for CSCI 591: Research Colloquium. Please note, due to limited capacity in OHE 100D, seats will be first come first serve.


Biography: Taylor Berg-Kirkpatrick joined the Language Technologies Institute at Carnegie Mellon University as an Assistant Professor in Fall 2016. Previously, he was a Research Scientist at Semantic Machines Inc. and, before that, completed his Ph.D. in computer science at the University of California, Berkeley. Taylor's research focuses on using machine learning to understand structured human data, including language but also sources like music, document images, and other complex artifacts.

Host: Computer Science Department

Location: Olin Hall of Engineering (OHE) - 100D

Audiences: Everyone Is Invited

Contact: Computer Science Department