CS Colloquium: Junzhe Zhang - Towards Causal Reinforcement Learning

Speaker: Junzhe Zhang, Columbia University

Talk Title: Towards Causal Reinforcement Learning

Abstract: Causal inference provides a set of principles and tools that allows one to combine data and knowledge about an environment to reason with questions of a counterfactual nature - i.e., what would have happened if the reality had been different - even when no data of this unrealized reality is currently available. Reinforcement learning provides a collection of methods that allows the agent to reason about optimal decision-making under uncertainty by trial and error - i.e., what would the consequences (e.g., subsequent rewards, states) be had the action been different? While these two disciplines have evolved independently and with virtually no interaction, they operate over various aspects of the same building block, i.e., counterfactual reasoning, making them umbilically connected.   This talk will present a unified theoretical framework, called causal reinforcement learning, that explores the nuanced interplays between causal inference and reinforcement learning. I will discuss a recent breakthrough in partial identification that allows one to infer unknown causal effects from a combination of model assumptions and available data. Delving deeper, I will then demonstrate how this method could be applicable to address some practical challenges in classic reinforcement learning tasks, including robust off-policy evaluation from confounded observations and accelerating online learning with offline data.     This lecture satisfies requirements for CSCI 591: Research Colloquium

Biography: Junzhe Zhang is a postdoctoral research scientist in the Causal AI lab at Columbia University. He obtained his doctoral degree in Computer Science at Columbia University, advised by Elias Bareinboim. His research centers on causal inference theory and its applications in reinforcement learning, algorithmic fairness, and explainability. His works have been selected for oral presentations in top refereed venues such as NeurIPS.

Host: Sven Koenig

Location: Olin Hall of Engineering (OHE) - 132

Audiences: Everyone Is Invited

Contact: CS Faculty Affairs