Past Event: Babuška Forum

Bisimulation and Reinforcement Learning

Amy Zhang, Assistant Professor, Chandra Family Department of Electrical and Computer Engineering, UT Austin

Abstract

Most real world environments contain structure that would aid sample efficiency and generalization in control tasks if it were leveraged by algorithms.  In this talk, we propose to incorporate different assumptions that better reflect the real world and allow the design of novel algorithms with theoretical guarantees to address this fundamental problem. We first present how state abstractions can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn state abstractions that both provide for effective downstream control and invariance to task-irrelevant details. We use bisimulation metrics to quantify behavioral similarity between states, and learn robust latent representations which encode only the task-relevant information from observations. We provide theoretical guarantees for the learned approximate abstraction and extend this notion to multi-task settings.

Biography

Dr. Zhang is an assistant professor at UT Austin in the Chandra Family Department of Electrical and Computer Engineering. She is also a Texas Instruments/Kilby Fellow in the Department of Electrical and Computer Engineering starting Spring 2023 and an affiliate member of the Texas Robotics Consortium. Her work focuses on improving generalization in reinforcement learning through bridging theory and practice in learning and utilizing structure in real world problems. Previously, she was a research scientist at Meta AI - FAIR and a postdoctoral fellow at UC Berkeley. Zhang obtained her PhD from McGill University and the Mila Institute, and also previously obtained an M.Eng. in EECS and dual B.Sci. degrees in Mathematics and EECS from MIT.