Ph.D. Thesis Defense - Yuan Meng

PhD Thesis Defense - Yuan Meng    
 
Committee Members: Prof Viktor K. Prasanna (Chair), Prof. Bhaskar Krishnamachari, Prof. Yue Zhao    
 
Title: Accelerating Reinforcement Learning using Heterogeneous Platforms: Co-Designing Hardware, Algorithm, and System Solutions    
 
Abstract: Reinforcement Learning (RL) is an area of AI that constitutes a wide range of algorithms, enabling autonomous agents to learn optimal decisions through online environment interactions, data collection, and training. Recently, certain categories of RL algorithms have witnessed widespread adoption due to their generalizability and reliability, including model-free RL based on policy/value optimizations and model-based RL using Monte Carlo Tree Search. General-purpose processors fail to optimally achieve efficient execution speed for RL due to the intrinsic heterogeneous characteristics among various RL primitives and algorithms. Optimized acceleration systems that exploit heterogeneity across different architectures to support the variations of compute kernels and memory characteristics in RL are crucial to fast and efficient application development.   In this dissertation, we develop acceleration frameworks for two key categories of RL algorithms, i.e., model-free Deep RL, and model-based RL using Monte Carlo Tree Search (MCTS). We implement these frameworks by addressing two objectives: 1. We develop algorithm-hardware co-optimized accelerators for the fundamental primitives in the key categories of RL algorithms. This includes inference and training of DNN policy models, as well as dynamic tree-based operations in MCTS. 2. We create portable system solutions that identify the optimal primitive scheduling, mapping, and design configurations onto heterogeneous devices based on the task dependency, compute, and memory characteristics of the target RL algorithms.   Experiments on various platforms consisting of interconnected CPUs, FPGAs, and GPUs showcase superior performance enhancements across diverse models, algorithms, hardware platforms, and benchmark environments compared to state-of-the-art RL libraries.   
 
Bio: Yuan Meng is a fifth-year PhD candidate in Computer Engineering, advised by Professor Viktor K. Prasanna. She obtained her BS degree in electrical and computer engineering at Rensselaer Polytechnic Institute. Her research interests include parallel computing, deep learning acceleration, heterogeneous computing, and reinforcement learning.     
 
Date: Thursday, June 20th, 2024
Time: 2pm
Location:  EEB 132
Zoom Link: https://usc.zoom.us/j/8629150353  

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 132

Audiences: Everyone Is Invited

Contact: CS Events

Event Link: ://usc.zoom.us/j/8629150353