CS Colloquium - Pavithra Prabhakar (Kansas State University) - Safety Analysis of AI-enabled Cyber-Physical Systems (CPS): A Formal Approach

Speaker: Prof. Pavithra Prabhakar, Kansas State University

Talk Title: Safety Analysis of AI-enabled Cyber-Physical Systems (CPS): A Formal Approach

Abstract: AI-based components have become an integral part of Cyber-Physical Systems enabling transformative functionalities. With the ubiquitous use of Machine Learning components in perception, control and decision making in safety critical application domains such as automotive and aerospace, rigorous analysis of these systems has become imperative toward real-world deployment. In this talk, we will present a formal approach to verifying the safety of AI-enabled CPS. We consider a closed-loop system consisting of a dynamical system model of the physical plant and a neural network model of the perception/control modules and analyze the safety of this system through reachable set computation. 
 
 
One of the main challenges with reachable set computation of neural network-controlled CPS is the scalability of the methods to large networks and complex dynamics. We present a novel abstraction technique for neural network size reduction that provides soundness guarantees for safety analysis and indicates a promising direction for scalable analysis of the closed-loop system.  Specifically, our abstraction consists of constructing a simpler neural network with fewer neurons, albeit with interval weights called interval neural network (INN), which over-approximates the output range of the given neural network. We present two methods for computing the output range analysis problem on the INNs, one by reducing it to solving a mixed integer linear programming problem, and the other a symbolic computation method using a novel data structure called the interval star set. Our experimental results highlight the trade-off between the computation time and the precision of the computed output set. We will discuss other foundational questions on neural network size reduction by exploring the notion of equivalence and approximate equivalence. We will conclude by pointing to ongoing work on incorporating a camera model along with a neural network for perception in the closed-loop system framework.
 
 
This lecture satisfies requirements for CSCI 591: Research Colloquium.

Biography: Pavithra Prabhakar is professor in the department of computer science, and the Peggy and Gary Edwards Chair in Engineering at Kansas State University. She is currently serving the National Science Foundation as a Program Director in the Software and Hardware Foundations Cluster in the Computer and Information Science and Engineering Directorate, where she manages formal methods and verification portfolio. Specifically, she leads the Formal Methods in the Field  (FMitF) program, has been a founding program director for the Safe Learning Enabled Systems (SLES) program and is a cognizant program director for the Foundations of Robotics Research (FRR) and the Cyber-Physical Systems (CPS) program. 
 
 
She obtained her doctorate in computer science and a master's degree in applied mathematics from the University of Illinois at Urbana-Champaign, followed by a CMI postdoctoral fellowship at the California Institute of Technology. Prior to coming to K-State, she spent four years at the IMDEA Software Institute in Spain as a tenure-track assistant professor. She is the recipient of a Marie Curie Career Integration Grant from the European Union (2014), an NSF CAREER Award (2016), an ONR Young Investigator Award (2017), NITW distinguished young alumnus award (2021), and an Amazon Research Award (2022).

Host: Jyotirmoy Deshmukh

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 248

Audiences: Everyone Is Invited

Contact: CS Events