Embodied Artificial Intelligence Safety
Spring 2025. 16-886. Monday / Wednesday 11:00-12:20.
Announcements
Course Overview
Safety is a nuanced concept. For embodied systems, like robots, we commonly equate safety with collision-avoidance. But out in the “open world” it can be much more: for example, a safe mobile manipulator should understand when it is not confident about a requested task and understand that areas roped off by caution tape should never be breached. However, designing systems with such a nuanced understanding is an outstanding challenge, especially in the era of large robot behavior models.
In this graduate seminar class, we study the question of if (and how) the rise of modern artificial intelligence (AI) models (e.g., deep neural trajectory predictors, large vision-language models, and latent world models) can be harnessed to unlock new avenues for generalizing safety to the open world. From a foundations perspective, we study safety methods from two complementary communities: control theory (which enables the computation of safe decisions) and machine learning (which enables uncertainty quantification and anomaly detection). Throughout the class, there will also be several guest lectures from experts in the field. Students will practice essential research skills including reviewing papers, writing project proposals, and technical communication.
Prerequisites
The course is open to graduate students and advanced undergraduates. While there are no strict prerequisites, familiarity with sequential decision-making, machine learning, optimization, and probability are highly encouraged. Experience with high-level programming languages like Python or MATLAB are also strongly encouraged.
Schedule (Tentative)
Control-Theoretic Safety Foundations
- Jan. 13
- Jan. 15
- Sequential Decision-Making [Notes]
- Jan. 20
- No Class MLK Day
- Jan. 22
- Jan. 27
- Safety Filter Synthesis via Optimal Control [Notes]
- HJ Reachability Overview, HJ Viscosity Solution
- Jan. 29
- Synthesis & Robust Safety I [Notes]
- Differential Games, HJ Reach-Avoid Games I, HJ Reach-Avoid Games II
- Feb. 3
- Robust Safety II [Notes]
- Feb. 5
- Computational Frameworks for Safety [Notes]
- HW #1 Due Discounted Reachability, ISAACS, DeepReach
Frontiers I
- Feb. 10
- Updating Safety Online [Notes] [Slides]
- Reachability Adapted with Gaussian Processes, Local Updates, Parameterized Reachability
- Feb. 12
- Semantic Safety
- Paper Reading Safety Representations from Language, SALT
- Feb. 17
Belief-Space SafetyProject Brainstorming [Slides]- Deception Game, Analyzing Models that Adapt Online
- Feb. 19
- From Belief to Latent-Space Safety [Notes] [Slides]
- Deception Game, Latent Safety Filters
- Feb. 24
- Latent-Space Safety
- Paper Reading Latent Safety Filters, LS3
- Feb. 26
- Failure Monitoring & Recovery via VLMs
- HW #2 Due: Feb 28 Paper Reading LLM Fallbacks, FOREWARN
- Mar. 3
- No Class Spring Break 🏝️
- Mar. 5
- No Class Spring Break 🏝️
Machine Learning & Statistical Safety Foundations
- Mar. 10
- Uncertainty Quantification I [Notes]
- On the Calibration of Modern NNs, Prof. Eric Nalisnick’s research and talks
- Mar. 12
- Uncertainty Quantification II [Notes]
- Mid-term Report Due: March 14Deep Ensembles, Deep Laplace Approx, Gaussian Processes Book, What Are Bayesian NN Posteriors Really Like?
- Mar. 17
- Uncertainty in Large Behavior Models
- Paper Reading EnsembleDAgger, Robots that Ask for Help
- Mar. 19
- Guest Lecture Prof. Anushri Dixit (UCLA), Conformal Prediction [Video]
- Gentle Intro to Conformal, Perceive With Confidence
- Mar. 24
- “System-level” Anomalies [Notes] [Slides]
- Not All Errors, System-Level OOD, BYOVLA
- Mar. 26
- Risk-Aware Decision-Making
- Paper Reading What is Risk in Robotics?, Risk-Calibrated Interaction
- Mar. 31
- Guest Lecture Ran (Thomas) Tian (UC Berkeley) Alignment
- CPL, Max Alignment Min Feedback
Frontiers II
- Apr. 2
- Guest Lecture Prof. Max Simchowitz (CMU), Mathematical Foundations of Robotic Behavior Cloning
- HW #3 Due
- Apr. 7
- Guest Lecture Dr. Masha Itkina (TRI), Out-of-Distribution and Failure Detection
- Apr. 9
- Controlling In-Distribution
- Paper Reading In-D CBF, Lyapunov Density Models
- Apr. 14
- Guest Lecture Dr. Haruki Nishimura (TRI), Statistical Assurances for Learned Policies
- Apr. 16
- Statistical Assurances
- Paper Reading Statistical Safe Set Verification, How Generalizable is My BC Policy?
Project Presentations
- Apr. 21
- Project Presentations
- Slides Due 11:59 pm ET, April 20 Presenters TBD
- Apr. 23
- Project Presentations
- Project Report Due May 1 Presenters TBD
Instructor

Teaching Assistant
