Embodied Artificial Intelligence Safety
Spring 2025. 16-886. Monday / Wednesday 11:00-12:20.
Announcements
Course Overview
Safety is a nuanced concept. For embodied systems, like robots, we commonly equate safety with collision-avoidance. But out in the “open world” it can be much more: for example, a safe mobile manipulator should understand when it is not confident about a requested task and understand that areas roped off by caution tape should never be breached. However, designing systems with such a nuanced understanding is an outstanding challenge, especially in the era of large robot behavior models.
In this graduate seminar class, we study the question of if (and how) the rise of modern artificial intelligence (AI) models (e.g., deep neural trajectory predictors, large vision-language models, and latent world models) can be harnessed to unlock new avenues for generalizing safety to the open world. From a foundations perspective, we study safety methods from two complementary communities: control theory (which enables the computation of safe decisions) and machine learning (which enables uncertainty quantification and anomaly detection). Throughout the class, there will also be several guest lectures from experts in the field. Students will practice essential research skills including reviewing papers, writing project proposals, and technical communication.
Prerequisites
The course is open to graduate students and advanced undergraduates. While there are no strict prerequisites, familiarity with sequential decision-making, machine learning, optimization, and probability are highly encouraged. Experience with high-level programming languages like Python or MATLAB are also strongly encouraged.
Schedule (Tentative)
Control-Theoretic Safety Foundations
- Jan. 13
- Course Overview
- Syllabus
- Jan. 15
- Sequential Decision-Making
- Jan. 20
- No Class MLK Day
- Jan. 22
- Safety Filtering
- Data-Driven Safety Filters, Model Predictive Sheilding, Safety & Liveness of Filters
- Jan. 27
- Safety Filter Synthesis via Optimal Control
- Jan. 29
- Robust Safety
- Differential Games I, HJI
- Feb. 3
- Computational Frameworks for Safety I
- Discounted Reachability, ISAACS
- Feb. 5
- Computational Frameworks for Safety II
- HW #1 Due DeepReach, One Filter to Deploy Them All
Frontiers I
- Feb. 10
- Semantic Safety I
- Safety Representations from Language, Local Updates
- Feb. 12
- Semantic Safety II
- Paper Reading Semantically Safe Robot Manipulation, SALT
- Feb. 17
- Belief-Space Safety
- Deception Game, Analyzing Models that Adapt Online
- Feb. 19
- Feb. 24
- Latent-Space Safety II
- Paper Reading Human-AI Safety
- Feb. 26
- Failure Monitoring & Recovery via VLMs
- HW #2 Due Paper Reading LLM Fallbacks, AHA
- Mar. 3
- No Class Spring Break 🏝️
- Mar. 5
- No Class Spring Break 🏝️
Machine Learning & Statistical Safety Foundations
- Mar. 10
- Bayesian Uncertainty Quantification
- Gaussian Process Lecture Notes from Drew Bagnell, GPs Book
- Mar. 12
- Ensembles
- Mid-term Report DueEnsembles, EnsembleDAgger
- Mar. 17
- Uncertainty in Large Behavior Models
- Paper Reading Diffusion Policy, TBD
- Mar. 19
- Guest Lecture Prof. Anushri Dixit (UCLA), Conformal Prediction
- Gentle Intro to Conformal, KnowNo, Perceive With Confidence
- Mar. 24
- Alignment
- CPL, Max Alignment Min Feedback
- Mar. 26
- Risk-Aware Decision-Making
- Paper Reading What is Risk in Robotics?, Risk-Calibrated Interaction
- Mar. 31
- “System-level” Anomalies
- Not All Errors, System-Level OOD, BYOVLA
- Apr. 2
- Guest Lecture Prof. Max Simchowitz (CMU), Mathematical Foundations of Robotic Behavior Cloning
- HW #3 Due
Frontiers II
- Apr. 7
- Guest Lecture Dr. Masha Itkina (TRI), Out-of-Distribution and Failure Detection
- Apr. 9
- Controlling In-Distribution
- Paper Reading In-D CBF, Lyapunov Density Models
- Apr. 14
- Guest Lecture Dr. Haruki Nishimura (TRI), Statistical Assurances for Learned Policies
- Apr. 16
- Statistical Assurances
- Paper Reading Statistical Safe Set Verification, How Generalizable is My BC Policy?
Project Presentations
- Apr. 21
- Project Presentations
- Slides Due 11:59 pm ET, April 20 Presenters TBD
- Apr. 23
- Project Presentations
- Project Report Due May 1 Presenters TBD
Instructor
Teaching Assistant
