Imitation Learning for Robotics
This course covers how robots can acquire skills from human demonstrations and interaction. It moves from physics-centric approaches, where motion is encoded as stable dynamical systems and movement primitives with formal guarantees, to data-centric approaches such as behavior cloning, interactive imitation, and generative policies that scale to visually complex, high-dimensional tasks.
Course Logistics
Class timings: Tuesday and Thursday, 15:30 to 17:00; Classroom: G10, IDR; Microsoft Teams code: zo0qgmz
Course Contact
Instructor: Prof. Ravi Prakash (ravipr@iisc.ac.in, or message on Teams)
Syllabus
Foundations
Imitation learning versus explicit-cost methods such as RL and planning; dynamical systems as implicit cost and stability; regression techniques (linear, polynomial, and kernel) for mapping states to actions.
Physics-Centric Models
Stable estimators with Lyapunov guarantees (SEDS, GMM-GMR); phase-based and task-parameterized models (DMP, ProMP, TP-GMM) for temporal patterns and frame-relative adaptation.
Physical Human-Robot Interaction
Variable impedance control; balancing stiffness and compliance for safe physical interaction; haptic and kinesthetic teaching.
Data-Driven and Generative Policies
Behavior cloning from pixels to actions; distribution shift and interactive correction with DAgger; diffusion policies and action chunking (ACT) for multi-modal, dexterous tasks; vision-language-action models for semantic task specification.
Class and Theory
Lectures develop each family of methods from first principles, contrasting what is learned, which guarantees hold, and how each approach generalizes to new situations.
Lab and Hands-on Sessions
Each module pairs with a lab in Python and MuJoCo simulation, and on Franka robot arms for haptic teaching and physical interaction.
Final Project
Teams of two choose a research paper from the provided pool, replicate its results, and extend it with their own ideas.
Prerequisites
A basic understanding of linear algebra, probability, and machine learning, along with some exposure to coding in Python.
References
C. Celemin, R. Pérez-Dattari, E. Chisari, G. Franzese, L. de Souza Rosa, R. Prakash, Z. Ajanović, M. Ferraz, A. Valada, and J. Kober, Interactive Imitation Learning in Robotics: A Survey, Foundations and Trends in Robotics, 2022.
T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, An Algorithmic Perspective on Imitation Learning, Foundations and Trends in Robotics, 2018.
S. Chernova and A. L. Thomaz, Robot Learning from Human Teachers, Morgan & Claypool, 2014.
A. Billard, S. Mirrazavi, and N. Figueroa, Learning for Adaptive and Reactive Robot Control: A Dynamical Systems Approach, MIT Press, 2022.
Course Evaluation
Format to be decided
Written examination
Programming assignments based on lectures
Teams of two select a research paper from the provided pool. Evaluation considers replication of results and novel ideas.
Course Plan
| Module | Topic and key concepts | Lab or practical activity |
|---|---|---|
| I: Foundations | Introduction to imitation learning and dynamical systems: explicit cost (RL, planning) versus implicit cost and stability | Vector fields: coding stable reaching attractors in Python |
| II: Foundational math | Regression techniques (linear, polynomial, kernel): state-to-action mapping as a predictive task | Fitting trajectories with standard regression models |
| III: Physics-centric | State-based models: SEDS and GMM-GMR, stable estimators with Lyapunov guarantees | Implementing SEDS; testing robustness to physical pushes |
| IV: Physics-centric | Phase and task parameters: DMP, ProMP, and TP-GMM for temporal patterns and frame-relative adaptation | Training DMPs for pick-and-place tasks |
| V: Human-robot interaction | Impedance and safety: variable impedance, balancing stiffness and compliance for physical interaction | Franka lab: haptic teaching and physical interaction |
| VI: Data-driven | Behavior cloning and DAgger: supervised mapping from pixels to actions, addressing distribution shift | MuJoCo: training a navigation policy, using DAgger for correction |
| VII: High-capacity | Diffusion policies and ACT: multi-modality and sequence modeling for dexterous tasks | Training generative policies for high-DOF manipulation (e.g., pouring) |
| VIII: Semantic (if time permits) | Vision-language-action models: LLMs for high-level task parameterization and compositionality | Language-to-action sequence generation |