Robot Manipulation
This course introduces the fundamental algorithmic approaches used to create robot systems that can autonomously manipulate physical objects in unstructured environments such as homes and healthcare settings.
Course Logistics
Class timings: Tuesday and Thursday, 11:30 to 13:00; Classroom: G12, TSH; Microsoft Teams code: qbjin4o
Course Contact
Instructor: Prof. Ravi Prakash (ravipr@iisc.ac.in)
Teaching assistants: Akhil R Kurup (akhilkurup@iisc.ac.in), Ipsita Basak (ipsitabasak@iisc.ac.in), Dasari Indu Krishna (dasarik@iisc.ac.in)
Syllabus
Modelling and Mechanics
Anatomy of manipulation systems; model-based design and simulation in Drake; coordinate frames; forward, inverse, and differential kinematics; statics, contact, and grasping.
Robot Perception
Cameras and 3D geometry; point clouds and registration; object recognition and segmentation; affordance perception for manipulation.
Planning
Sampling-based motion planning; local trajectory optimization; global optimization using Graphs of Convex Sets; task-and-motion planning; planning through contact and under uncertainty.
Control and Robot Policies
Position, force, and manipulator control; model predictive control; scripts, state machines, and behaviour trees; reinforcement learning, policy gradients, PPO, and visuomotor behaviour cloning.
Class and Theory
Topics include perception using deep learning and 3D geometry; grasping; robot kinematics and trajectory generation; collision-free motion planning; task-and-motion planning; planning under uncertainty; and model-based and learning-based dynamics and control for robotic manipulation.
Lab and Hands-on Sessions
Students build a software stack in simulation or on a real robot, enabling a robotic arm to manipulate objects in cluttered scenes such as a kitchen. The lab connects theory with practical implementation and develops the problem-solving skills needed for ambitious projects in simulation or on hardware.
Final Project
Students explore a specific aspect of robot manipulation in greater depth. Lab hardware is available for ambitious projects, while cloud resources make simulation-based projects accessible as well.
Course Evaluation
Format to be decided
Written examination
Notebooks and programming assignments based on lectures
Teams of two select a research paper from the provided pool and submit a proposal, progress update, report, and video. Evaluation considers replication of results and novel ideas.
Course Calendar
The calendar below follows the schedule presented in the course introduction.
| Date | Lecture and topic | Task or milestone |
|---|---|---|
| Aug 6 | Lecture 1: Anatomy of a manipulation system | First day of classes |
| Aug 11 | Lecture 2: Model-based design and simulation in Drake | None |
| Aug 13 | Lecture 3: Basic pick and place, frames | PS1: Simulation released Aug 14 |
| Aug 18 | Lecture 4: Basic pick and place, kinematics | None |
| Aug 20 | Lecture 5: Differential kinematics via optimization | PS1 due Aug 21; PS2: Kinematics I released |
| Aug 25 & 27 | No class | None |
| Sep 1 | Lecture 6: Geometric perception, cameras, point clouds, registration | None |
| Sep 3 | Lecture 7: Manipulation in clutter, statics and contact | PS2 due Sep 4; PS3: Kinematics II and ICP released |
| Sep 8 | Lecture 8: Manipulation in clutter, grasping | Project paper pool released |
| Sep 10 | Lecture 9: Motion planning, basics and sampling | PS3 due Sep 11; PS4: Grasping released |
| Sep 15 | Lecture 10: Motion planning, local optimization | None |
| Sep 17 | Lecture 11: Motion planning, global optimization (GCS) | PS4 due Sep 18; PS5: Planning I released |
| Sep 22 | Lecture 12: Position control | None |
| Sep 24 | Lecture 13: Force control | PS5 due Sep 25; PS6: Planning II released |
| Sep 29 | Lecture 14: Manipulator control | Project proposal due |
| Oct 1 | No lecture | Mid-term: lab exam and viva |
| Oct 2 | No lecture | PS6 due; PS7: Control released |
| Oct 6 | Lecture 15: Model predictive control | None |
| Oct 8 | Lecture 16: Programming behaviours, scripts, state machines, behaviour trees | None |
| Oct 13 | Lecture 17: Deep perception, object recognition and segmentation | None |
| Oct 15 | Lecture 18: Deep perception, affordances | PS7 due Oct 16 |
| Oct 20 | Lecture 19: Reinforcement learning overview | None |
| Oct 22 | Lecture 20: RL, policy gradients and PPO | None |
| Oct 27 | Lecture 21: Visuomotor policies, behaviour cloning | Project progress checkpoint |
| Oct 29 | Lecture 22: Task and motion planning | PS8: Deep perception and RL released Oct 30 |
| Nov 3 | Lecture 23: Planning through contact | PS8 due Nov 6 |
| Nov 5 & 10 | Buffer / holiday | None |
| Nov 12 | Lecture 24: Project report summaries | Final report and video due; last day of classes |
Prerequisites
A basic understanding of linear algebra, probability, and algorithms, along with some exposure to coding in Python.
Reference
John J. Craig, Introduction to Robotics: Mechanics and Control, Addison-Wesley.