Course Schedule
Lecture slides are posted here and on [Piazza]. All due dates are at 11:59 PM ET.
| Date | Lecture | Optional Readings | Logistics | Topic Groups |
|---|---|---|---|---|
| 01/13 | Lecture 1: Course Introduction: What is Robot Learning? [slides] | [Building Machines That Learn and Think Like People] | 🔴 Introduction | |
| 01/15 | Lecture 2: Robot Learning: An Overview [slides] | [RL Textbook, Ch 1] | 🔴 Introduction | |
| 01/20 | MLK Jr Day | |||
| 01/22 | Lecture 3: ML/DL Refresher Part 1 [slides] | [DL Textbook, Ch 5-10] | HW1 Out | 🟠 ML/DL Refresher |
| 01/27 | Lecture 4: ML/DL Refresher Part 2 [slides] | [DL Textbook, Ch 5-10] | 🟠 ML/DL Refresher | |
| 01/29 | Lecture 5: MDP Basics and Imitation Learning Part 1 [slides] | [ICML Tutorial][An Invitation to Imitation] | 🟣 Imitation Learning | |
| 02/03 | Lecture 6: Imitation Learning Part 2 [slides] | [DAgger][GAIL][Diffusion Policy][pi-zero] | 🟣 Imitation Learning | |
| 02/05 | Lecture 7: RL Basics: Value/Policy Iteration [slides] | [RL Textbook, Ch 3-4][Key Concepts in RL][Kinds of RL Algorithms] | 🟢 Model-Free RL | |
| 02/10 | Lecture 8: Q-Learning and Variants [slides] | [RL Textbook, Ch 5-7][DQN] | HW1 Due; HW2 Out | 🟢 Model-Free RL |
| 02/12 | Lecture 9: Policy Gradient Methods [slides] | [RL Textbook, Ch 13][Intro to Policy Gradient] | 🟢 Model-Free RL | |
| 02/17 | Lecture 10: Actor-Critic Methods [slides] | [RL Textbook, Ch 13] | 🟢 Model-Free RL | |
| 02/19 | Lecture 11: Advanced RL Algorithms Part 1 [slides] | [PPO][TRPO][DDPG][SAC] | 🟢 Model-Free RL | |
| 02/24 | Lecture 12: Advanced RL Algorithms Part 2 | HW2 Due; HW3 Out | 🟢 Model-Free RL | |
| 02/26 | Lecture 13: Model-Based Control Basics | [Feedback Systems Textbook] | 🔵 Model-Based RL | |
| 03/03 | Spring Break | Project Proposal Due | ⛱️ | |
| 03/05 | Spring Break | ⛱️ | ||
| 03/10 | Lecture 14: Optimal Control and Planning Part 1 | [Murray's Notes][iLQR][DDP][SCP][DIAL-MPC] | 🔵 Model-Based RL | |
| 03/12 | Lecture 15: Optimal Control and Planning Part 2 | [PETS][Neural-Control Family][MPPI][PILCO][MBPO] | 🔵 Model-Based RL | |
| 03/17 | Lecture 16: Deep Model-Based RL | [Dreamer][TD-MPC] | HW3 Due; HW4 Out | 🔵 Model-Based RL |
| 03/19 | Lecture 17: Bandits and Preference-Based Learning | [RL Textbook, Ch 2][Dueling Bandits] | 🟡 Bandits and Exploration | |
| 03/24 | Lecture 18: Exploration | [Curiosity][RND] | 🟡 Bandits and Exploration | |
| 03/26 | Lecture 19: Guest Lecture: Offline RL (Aviral Kumar) | [NeurIPS Tutorial][IQL][Diffuser] | ⚪ RL from Offline Data | |
| 03/31 | Lecture 20: Inverse RL | [Maximum Entropy IRL][LP-IRL] | ⚪ RL from Offline Data | |
| 04/02 | Lecture 21: Foundation Models in Robotics (Yunzhu Li) | [Survey][SayCan][CLIPort][RT-1][Code as Policies] | HW4 Due | 🟤 Specialized Topics |
| 04/07 | Lecture 22: Robot Simulation and Sim2Real | [Genesis][Domain Randomization][Champion-Level Drone Racing] | 🟤 Specialized Topics | |
| 04/09 | Lecture 23: Safe RL and Safe Robot Learning | [Safe Robot Learning Survey][Data-Driven Safety Filters] | 🟤 Specialized Topics | |
| 04/14 | Lecture 24: Multi-Task/Adaptable/Transferable Robot Learning | [Teacher-Student][RMA][Neural-Fly] | 🟤 Specialized Topics | |
| 04/16 | Lecture 25: Course Summary | ⚫ Project | ||
| 04/21 | Lecture 26: Student Project Presentations | ⚫ Project | ||
| 04/23 | Lecture 27: Student Project Presentations | ⚫ Project | ||
| 05/03 | Final Report Due |