## Course Schedule

Lecture slides are posted here and on Piazza. All due dates are at 11:59 PM ET.

Date | Lecture | Optional Readings | Logistics | Topic Groups |
---|---|---|---|---|

08/27 | Lecture 1: Course Introduction: What is Robot Learning? [slides] | [Building Machines That Learn and Think Like People] | 🔴 Introduction | |

08/29 | Lecture 2: Robot Learning: An Overview [slides] | [RL Textbook, Ch 1] | 🔴 Introduction | |

09/03 | Lecture 3: ML/DL Refresher Part 1 [slides] | [DL Textbook, Ch 5-10] | HW1 Out | 🟠 ML/DL Refresher |

09/05 | Lecture 4: ML/DL Refresher Part 2 [slides] | [DL Textbook, Ch 5-10] | 🟠 ML/DL Refresher | |

09/10 | Lecture 5: MDP Basics and Imitation Learning Part 1 [slides] | [ICML Tutorial][An Invitation to Imitation] | 🟣 Imitation Learning | |

09/12 | Lecture 6: Imitation Learning Part 2 [slides] | [DAgger][GAIL][Diffusion Policy][Transporter] | 🟣 Imitation Learning | |

09/17 | Lecture 7: RL Basics: Value/Policy Iteration [slides] | [RL Textbook, Ch 3-4][Key Concepts in RL][Kinds of RL Algorithms] | 🟢 Model-Free RL | |

09/19 | Lecture 8: Q-Learning and Variants [slides] | [RL Textbook, Ch 5-7][DQN] | HW1 Due; HW2 Out | 🟢 Model-Free RL |

09/24 | Lecture 9: Policy Gradient Methods [slides] | [RL Textbook, Ch 13][Intro to Policy Gradient] | 🟢 Model-Free RL | |

09/26 | Lecture 10: Actor-Critic Methods [slides] | [RL Textbook, Ch 13] | 🟢 Model-Free RL | |

10/01 | Lecture 11: Advanced RL Algorithms [slides] | [PPO][TRPO][DDPG][SAC] | 🟢 Model-Free RL | |

10/03 | Lecture 12: Model-Based Control Basics [slides] | [Feedback Systems Textbook] | HW2 Due; HW3 Out (Oct 5) | 🔵 Model-Based RL |

10/08 | Lecture 13: Optimal Control and Planning Part 1 [slides] | [Murray's Notes][iLQR][DDP][SCP] | 🔵 Model-Based RL | |

10/10 | Lecture 14: Optimal Control and Planning Part 2 [slides] | [PETS][Neural-Control Family][MPPI][PILCO][MBPO] | Project Proposal Due (Oct 11) | 🔵 Model-Based RL |

10/15 | Fall Break | ⛱️ | ||

10/17 | Fall Break | ⛱️ | ||

10/22 | Lecture 15: Deep Model-Based RL | [Dreamer][TD-MPC] | 🔵 Model-Based RL | |

10/24 | Lecture 16: Bandits and Preference-Based Learning | [RL Textbook, Ch 2][Dueling Bandits] | 🟡 Bandits and Exploration | |

10/29 | Lecture 17: Exploration | [Curiosity][RND] | HW4 Out | 🟡 Bandits and Exploration |

10/31 | Lecture 18: Guest Lecture: Offline RL (Aviral Kumar) | [NeurIPS Tutorial][IQL][Diffuser] | HW3 Due | ⚪ RL from Offline Data |

11/05 | Democracy Day | 🗳 | ||

11/07 | Lecture 19: Inverse RL | [Maximum Entropy IRL][LP-IRL] | ⚪ RL from Offline Data | |

11/12 | Lecture 20: Robot Simulation and Sim2Real | [Domain Randomization][Champion-Level Drone Racing] | 🟤 Specialized Topics | |

11/14 | Lecture 21: Safe RL and Safe Robot Learning | [Safe Robot Learning Survey][Data-Driven Safety Filters] | HW4 Due | 🟤 Specialized Topics |

11/19 | Lecture 22: Guest Lecture: Foundation Models in Robotics (Yonatan Bisk) | [Survey][SayCan][CLIPort][RT-1][Code as Policies] | 🟤 Specialized Topics | |

11/21 | Lecture 23: Multi-Task/Adaptable/Transferable Robot Learning | [Teacher-Student][RMA][Neural-Fly] | 🟤 Specialized Topics | |

11/26 | Lecture 24: Course Summary | ⚫ Project | ||

11/28 | Thanksgiving | 🦃️ | ||

12/03 | Lecture 25: Student Project Presentations | ⚫ Project | ||

12/05 | Lecture 26: Student Project Presentations | ⚫ Project | ||

12/13 | Project Report Due | ✨ |