Lecture Logistics#
Lectures will be held in person, in SH 105.
Course Schedule#
Lecture slides are posted on Piazza.
| Event | Date | Lecture | Optional Readings | Logistics |
|---|---|---|---|---|
| Lecture 1 | 08/29/2023 | Intro to Machine Learning, Decision Making, RL vs Supervised Learning. | ||
| Lecture 2 | 08/31/2023 | Supervised Learning | ||
| Lecture 3 | 09/05/2023 | ML/DL Refresher (Optimization, Backprop) Part I | HW 1 Released | |
| Lecture 4 | 09/07/2023 | ML/DL Refresher (Training, Advanced Topics) Part II | ||
| Lecture 5 | 09/12/2023 | Imitation Learning (Behavior Cloning, Dagger) | DAgger | |
| Lecture 6 | 09/14/2023 | Imitation Learning & DAGGER Analysis | ||
| Lecture 7 | 09/19/2023 | Intro to RL, MDP, Value/Policy Iteration | Key Concepts in RL, Kinds of RL Algorithms | |
| Lecture 8 | 09/21/2023 | Off Policy, Model-free, Q Learning | Reinforcement Learning: An Introduction Ch. 3-7, 12, DQN | HW 1 Due; HW 2 Released |
| Lecture 9 | 09/26/2023 | Intro to Policy Gradients | Reinforcement Learning: An Introduction Ch. 13, Intro to Policy Optimization | |
| Lecture 10 | 09/28/2023 | Policy Gradients, REINFORCE, Actor Critic | ||
| Lecture 11 | 10/03/2023 | Policy Gradients Part II and Popular RL Algos | ||
| Lecture 12 | 10/05/2023 | Popular RL Algos Part II (TRPO, PPO and DQN variants) | PPO, TRPO | HW 2 Due; HW 3 Released |
| Lecture 13 | 10/10/2023 | Multi-arm Bandits, UCB, Thompson Sampling Part I | ||
| Lecture 14 | 10/12/2023 | Multi-arm Bandits, UCB, Thompson Sampling Part II | ||
| Holiday | 10/17/2023 | No Class - Fall Break | ||
| Holiday | 10/19/2023 | No Class - Fall Break | ||
| Lecture 15 | 10/24/2023 | Advanced RL: DDPG, Soft Q Learning, Max Ent RL and their Equivalence | HW 3 Due, HW 4 Released | |
| Lecture 16 | 10/26/2023 | Model-based learning and planning. CEM, MPPI, SVG, PAL/MAL | ||
| Lecture 17 | 10/31/2023 | Model-based Planning: linear models (LQR, iLQR) extensions to deep nets. | ||
| Lecture 18 | 11/02/2023 | Deep model-based learning: dreamer etc. | Dreamer | |
| Holiday | 11/07/2023 | No Class - Democracy Day | ||
| Lecture 19 | 11/09/2023 | Basics of Visual Learning, 3D Vision, Self-supervised Robot Learning and exploration | Curiosity, RND | Project Proposal Due |
| Lecture 20 | 11/14/2023 | Exploration Part II | ||
| Lecture 21 | 11/16/2023 | Guest Lecture: Offline RL | CQL, AWAC, IQL | HW 4 Due |
| Lecture 22 | 11/21/2023 | RL for Robotics -- Sim2Real ++ | ||
| Holiday | 11/23/2023 | No Class - Thanksgiving | ||
| Lecture 23 | 11/28/2023 | Inv RL, Learning from offline data. | LP-IRL, MaxEnt IRL | |
| Lecture 24 | 11/30/2023 | Advanced: Diffusion Models, Transformers, combining LLMs for Robot Learning | ||
| Lecture 25 | 12/05/2023 | Student Project Presentations | Project Presentation Due | |
| Lecture 26 | 12/07/2023 | Student Project Presentations | ||
| Project Report Due | 12/16/2023 | Project Report Due |