Course Logistics
Course Description#
Robots need to make sequential decisions to operate in the world and generalize to diverse environments. How can they learn to do so? This is what we call the "robot learning" problem and it spans topics in machine learning, visual learning, and reinforcement learning. In this course, we will learn the fundamentals of topics in machine/deep/visual/reinforcement learning and how such approaches are applied to robot decision-making. We will study fundamentals of 1) machine (deep) learning with an emphasis on approaches relevant to cognition, 2) reinforcement learning: model-based, model-free, on-policy (policy gradients), off-policy (q-learning), etc.; 2) imitation learning: behavior cloning, dagger, inverse RL and offline RL.; 3) visual learning geared towards cognition and decision making including topics like generative models and their use for robotics, learning from human videos, passive internet videos, language models; and 4) leveraging simulations, building differentiable simulations and how to transfer policies from simulation to the real world; 5) we will also briefly touch topics in neuroscience and psychology that provide cognitive motivations for several techniques in decision making. Throughout the course, we will look at many examples of how such methods can be applied to real robotics tasks as well as broader applications of decision-making beyond robotics (such as online dialogue agents, etc.). The course will provide an overview of relevant topics and open questions in the area. There will be a strong emphasis on bridging the gap between many different fields of AI. The goal is for students to get both a high-level understanding of important problems and possible solutions, as well as a low-level understanding of technical solutions. We hope that this course will inspire you to approach problems in cognition and embodied learning from different perspectives in your research.
Course Information#
Class Time and Location#
Time: Tuesday/Thursday 12:30pm-1:50pm
Location: SH 105
Office Hours#
| Office Hours | |
|---|---|
| Deepak Pathak | Available by email appointment |
| Lili Chen | Monday 11:45am-12:30pm, NSH 4513 |
| Tanmay Shankar | Wednesday 4:15-5:00pm, Zoom |
| Eliot Xing | Friday 11:30am-12:15pm, Zoom |
| M. Nomaan Qureshi | Tuesday 10:45am-11:30am, NSH 3104 |
Grading Policy#
| Percentage | |
|---|---|
| HW 1 | 15% |
| HW 2 | 15% |
| HW 3 | 15% |
| HW 4 | 20% |
| Project Presentation | 12.5% |
| Project Report | 22.5% |
Collaboration Policy#
Collaboration is encouraged, but the work you submit for assignments is expected to be entirely your own. That is, the writing and code must be yours, and you must fully understand everything that you submit. Discussing a paper or the details of how to solve a problem is fine, but you must write your submission yourself. Please list collaborators whom you discussed with in the assignment write-up. If we find highly identical work without proper accreditation of collaborators, we will take action according to university policies. For more, see the CMU academic integrity guidelines.
Late Policy#
For the homework assignments only, students will be allowed a total of five late days per semester. Any work submitted late after the five late days have been used will be given an automatic zero on the assignment. Make sure to start early and complete your assignments on time! Please note that the late days do not apply to any part of the final project. This policy will be enforced strictly.