Reinforcement Learning (COMP-762)
Winter 2017


General Information

Location:McConnell Engineering room 103
Times:Friday, 10:05am-12:55pm
Instructor:Doina Precup and Pierre-Luc Bacon, School of Computer Science
Office:McConnell Engineering building, room 111N and 107
Phone:(514) 398-6443 (Doina) and
Office hours:
TBA (see course home page)
Meetings at other times by appointment only!
Class web page:
IMPORTANT: This is where class notes, announcements and homeworks are posted!

Course Description

The goal of this class is to provide an introduction to reinforcement learning, a very active part of machine learning. Reinforcement learning is concerned with building programs which learn how to predict and act in a stochastic environment, based on past experience. Applications of reinforcement learning range from classical control problems, such as powerplant optimization or dynamical system control, to game playing, inventory control, and many other fields. Notably, reinforcement learning has also produced very compelling models of animal and human learnign. During this course, we will study theoretical properties and practical applications of reinforcement leanring. We will follwo the second edition of the classic textbook by Sutton & Barto (available online), and supplement it as needed with papers and other materials.


Basic knowledge of a programming language is required. Knowledge of probabilities/statistics, calculus and linear algebra is required. Example courses at McGill providing sufficient background in probability are MATH-323 or ECSE-305. Machine learning background, as provided for example by COMP-551 or COMP-652 is required. If you have doubts regarding your background, please contact Doina to discuss it.

Reference Materials

Required textbook: Additional textbooks: Lecture notes and other relevant materials are linked to the lectures web page.

MyCourses will be used only for bulletin board, discussion groups and assignment submission and grading.

Class Requirements

The class grade will be based on the following components:

  1. Five assignments - 60%. Each week, part of the course will be devoted to student presentations and demonstrations. Each week, on Friday evening, a set of papers and possible experimental demonstration topics will be posted. Students can choose a topic from this list and be prepared to do a short presentation in the following week on this topic. Every student MUST subscribe to doing this 5 times during the term. Students can work individually or in teams of 2. Due to time constraints, not all students or teams who volunteered in a given week may be picked to present during that week, but they have to post their material on a shared site. This material will be graded.
  2. A final project - 40%. For the final project, students can work individually or in groups of up to 3 students, on a topic of their choice. Students must send by email to the course instructors a brief description of their topic by March 10. Project presntation will be scheduled outside of class hours, and project reports wil be due after the presentations.
  3. Participation in class discussions - up to 1% extra credit.
Minor changes to the evaluation scheme (if any) will be announced in class by Friday, January 13 (pending in-class discussion and the estimated total enrollment).

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures (see for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.