Syllabus for COMP-451 Topics in Computer Science: Fundamentals of Machine Learning (Winter 2021)

General Information

Course Description

The course will introduce the core concepts of machine learning, with an emphasis on the computational, statistical and mathematical foundations of the field. We will study models for both supervised learning and unsupervised learning, introducing these models alongside foundational machine learning concepts, such as maximum likelihood estimation, regularization, information theory, and gradient-based optimization. The course concludes with a brief introduction to neural networks and deep learning.

Evaluation Criteria

The class grade will be based on the following components:

Course content (subject to minor changes):

  1. Supervised learning
  2. Maximum likelihood
  3. Naive Bayes
  4. Logistic regression
  5. Gradient descent
  6. Linear regression
  7. Regularization
  8. Information theory
  9. Decision trees
  10. Representation learning / PCA
  11. Clustering
  12. Latent variable models
  13. Neural networks
  14. Boltzman machines

Reference Materials

There is no required textbook. Lecture notes will be released and available from the course web page. The following texts can also be very useful:
  1. Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press. 2014. Available free online.
  2. Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. Springer. 2009. Available free online.
  3. Christopher Bishop. Pattern Recognition and Machine Learning. Springer. 2007. Available here with some solutions here.
  4. Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. The MIT Press. 2016. free online
  5. Kevin Murphy. Machine Learning: A Probabilistic Perspective. The MIT Press. 2012.
  6. David MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press. 2003.
  7. Richard Duda, Peter Hard and David Stork. Pattern Classification. 2nd Edition. Wiley & Sons. 2001.

Prerequisites / Anterequesites

Basic knowledge of a programming language is required. Basic knowledge of probabilities/statistics, calculus and linear algebra is required. The course prerequisites are as follows: COMP 251, MATH 222, MATH 223, and MATH 323. Students cannot take both COMP 451 and COMP 551.

The courses is intended for hard-working, technically skilled, highly motivated students. Participants will be expected to display initiative, creativity, scientific rigour, critical thinking, and good communication skills.

Evaluation Policy

All course work should be submitted online (details to be given in class), by 11:59pm, on the assigned due date. Late work will be automatically subject to a 20% penalty, and can be submitted up to 5 days after the deadline.

We strongly encourage students to form study groups. Students may discuss and work on homework problems in groups. However, each student must write down their solutions independently, i.e., each student must understand the solution well enough in order to reconstruct it by him/herself. Students should clearly mention the names of all the other students who were part of their discussion group. Copying code or solutions obtained from the web (GitHub/Google/Chegg) is considered plagiarism. We check all the submissions for plagiarism. Referring to research papers, textbooks, existing code bases, or online notes to aid in finding a solution is acceptable, as long as the final solution is written down independently and the student can reconstruct the solution independently if asked. It is also the responsibility of each student to carefully acknowledge all sources in this case (papers, code, books, websites, individual communications) using appropriate referencing style when submitting work.

We will use automated systems to detect possible cases of text or software plagiarism. Cases that warrant further investigation will be referred to the university disciplinary officers. Students who have concerns about how to properly use and acknowledge third-party software should consult the course instructor or TAs.

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/students/srr/honest/ ) for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.

As the instructor of this course I endeavor to provide an inclusive learning environment. However, if you experience barriers to learning in this course, do not hesitate to discuss them with me and the Office for Students with Disabilities, 514-398-6009.