General Information

Course Description

The course will cover selected topics and new developments in Data mining and Machine learning, with a particular emphasis on good methods and practices for effective deployment of real systems. We will study commonly used algorithms and techniques, including clustering, neural networks, support vector machines, decision trees. We will also discuss methods to address practical issues such as feature selection and dimensionality reduction, error estimation and empirical validation, algorithm design and parallelization, and handling of large datasets.

Course content (subject to minor changes):

• Linear regression. Linear classification.
• Performance evaluation, overfitting, cross-validation, bias-variance analysis, error estimation.
• Naive Bayes.
• Decision trees. Regression trees and ensemble methods.
• Cost-sensitive learning.
• Support vector machines.
• Artificial neural networks. Deep learning.
• Feature selection. Dimensionality reduction. Regularization.
• Unsupervised learning and clustering.
• Applications.

Reference Materials

• Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press. 2014. Available free online.
• Trevor Hastie, Robert Tibshirani and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. Springer. 2009. Available free online.
• Christopher Bishop. Pattern Recognition and Machine Learning. Springer. 2007.
• Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. The MIT Press. 2016. free online
• Kevin Murphy. Machine Learning: A Probabilistic Perspective. The MIT Press. 2012.
• David MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press. 2003.
• Richard Duda, Peter Hard and David Stork. Pattern Classification. 2nd Edition. Wiley & Sons. 2001.
• Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning . The MIT Press. 2006. free online
• Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning . The MIT Press. 2016. free online

Prerequisites / Anterequesites

Students who took COMP-652 in Winter 2013 or before CANNOT take COMP-551. Starting in Fall 2013, COMP-551 and COMP-652 were designed to avoid significant overlap; you can take either or both.

The courses is intended for hard-working, technically skilled, highly motivated students. Participants will be expected to display initiative, creativity, scientific rigour, critical thinking, and good communication skills.

Evaluation Criteria

• One written midterm examination - 35%
• Five projects (data analysis case studies + peer reviews) - 65%
  Three mini-projects, one project based on a Kaggle competition, one final project (reproducibility study).

The midterm is designed to assess in-depth understanding of fundamental methods and algorithms. It will be scheduled towards the later end of the semester (early April). There is no final exam.

The projects will require reading, writing, programming and experiments to gain hands-on experience with the application of recent machine learning methods, including concepts covered in the lectures, and concepts drawn from the literature. Students will be responsible for characterizing the problem, developing methods of analysis, and presenting the results of their work. Some projects may be individual, most will be done in groups (usually of 3 students).

We will use a peer-review system to evaluate some of the data analysis case studies. Each student will be asked to read and evaluate submissions of their colleagues. The emphasis will be placed on providing constructive feedback on the methodology and presentation.

Evaluation Policy

No make-up quizzes or midterm will be given.

Some of the course work will be individual, other components can be completed in groups. It is the responsibility of each student to understand the policy for each work, and ask questions of the instructor if this is not clear. It is also the responsibility of each student to carefully acknowledge all sources (papers, code, books, websites, individual communications) using appropriate referencing style when submitting work.

We will use automated systems to detect possible cases of text or software plagiarism. Cases that warrant further investigation will be referred to the university disciplinary officers. Students who have concerns about how to properly use and acknowledge third-party software should consult the course instructor or TAs.

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/students/srr/honest/ ) for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.