All the lecture notes are linked to this web page, in PDF format. The reader for PDF files is available free from Adobe for UNIX, Apple Macintosh, and Windows.
Introduction | Slides | ||
Conditional independence | Slides | ||
Belief networks. Bayes ball algorithm. | Slides, MJ, Sec. 2.1 | ||
Independence maps. Factorization | Slides, MJ, Sec. 4.2 | ||
d-separation. Belief nets in practice | Slides, MJ, Sec. 4.2, optionally 4.4 | Homework 1 posted | |
Markov random fields | Slides, MJ, Sec. 2.2, Sec. 4.3, optionally 4.5 | ||
Variable elimination algorithm | Slides, MJ, Chapter 3 | ||
Sum-product algorithm | Slides, MJ, Sec. 4.1, 4.3, 4.4 | ||
Junction tree algorithm | Slides, MJ, Chapter 17 | Homework 2 posted | |
Approximate inference: Forward sampling in belief networks | Slides, Russell and Norvig, Section 14.5 | ||
Approximate inference: Markov Chain Monte Carlo (MCMC) | Slides, Russell and Norvig, Section 14.5 | ||
Learning in graphical models: Parameter estimation | MJ, Sec.9.1, 9.2, Slides. Optional: RN, Sec. 20.2 | ||
Learning Bayes nets (part 2) | Slides. Optional: Tutorial by Heckerman | Homework 4 posted | |
Learning Bayes net structure: Scoring functions | Slides | ||
Parameter learning with missing values. Introduction to expectation maximization (EM) | MJ, Sec.9.4, Slides. Optional: RN, Sec. 20.3 | ||
EM for mixture models | MJ, Sec. 10.1, MJ, Sec. 11.1, 11.2, Slides | ||
Learning structure with missing data | Slides. Optional:Paper on structural EM by Nir Friedman | ||
Spring break | |||
Introduction to decision making | RN, Chapter 16 (review) | ||
No class | |||
Decision graphs | MJ, Chapter 29, Sec. 1.1 | ||
No class | |||
Inference in decision graphs | MJ, Chapter 29, Sec.1.2 except 1.2.6 | ||
Temporal models: Hidden Markov Models (HMM), Kalman filters, Dynamic Bayes nets (DBN) | Note that March 11 was a make-up lecture for March 1, 8 RN, Chapter 15, MJ, Chapter 12. Optional: MJ Chapter 18, |
||
Introduction to Markov Decision Processes (MDPs) | Sutton & Barto, Chapters 1 and 3 | ||
Dynamic programming in MDPs | Sutton & Barto, Chapter 4 | Homework 5 posted | |
Dynamic programming (2). Introduction to learning | |||
Learning in MDPs: Monte Carlo, TD | Sutton & Barto,Sec. 5.1, 5.2, 6.1, 6.2, 6.3 | ||
Learning control: Sarsa, Q-learning, exploration | |||
Actor-critic methods | Programming homework Skeleton code | ||
Homework 6 |