Lectures for Probabilistic Reasoning in AI (COMP-526)

Winter 2004


All the lecture notes are linked to this web page, in PDF format. The reader for PDF files is available free from Adobe for UNIX, Apple Macintosh, and Windows.


Schedule

Date
Topic
Readings
Notes
Jan. 5
 Introduction  Slides  
Jan. 7
 Conditional independence  Slides  
Jan. 9
 Belief networks. Bayes ball algorithm.  Slides, MJ, Sec. 2.1  
Jan. 12
 Independence maps. Factorization  Slides, MJ, Sec. 4.2  
Jan. 14
 d-separation. Belief nets in practice  Slides, MJ, Sec. 4.2, optionally 4.4  Homework 1 posted
Jan. 16
 Markov random fields  Slides, MJ, Sec. 2.2, Sec. 4.3, optionally 4.5  
Jan. 19
 Variable elimination algorithm  Slides, MJ, Chapter 3  
Jan. 21
 Sum-product algorithm  Slides, MJ, Sec. 4.1, 4.3, 4.4  
Jan. 23, 26, 30
 Junction tree algorithm  Slides, MJ, Chapter 17  Homework 2 posted
Feb. 2
 Approximate inference: Forward sampling in belief networks  Slides, Russell and Norvig, Section 14.5  
Feb. 4
 Approximate inference: Markov Chain Monte Carlo (MCMC)  Slides, Russell and Norvig, Section 14.5  
Feb. 6
 Learning in graphical models: Parameter estimation  MJ, Sec.9.1, 9.2, Slides. Optional: RN, Sec. 20.2  
Feb. 9
 Learning Bayes nets (part 2)  Slides. Optional: Tutorial by Heckerman  Homework 4 posted
Feb. 11
 Learning Bayes net structure: Scoring functions  Slides  
Feb. 13
 Parameter learning with missing values. Introduction to expectation maximization (EM)  MJ, Sec.9.4, Slides. Optional: RN, Sec. 20.3  
Feb. 16
 EM for mixture models  MJ, Sec. 10.1, MJ, Sec. 11.1, 11.2, Slides  
Feb. 18, 20
 Learning structure with missing data  Slides. Optional:Paper on structural EM by Nir Friedman  
Feb. 23-27
 Spring break    
Mar. 1
 Introduction to decision making    RN, Chapter 16 (review)
Mar. 3
 No class    
Mar. 5
 Decision graphs  MJ, Chapter 29, Sec. 1.1  
Mar. 8
 No class    
Mar. 10
 Inference in decision graphs  MJ, Chapter 29, Sec.1.2 except 1.2.6  
Mar. 11, 12
 Temporal models: Hidden Markov Models (HMM), Kalman filters, Dynamic Bayes nets (DBN)  Note that March 11 was a make-up lecture for March 1, 8
 RN, Chapter 15, MJ, Chapter 12. Optional: MJ Chapter 18,
 
Mar. 15
 Introduction to Markov Decision Processes (MDPs)  Sutton & Barto, Chapters 1 and 3 
Mar. 17
 Dynamic programming in MDPs  Sutton & Barto, Chapter 4 Homework 5 posted
Mar. 19
 Dynamic programming (2). Introduction to learning    
Mar. 22
 Learning in MDPs: Monte Carlo, TD  Sutton & Barto,Sec. 5.1, 5.2, 6.1, 6.2, 6.3  
Mar. 24
 Learning control: Sarsa, Q-learning, exploration    
Mar. 26
 Actor-critic methods   Programming homework Skeleton code
Apr.2
    Homework 6


Prof. Doina PRECUP
Last modified: Fri Apr 2 13:24:39 EST 2004