Lectures for Probabilistic Reasoning in AI (COMP-526)

Winter 2008


All the lecture notes are linked to this web page, in PDF format, with 2 slides per page. The reader for PDF files is available free from Adobe for UNIX, Apple Macintosh, and Windows.

I am happy to provde source files for all these slides upon request for teaching purposes.


Tentative Schedule

Lec.
Date
Topic
Readings
Notes
1
Jan. 4
 Introduction  RN Chapter 13
 Slides: PDF
 
2
Jan. 7
 Conditional independence. Introduction to belief networks  RN, Sec. 14.1, 14.2; MJ, Sec. 2.1
 Slides: PDF
 
3
Jan. 9
 Independence maps. Bayes ball algorithm  
 Slides: PDF
Homework 1 posted
4
Jan. 11
 More on Bayes nets: d-separation, moral graph, practical considerations  
 Slides: PDF
 
5
Jan. 14
 Undirected graphical models  
 Slides: PDF
 
6
Jan. 16
 Exact inference: Variable elimination  
 Slides: PDF
Homework 1 due!
7
Jan. 18
 Exact inference: Message passing in trees. Clique trees  
 Slides: PDF
Homework 2 posted
8
Jan. 21
 Message passing in clique trees  
 Slides:
 
9
Jan. 23
 Junction trees  
 Slides:
 
10
Jan. 25
 Belief propagation in polytrees  
 Slides:
Homework 2 due!
-
Jan. 28
 No class  
 Slides:
11
Jan. 30
 Loopy belief propagation  
 Slides:
 
12
Feb. 1
 Approximate inference: Likelihood weighting  
 Slides:
 
13
Feb. 4
 Importance sampling and particle filters  
 Slides:
 
-
Feb. 6
 No class  
 Slides:
 
-
Feb. 8
 No class  
 Slides:
 
14
Feb. 11
 Approximate inference: Gibbs sampling. MCMC methods  
 Slides:
Homework 3 posted Bayes net toolbox Insurance Bayes net
15
Feb. 13
 Introduction to learning. Maximum likelihood  
 Slides:
 
16
Feb. 15
 Bayesian learning  
 Slides:
 
17
Feb. 18
 Structure learning in Bayes nets with complete data  
 Slides:
Homework 3 due!
18
Feb. 20
 Expectation maximization  
 Slides:
 
19
Feb. 22
 More on expectation maximization. Applications to clustering  
 Slides:
 
-
Mar. 3
 No class  
 Slides:
 
20
Mar. 5
 Exponential family distributions  
 Slides:
Homework 4 posted
21
Mar. 7
 Learning undirected graphical models  
 Slides:
 
22
Mar. 10
 Evaluation and comparison of different algorithms  
 Slides:
 
23
Mar. 12
 Hidden Markov models. Forward-backward inference algorithm  
 Slides:
Homework 4 due!
Homework 5 posted
24
Mar. 14
 Expectation maximization for learning HMMs. Dynamic Bayesian networks (DBN)  
 Slides:
 
25
Mar. 17
 Kalman filter and related models  
 Slides:
 
26
Mar. 19
 Introduction to decision making. Utility theory  
 Slides:
Homework 5 due!
Homework 6 posted
27
Mar. 20
 Markov Decision Processes. Policies and value functions. Policy evaluation  
 Slides:
28
Mar. 26
 Optimal policies and value functions. Policy iteration and value iteration algorithms  
 Slides:
 
29
Mar. 28
 Monte Carlo and temporal difference learning for policy evaluation  
 Slides:
Homework 6 due!
Project started
30
Mar. 31
 On-policy control learning: Sarsa. Exploration-exploitation trade-off  
 Slides:
 
31
Apr. 2
 Off-policy learning: Policy evaluation, Q-learning  
 Slides:
 
32
Apr. 4
 Actor-critic algorithms  
 Slides:
 
33
Apr. 7
 Partially Observable Markov Decision Processes (POMDPs). Planning methods for POMDPs  
 Slides:
 
34
Apr. 9
 More on planning for POMDPs. Learning methods for POMDPs  
 Slides:
 
35
Apr. 11
 Other methods for decision making under uncertainty  
  Slides:
Project due!


Prof. Doina PRECUP
Last modified: Thu Jan 3 17:05:46 EST 2008