University of California at
Berkeley
Dept. of Electrical Engineering
& Computer Science
Dept. of Statistics
EECS 281a / STAT 241a
Statistical Learning Theory  Graphical Models
Fall Semester 2012
Practical information
Lectures:
Tues/Thurs 14:0015:30, LeConte Hall 2.
Recitations (optional): Wednesday 09:0010:30, 306 Soda Hall.
Course reader: An Introduction to Probabilistic Graphical Models, by M. Jordan. Available at
Copy Central, 44 Shattuck Square, starting 8/28.
Grading: Homework (60%) and Course Projects (40%), OR
Homework (60%), Course Project (20%) and Exam (20%)
Instructors:
Martin Wainwright
 Office Hours: Tues, Thurs 3:304:30, 263 Cory

Email: wainwrig AT eecs DOT berkeley
DOT edu
 Phone: 6431978
 Office: 263 Cory
Hall
Graduate student instructors:
 Andre Wibisono
 Office Hours: Monday 45 pm, 411 Soda Hall
 Email: wibisono AT eecs DOT berkeley DOT edu
 Hongwei Li
 Office Hours: Friday 23 pm, 307 Evans Hall
 Email: hwli AT stat DOT berkeley DOT edu
Back to top
Course description: This course is a 3unit course that
provides an introduction to the area of probabilistic models based on
graphs. These graphical models provide a very flexible and powerful
framework for capturing statistical dependencies in complex,
multivariate data. Key issues to be addressed include representation,
efficient algorithms, inference, and statistical estimation. These
concepts will be illustrated using examples drawn from various
application domains, including machine learning, signal processing,
communication theory, computational biology, computer vision, etc.
Outline:
 Basics on graphical models, Markov properties, recursive decomposability,
elimination algorithms
 Sumproduct algorithm, factor graphs, semirings
 Markov properties of graphical models
 Junction tree algorithm
 Chains, trees, factorial models, coupled models, layered models
 Kalman filtering and RauchTungStriebel smoothing
 Hidden Markov models (HMM) and forwardbackward
 Exponential family, sufficiency, conjugacy
 Frequentist and Bayesian methods
 The EM algorithm
 Conditional mixture models, hierarchical mixture models
 Factor analysis, principal component analysis (PCA), canonical correlation analysis (CCA), independent component analysis (ICA)
 Importance sampling, Gibbs sampling, MetropolisHastings
 Variational algorithms: mean field, belief propagation, convex relaxations
 Dynamical graphical models
 Model selection, marginal likelihood, AIC, BIC, and MDL
 Applications to signal processing, bioinformatics,
communication, computer vision, etc.
Required background:
The prerequisites are previous coursework in linear algebra, multivariate calculus, basic probability and
statistics (at the level of EE 126). Some degree of mathematical maturity is also required. Coursework
or background in graph theory, information theory, optimization theory, and statistical physics
is relevant, and could be helpful but is not required. Familiarity with a matrixoriented programming
language (e.g., MATLAB, R, Splus, etc.) will be necessary.
Homework:
Although it is acceptable for students to discuss the homework
assignments with one another, each student must write up his/her
homework on an individual basis. Each student must indicate with whom
(if anyone) they discussed the homework problems. Homeworks must be
turned in at the beginning of class on the due date. Late homeworks
will not be accepted. We will not accept electronic submissions.
Course project:
The course project will involve independent work on a topic of the student's own
choosing. Course projects will be presented in an informal poster session at the end of semester, and
the work will be summarized in a writeup.
Back to top
Updates and Announcements
First class will be held on Tuesday, August 28.
Back to top
Handouts
Wed, Dec 12: Homework #7 Solutions .
Sat, Dec 8: Homework #6 Solutions .
Tues, Nov 20: Wainwright and Jordan paper on variational methods.
Chapter 5 is on mean field; see in particular Example 5.2 (p. 134) for
details on naive mean field on Ising model.
Sun, Nov 18: Posted some reading material about neighborhoodbased
graph selection
Logistic regression and Ising models: Paper
Tues, Nov 13: Homework #7 due Thurs
Nov 29.
Mon, Nov 5: Homework #5 Solutions .
Thurs, Nov 1: Homework #4 Solutions .
Tue, Oct 30: Homework #6, due Tuesday,
November 13. Data files: Y.dat,
Lambda.dat,
Ymodel.dat, Xmodel.dat,
Ynew.dat, Xnew.dat.
Thu, Oct 25: Homework #3 solution.
Wed, Oct 17: Information sheet
on course projects. Poster presentations will be given on Monday, December 10
from 35pm in the Wozniak Lounge, Soda Hall.
Tue, Oct 16: Homework #5, due Tuesday, October 30. Data files: hmmgauss.dat, hmmtest.dat, Pairwise.dat.
Thu, Oct 4: Homework #2 solution.
Tue, Oct 2: Homework #4, due Tuesday, October 16.
Tue, Sep 18: Homework #3, due Tuesday, October 2.
Thu, Sep 13: Homework #1 solution.
Tue, Sep 4: Homework #2, due on Thursday, September 13. Here are the auxiliary files: lms.dat, classification2d.dat, and testing.dat. Please note that you must turn in a paper copy of your homework in class. Also, as per course policy, it is not possible to consider
late homeworks. Corrections: In 2.5(a), "binomial entropy" should be "Bernoulli entropy". In 2.6(a), \phi_1 should be T_1.
Fri, Aug 31: Chapter 6 and chapter 8 from the reader.
Wed, Aug 29: Slides from the first recitation (review on probability, statistics, and linear algebra).
Tues, Aug 28: Homework #1, due on Tuesday, September 4.
Homework #1 is purely on undergraduate review material; if you are
not familiar with it, then you do not have the appropriate background
for this course, and will likely not benefit from taking it.
Tues, Aug 28: Syllabus
Back to top
Supplementary reading
Back to top