Location: Office is in the AMPLab, fourth floor of Soda Hall.
Teaching Assistant:
Di Wang
Email: wangd ATSYMBOL eecs.berkeley.edu
Office hours: By appointment.
Class time and Location:
Tue-Thu 9:30-11:00AM, in 320 Soda (First meeting is Thu Jan 22, 2015.)
Notes:
(5/31) My scribed version of the class lectures are available below. Feedback welcome!
(1/29) I'll be posting notes on Piazza, not here.
(1/15) All students, including auditors, are requested to register for the
class. Auditors should register S/U; an S grade will be awarded for class
participation and satisfactory scribe notes.
Course description:
Spectral graph methods use eigenvalues and eigenvectors of matrices
associated with a graph, e.g., adjacency matrices or Laplacian matrices,
in order to understand the properties of the graph. They have a rich
algorithmic and statistical theory, including connections with random
walks, inference, and expanders; and they are useful in applications
ranging from parallel computing to computer vision to social network
analysis. The course will cover advanced topics in the underlying
algorithmic and statistical theory, with a bias toward theoretical
aspects of methods that are practically useful in modern machine
learning and data analysis.
Topics to include a subset of:
underlying theory, including Cheeger's inequality and its connections with partitioning, isoperimetry, and expansion;
algorithmic and statistical consequences, including explicit and implicit regularization and connections with other graph partitioning methods;
applications to semi-supervised and graph-based machine learning;
applications to clustering and related community detection methods in statistical network analysis;
local and locally-biased spectral methods and personalized spectral ranking methods;
applications to graph sparsification and fast solving linear systems; etc.
Appropriate for advanced graduate students in statistics, computer
science, and mathematics, as well as computationally-inclined students
from application domains.
Prerequisites:
General mathematical sophistication; and a
solid understanding of Algorithms, Linear Algebra, and Probability Theory,
at the advanced undergraduate or beginning graduate level, or equivalent.
Course requirements:
Most likely,
three homeworks (ca. 15-20% each),
scribe two lectures (ca. 10%),
and a major project (ca. 40%).
Primary references:
We will be reading reviews and primary sources.
Here are several things to get started.
Additional articles for particular topics and particular classes are listed
below.
Introduction, background, and overview:
Belkin and Niyogii,
"Laplacian Eigenmaps for Dimensionality Reduction and Data Representation"
Doyle and Snell,
"Random Walks and Electric Networks"
Gleich,
"PageRank beyond the Web"
Hoory, Linial, and Wigderson,
"Expander graphs and their applications"
Jeub, Balachandran, Porter, Mucha, and Mahoney,
"Think Locally, Act Locally: The Detection of Small, Medium-Sized, and Large Communities in Large Networks"
Schaeffer,
"Graph clustering"
von Luxburg,
"A Tutorial on Spectral Clustering"
Individual Lectures:
Here are readings for each class as well as lecture notes.
January 22, 2015: Introduction and Overview
Readings:
Take a quick look at those listed in "Introduction, background, and overview" above
Scribed lectures: pdf.
January 27, 2015: Basic Matrix Results (1 of 3)
Readings:
Lecture notes from Spielman's Spectral Graph Theory class, Fall 2009 and 2012
Scribed lectures: pdf.
January 29, 2015: Basic Matrix Results (2 of 3)
Readings:
Same as last class.
Scribed lectures: pdf.
February 03, 2015: Basic Matrix Results (3 of 3)
Readings:
Same as last class.
Scribed lectures: pdf.
February 05, 2015: Overview of Graph Partitioning
Readings:
"Survey: Graph clustering," in Computer Science Review, by Schaeffer
"Geometry, Flows, and Graph-Partitioning Algorithms," in CACM, by Arora, Rao, and Vazirani
Scribed lectures: pdf.
February 10, 2015: Spectral Methods for Partitioning Graphs (1 of 2)
Readings:
"Lecture Notes on Expansion, Sparsest Cut, and Spectral Graph Theory," by Trevisan
Scribed lectures: pdf.
February 12, 2015: Spectral Methods for Partitioning Graphs (2 of 2)
Readings:
Same as last class.
Scribed lectures: pdf.
February 17, 2015: Expanders, in theory and in practice (1 of 2)
Readings:
"Expander graphs and their applications," in Bull. Amer. Math. Soc., by Hoory, Linial, and Wigderson
Scribed lectures: pdf.
February 19, 2015: Expanders, in theory and in practice (2 of 2)
Readings:
Same as last class.
Scribed lectures: pdf.
February 24, 2015: Flow-based Methods for Partitioning Graphs (1 of 2)
Readings:
"Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms, in JACM, by Leighton and Rao
"Efficient Maximum Flow Algorithms," in CACM, by Goldberg and Tarjan
Scribed lectures: pdf.
February 26, 2015: Flow-based Methods for Partitioning Graphs (2 of 2)
Readings:
Same as last class.
Scribed lectures: pdf.
March 03, 2015: Some Practical Considerations (1 of 4)
Readings:
"A Tutorial on Spectral Clustering," in Statistics and Computing, by von Luxburg
Scribed lectures: pdf.
March 05, 2015: Some Practical Considerations (2 of 4)
Readings:
"A kernel view of the dimensionality reduction of manifolds," in ICML, by Ham, et al.
Scribed lectures: pdf.
March 10, 2015: Some Practical Considerations (3 of 4)
Readings:
"Laplacian Eigenmaps for dimensionality reduction and data representation," in Neural Computation, by Belkin and Niyogi
"Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization, in IEEE-PAMI, by Lafon and Lee
Scribed lectures: pdf.
March 12, 2015: Some Practical Considerations (4 of 4)
Readings:
"Transductive learning via spectral graph partitioning," in ICML, by Joachims
"Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions," in ICML, by Zhu, Ghahramani, and Lafferty
"Learning with local and global consistency," in NIPS, by Zhou et al.
Scribed lectures: pdf.
March 17, 2015: Modeling Graphs with Electrical Networks
Readings:
"Random Walks and Electric Networks," in arXiv, by Doyle and Snell
Scribed lectures: pdf.
March 19, 2015: Diffusions and Random Walks as Robust Eigenvectors
Readings:
"Implementing regularization implicitly via approximate eigenvector computation," in ICML, by Mahoney and Orecchia
"Regularized Laplacian Estimation and Fast Eigenvector Approximation," in NIPS, by Perry and Mahoney
Scribed lectures: pdf.
March 31, 2015: Local Spectral Methods (1 of 4)
Readings:
"Spectral Ranking", in arXiv, by Vigna
"PageRank beyond the Web," in arXiv, by Gleich
Scribed lectures: pdf.
April 02, 2015: Local Spectral Methods (2 of 4)
Readings:
"The Push Algorithm for Spectral Ranking", in arXiv, by Boldi and Vigna
"Local Graph Partitioning using PageRank Vectors" in FOCS, by Andersen, Chung, and Lang
Scribed lectures: pdf.
April 07, 2015: Local Spectral Methods (3 of 4)
Readings:
"A Local Spectral Method for Graphs: with Applications to Improving Graph Partitions and Exploring Data Graphs Locally," in JMLR, by Mahoney, Orecchia, and Vishnoi
Scribed lectures: pdf.
April 09, 2015: Local Spectral Methods (4 of 4)
Readings:
"Anti-differentiating Approximation Algorithms: A case study with Min-cuts, Spectral, and Flow," in ICML, by Gleich and Mahoney
Scribed lectures: pdf.
April 14, 2015: Some Statistical Inference Issues (1 of 3)
Readings:
"Towards a theoretical foundation for Laplacian-based manifold methods," in JCSS, by Belkin and Niyogi
Scribed lectures: pdf.
April 16, 2015: Some Statistical Inference Issues (2 of 3)
Readings:
"Consistency of spectral clustering, in Annals of Statistics, by von Luxburg, Belkin, and Bousquet
Scribed lectures: pdf.
April 21, 2015: Some Statistical Inference Issues (3 of 3)
Readings:
"Spectral clustering and the high-dimensional stochastic blockmodel," in The Annals of Statistics, by Rohe, Chatterjee, and Yu
"Regularized Spectral Clustering under the Degree-Corrected Stochastic Blockmodel," in NIPS, by Qin and Rohe
Scribed lectures: pdf.
April 23, 2015: Laplacian solvers (1 of 2)
Readings:
"Effective Resistances, Statistical Leverage, and Applications to Linear Equation Solving," in arXiv, by Drineas and Mahoney
"A fast solver for a class of linear systems," in CACM, by Koutis, Miller, and Peng
"Spectral Sparsification of Graphs: Theory and Algorithms," in CACM, by Batson, Spielman, Srivastava, and Teng
Scribed lectures: pdf.
April 28, 2015: Laplacian solvers (2 of 2)
Readings:
Same as last class.
Scribed lectures: pdf.