Stat 210B

Spring 2013

Theoretical Statistics


Tue, May 7:
Homework 5 solutions have been posted here.


Office hours
Instructor Peter Bartlett bartlett@stat Evans 399, Tue 1-2pm, Wed 1:30-2:30.
Siqi Wu
siqi@stat Evans 307, Mon 3:30-4:30pm, Tue 3:30-4:30pm.

Lectures:  332 Evans. Tue/Thu 11:00 - 12:30.


The course will cover a range of advanced topics in theoretical statistics, including:
Stochastic convergence. Projections. U-statistics.
Concentration inequalities. Uniform laws. Empirical process theory.
Delta method. Functional delta method.
Contiguity. Local asymptotic normality.
Efficiency of estimators. Efficiency of tests.
Nonparametric regression. Nonparametric density estimation.
M-estimators. Bootstrap estimators.


Asymptotic Statistics. Aad van der Vaart. Cambridge. 1998.
Convergence of Stochastic Processes. David Pollard. Springer. 1984. Available on-line at


Stat 210A and either Stat 205A or Stat 204.


Homework Assignments (60%): posted on the website.
Final Exam (40%): scheduled for Thursday, 5/16/13, 8-11am, in Evans 334.

Homework Assignments:

Homework 1: pdf
Thursday, February 7
Homework 2: pdf
Thursday, February 21
Homework 3: pdf
Thursday, March 7
Homework 4: pdf
Tuesday, April 9
Homework 5: pdf
Thursday, May 2


Tue, Jan 22Organizational issues. Course outline. Stochastic convergence.
1notes.pdf. vdV 1, 2
Thu, Jan 24Stochastic convergence. Concentration inequalities.
2notes.pdf. vdV 2.
Metric Characterization of Random Variables and Random Processes. Buldygin and Kozachenko. 2000.
Concentration inequalities. Boucheron, Lugosi, Bousquet. 2004.
Tue, Jan 29Concentration inequalities.
Thu, Jan 31Concentration inequalities.
Tue, Feb 5U-statistics
5notes.pdf. vdV 12
Thu, Feb 7U-statistics. Projections.
6notes.pdf. vdV 11, 12
A class of statistics with asymptotically normal distribution. Hoeffding. Ann. Math Statist. 19(3), 1948: 293-325.
Tue, Feb 12U-statistics. Projections.
7notes.pdf. vdV 11, 12
Thu, Feb 14Uniform laws of large numbers.
8notes.pdf. PII (esp. II.3).
Tue, Feb 19 Uniform laws of large numbers.
9notes.pdf. PII.
Thu, Feb 21 Uniform laws of large numbers.
10notes.pdf. PII
Notes on Rademacher averages:
A few notes on statistical learning theory. Mendelson.
Rademacher and Gaussian complexities: risk bounds and structural results. Bartlett and Mendelson.
Tue, Feb 26 Uniform laws of large numbers. Metric entropy.
11notes.pdf. PII.
Thu, Feb 28 Uniform laws of large numbers. Metric entropy.
12notes.pdf. PII.
Tue, Mar 5 Uniform laws of large numbers. Metric entropy.
13notes.pdf. PII.
Thu, Mar 7 Uniform laws of large numbers. Metric entropy.
14notes.pdf. PII.
Tue, Mar 12 M-estimators. Consistency of M-estimators.
15notes.pdf. vdV5.
Thu, Mar 14 M-estimators. Consistency of M-estimators. Delta method.
16notes.pdf. vdV3, vdV5.
Tue, Mar 19 Asymptotic equicontinuity.
17notes.pdf. vdV5, vdV18, vdV19, PVII.
Thu, Mar 21 Donsker classes.
18notes.pdf. vdV18, vdV19, PVII.
Tue, Mar 26 Spring
Thu, Mar 28Break
Tue, Apr 2 Functional delta method.
19notes.pdf. vdV20, vdV21.
Tue, Apr 9 Functional delta method. Quantile estimates.
20notes.pdf. vdV20, vdV21.
Thu, Apr 11 Contiguity. Le Cam's lemmas.
21notes.pdf. vdV6.
Tue, Apr 16 Local asymptotic normality.
22notes.pdf. vdV7.
Thu, Apr 18 Local asymptotic normality. Relative efficiency of tests.
23notes.pdf. vdV7. vdV14.
Tue, Apr 23 Relative efficiency of tests.
24notes.pdf. vdV14.
Thu, Apr 25 Relative efficiency of tests. Likelihood ratio tests.
25notes.pdf. vdV14, vdV15.
Tue, Apr 30 Likelihood ratio tests.
26notes.pdf. vdV15.
Thu, May 2 Nonparametric regression. Bootstrap estimators.
27notes.pdf. Some applications of concentration inequalities to statistics. Massart. (see Section 4.2)
Local Rademacher complexities. Bartlett, Bousquet, Mendelson.
Local Rademacher complexities and oracle inequalities in risk minimization. Koltchinskii