12/1. Suggested review exercises posted on "homework" page.
Revised office hours for RRR week -- see bottom of page.
STAT 150: Stochastic Processes (Fall 2015)
This is a second course in Probability, studying the mathematically basic kinds of random process,
intended for majors in Statistics and related quantitative fields.
The prerequisite is STAT 134 or similar upper-division course.
If you did not get at least a B+ in that course then you will find this course very tough.
Instructor: David Aldous
Class Time: MWF 10.00 - 11.00 in room 60 Evans.
GSI: Sourav Sarkar
Discussion sections: Tu 4.00-5.00 or 5.00-6.00 in room 334 Evans.
Office Hours: Instructor: Fridays 11.30 - 1.30 in 351 Evans.
GSI: Thursdays 2.00-4.00, Fridays 1.00-3.00 in 446 Evans.
An Introduction to Stochastic Modeling, Fourth Edition
by Mark Pinsky and Samuel Karlin (Academic Press).
Basic Stochastic Processes by
Zdzislaw Brzezniak and Tomasz Zastawniak (Springer).
PK is a traditional textbook for this level course.
BZ is a rather more sophisticated but concise account.
We will not cover all the material in these boks -- see the "outline of topics" below for the
topics we will cover.
many other books covering these topics,
Stochastic Processes (Ross)
Introduction to Stochastic Processes (Cinlar) or
Essentials of Stochastic Processes (Durrett).
There are also
online lecture notes by Jim Pitman
in a more concise and mathematical style than my own lectures.
Weekly homeworks will be posted here, due in class each Wednesday
(first homework due Wednesday 9 September).
Note that the PK text has both
Exercises (with solutions at end) and Problems: the homeworks are the "Problems".
And a good way to study for exams is to try some of the "Exercises".
Midterms: 2 in class midterms.
Monday 14 December, 8.00 - 11.00am
20% homework, 15% each midterm, 50% final.
Outline of topics
- (Week 1): [PK Chapter 1]: Introduction
- (Week 2): [PK Chapter 2]: Conditional Probability and Conditional Expectation.
- (Weeks 3-5): [PK Chapter 3; BZ Chapter 5]: Markov Chains: Introduction.
- (Weeks 5-6): [PK Chapter 4; BZ Chapter 5]: Long-Run Behavior of Markov Chains.
- (Weeks 7-8): [PK Chapter 5; BZ Chapter 6.2]: Poisson Processes.
- (Weeks 8-9): [PK Chapter 6]: Continuous-Time Markov Chains.
- (Week 10): [PK Chapter 2.5; BZ Chapters 3,4]: Martingales.
- (Weeks 11-13): [PK Chapter 8; BZ Chapter 6.3]: Brownian Motion.
You can bring to the midterm and final exam anything in your own handwriting but no
other written material. So you can bring your notes from class, your homework, summaries
of the course material you have made - provided these are literally in
your own handwriting without electronic intermediation.
Do bring blue book or paper to write on; and a basic calculator.
If you email us (email@example.com, firstname.lastname@example.org) please put "STAT 150" in subject.
Topic each class
Slides for the first few lectures will be posted here.
W 8/26: Review of STAT 134 material, via examples.
F 8/28: Review of STAT 134 material, via examples.
M 8/31: Review of STAT 134 material, via examples.
W 9/2: Conditional expectation as a RV; examples;
stochastic process; conditioning on first step.
- Read [PK] Chapter 2. Except for the "martingale" material in sec. 2.5, which I will
cover in more detail later.
F 9/4: Markov chains: definition and basic examples.
M 9/7: [holiday]
- Start to read [PK] Chapter 3.
W 9/9 : Markov chain theory:
t-step transition matrix and and first-step analysis.
F 9/11 : First-step analysis for random walk;
analysis for success runs;
death and immigration chain.
M 9/14 : Communicating classes;
definition of stationary distribution; special cases (doubly stochastic,
success runs, RW on weighted undirected graph).
- Read [PK] Chapter 3, sections 3.1 - 3.7 (we will cover 3.8 - 3.9 later)
W 9/16 : Limit theory I.
F 9/18 : Limit theory II.
Read [PK] Chapter 4. But I do not emphasize the examples in
section 4.2 or the matrix calculations in section 4.5.
M 9/21 : Birth-and-death chains. The stationary chain.
W 9/23 : Branching processes and generating functions I.
F 9/25 : Branching processes and generating functions II.
M 9/28 : Cash management model [PK] sec. 3.6.2
W 9/30 : Some proofs: recurrence/transience; class properties; finite irreducible is recurrent;
``occupation measure" formula for stationary distribution. RW on n-cycle.
F 10/2 [midterm 1]
M 10/5 : Poisson processes I.
W 10/7 : Poisson processes II.
F 10/9 : Poisson processes III.
M 10/12 : Poisson processes IV.
The ``approximation" examples are from this lecture on coincidences.
W 10/14 : Continuous-time Markov chains I.
F 10/16 : Continuous-time Markov chains II.
M 10/19 : Continuous-time Markov chains III.
W 10/21 : Renewal theory.
- Read [PK] Chapter 7 sections 1-4.
F 10/23 : Outline abstract background to martingale theory.
M 10/26 : Martingales I.
W 10/28 : Martingales II.
F 10/30 : Martingales III.
More about the real-world examples in this paper.
Links used in lecture.
Ladbrokes: 2016 Republican Presidential Nominee
PredictIt: Will Ted Cruz win the 2016 Republican presidential nomination?
RealClearPolitics Poll Average: 2016 Republican Presidential Nomination
M 11/2 : Martingales IV.
W 11/4 [midterm 2]
Lab this week on Monday November 2; time 5.00 - 6.00 only, in the usual room 334 Evans.
F 11/6 : Brownian motion I. Definition, scaling limit of RW, structural properties.
M 11/9 : Brownian motion II. Reflection principle, formulas arising from it.
- Read [PK] Chapter 8 sections 1-2.
W 11/11 [holiday]
F 11/13 : Brownian motion III. Formulas involving zeros of BM.
M 11/16 : Brownian motion IV. Gaussian processes. Brownian bridge and meander.
W 11/18 : Brownian motion V. Brownian motion with drift, geometric BM.
- Read [PK] Chapter 8 sections 3-4.
F 11/20 : Queueing theory I. General setup, M/M/s model.
M 11/23 : Queueing theory II. The M/G/1 model.
- Read [PK] Chapter 9 sections 1-3.
W 11/25 [holiday]
F 11/27 [holiday]
M 11/30 : Queueing theory III. M/G/1 model; M/G/infinity; spatial M/M/infinity .
W 12/2 : "Not on the Exam": Waves in a spatial queue and card-shuffling models.
F 12/4 : "Not on the Exam": One-dimensional diffusions.
M 12/7 [review] [in class] Martingale in birthday problem; circular bus route; examples 7, 23, 27 from
Markov chains -- exercises and solutions
- Office hours changed for RRR week -- see below.
Tu 12/8 : GSI office hours 2.00-4.00 in 446 Evans. No Lab -- moved to Friday.
W 12/9 [open questions] [in class]
Th 12/10 : GSI office hours 2.00-4.00 in 446 Evans.
F 12/11 [no class] Instructor office hours 10.00-11.00 in 351 Evans.
Lab/review 3.00pm - 5.00pm in room 166 Barrows.
M 12/14 FINAL EXAM 8.00-11.00am in 60 Evans.