Stat 150: Stochastic Processes (Fall 2023)

Course information

Instructor: Benson Au
Lectures: MWF 10:10a-11:00a (Stanley 106)
Office hours: MF 3:00p-4:00p (Evans 422)

GSI: Adam Quinn Jaffe
Discussion section (optional): Tu 2:10p-3:00p (Evans 342)
Office hours: Tu 1:00p-2:00p (Evans 428), Th 10:00a-12:00p (Evans 428)

Textbooks:

An Introduction to Stochastic Modeling by Pinsky and Karlin (freely available through the university library here)
Essentials of Stochastic Processes by Durrett (freely available through the university library here)

To reiterate, the textbooks are freely available through the university. Note that you must be connected to the university Wi-Fi or VPN to access the ebooks from the library links. Furthermore, the library links take some time to populate, so do not be alarmed if the webpage looks bare for a few seconds.

Exam schedule

Please bring your student ID to the exams.

Midterm #1: Oct 2

Midterm #2: Nov 6

Final: Monday, 11 Dec, 8:00a-11:00a in Latimer 120

Homework assignments

PK refers to Pinksy and Karlin. D refers to Durrett. Note that there are both exercises and problems in PK. Make sure you are doing the correct assignment.

Homework 0, due August 28th at 11:59 PM on Gradescope.

Homework 1 (.tex), due September 1st at 11:59 PM on Gradescope.

Homework 2, (.tex), due September 15th at 11:59 PM on Gradescope.

Homework 3, (.tex), due September 22nd at 11:59 PM on Gradescope.

Homework 4, (.tex), due September 22nd at 11:59 PM on Gradescope.

Homework 5, (.tex), due September 29th at 11:59 PM on Gradescope.

Homework 6, (.tex), due October 13th at 11:59 PM on Gradescope.

Homework 7, (.tex), due October 20th at 11:59 PM on Gradescope.

Homework 8, (.tex), due October 27th at 11:59 PM on Gradescope.

Homework 9, (.tex), due November 3rd at 11:59 PM on Gradescope.

Homework 10, (.tex), due November 20th at 11:59 PM on Gradescope.

Homework 11, (.tex), not due.

Course Calendar

The following calendar is subject to revision during the term. The section references are only a guide: our pace may vary from it somewhat. PK refers to Pinksy and Karlin. D refers to Durrett.

Week 1, Lec 1, Aug 23: Motivation, Conditional probability and conditional expectation (PK 2.1)
Additional reading: Probability review (D A.1-A.3, PK 1.1-1.6), Conditional expectation (PK 2.1)

Week 1, Lec 2, Aug 25: Discrete-time Markov chains (PK 3.1-3.2, D 1.1-1.2)

Week 2, Lec 3, Aug 28: Discrete-time Markov chains (PK 3.1-3.2, D 1.1-1.2)

Week 2, Lec 4, Aug 30: Discrete-time Markov chains (D 1.3 up to Theorem 1.2, PK 3.4, 3.7)
Optional reading: See D 1.9-1.10 for Durrett's treatment on absorbing Markov chains if you want another perspective.

Week 2, Lec 5, Sep 1: Discrete-time Markov chains (PK 3.4, 3.7)
Optional reading: See D 1.9-1.10 for Durrett's treatment on absorbing Markov chains if you want another perspective.

Week 3, Sep 4: Labor day

Week 3, Sep 6: Discrete-time Markov chains (PK 3.4, 3.7)
Optional reading: See D 1.9-1.10 for Durrett's treatment on absorbing Markov chains if you want another perspective.

Week 3, Sep 8: Branching processes (PK 3.8, 3.9)

Week 4, Sep 11: Branching processes (PK 3.8, 3.9)

Week 4, Sep 13: Branching processes (PK 3.8, 3.9), Long-run behavior of Markov chains (PK 4.1)

Week 4, Sep 15: Long-run behavior of Markov chains (PK 4.1, 4.2)

Week 5, Sep 18: Long-run behavior of Markov chains (PK 4.3)

Week 5, Sep 20: Long-run behavior of Markov chains (PK 4.3, 4.4)

Week 5, Sep 22: Long-run behavior of Markov chains (PK 4.4)

Week 6, Sep 25: Long-run behavior of Markov chains (PK 4.4)

Week 6, Sep 27: Poisson process (PK 5.1, D 2.1)

Week 6, Sep 29: Poisson process (PK 5.2, D 2.2)

Week 7, Oct 2: Midterm 1

Week 7, Oct 4: Poisson process (PK 5.3, 5.4, D 2.2)

Week 7, Oct 6: Poisson process (D 2.2, PK 5.3)

Week 8, Oct 9: Poisson process (PK 5.3)

Week 8, Oct 11: Poisson process (PK 5.4)

Week 8, Oct 13: Renewal process (PK 5.4)
Additional reading: Conditioning on a continuous random variable (PK 2.4), Thinning (D 2.3 through Example 2.8), Superposition (through Example 2.12)

Week 9, Oct 16: Renewal process (PK 7.1-7.3)

Week 9, Oct 18: Renewal process (D 3.1, PK 7.5)

Week 9, Oct 20: Renewal process (D 3.1, PK 7.5, D 3.2.1)

Week 10, Oct 23: Renewal process (D 3.2.3, 3.3.2)

Week 10, Oct 25: Renewal process (D 3.3.2)

Week 10, Oct 27: Continuous-time Markov chains (D 4.1)

Week 11, Oct 30: Continuous-time Markov chains (D 4.2)

Week 11, Nov 1: Continuous-time Markov chains (D 4.2, 4.3)

Week 11, Nov 3: Continuous-time Markov chains (D 4.3)

Week 12, Nov 6: Midterm 2

Week 12, Nov 8: Continuous-time Markov chains (D 4.3)

Week 12, Nov 10: Veterans Day

Week 13, Nov 13: Continuous-time Markov chains (D 4.4)

Week 13, Nov 15: Martingales (D 5.1)

Week 13, Nov 17: Martingales (D 5.1)

Week 14, Nov 20: Martingales (D 5.1, 5.2)

Week 14, Nov 22: Non-Instructional Day

Week 14, Nov 24: Thanksgiving

Week 15, Nov 27: Gambling Strategies, Stopping Times (D 5.3)

Week 15, Nov 29: Bounded convergence theorem (D 5.4), Exit times (D 5.4.1)

Week 15, Dec 1: Exit times (D 5.4.2), Cramér’s Estimate of Ruin (D Example 5.19).

Week 16, Dec 4: RRR week

Week 16, Dec 6: RRR week

Week 16, Dec 8: RRR week

Week 17, Dec 11: Final exam