Statistics 150: Stochastic Processes-- Spring 2010

Instructor: Jim Pitman, Department of Statistics, U.C. Berkeley.

Office hours: TBD in 303 Evans

Weekly homework assignments are drawn from the text An Intro to Stochastic Modeling (3rd ed) by Karlin and Taylor.

Midterm Exam:   Thursday March 11, in class.
Final Exam:   Thursday 5/13/10 3-6pm
GRADES: Overall scores will be computed as follows:
Larger of (0.1 * hwk + 0.4 * midterm + 0.5 * final) and (0.1 * hwk + 0.9 * final).

Lecture and Homework Schedule

Week 1

  • Lecture 1: Overview. Probability spaces, Expected value [.pdf].
  • Lecture 2: Conditional Expectation. Wald's Identity. Gambler's ruin for fair coin [.pdf].
  • Homework 1: (due 1/28) P. 79: 3.3, 3.4. P. 85: 4.3, 4.4. 4.6

Week 2

  • Lecture 3: Martingales. Gambler's ruin for biased coin. [.pdf]. Similar notes from a previous year: [.pdf]
  • Lecture 4: Conditional independence and Markov chains [.pdf]
  • Homework 2: (due 2/4) P. 94: 5.1, 5.2, 5.3, 5.4, 5.5

Week 3

  • Lecture 5: Markov chains. First step analysis I. [.pdf]
  • Lecture 6: Markov Chains. First step analysis II. [.pdf]
  • Additional notes (from a previous year): Transition Probabilities. Death and Immigration Chain [.pdf]
  • Homework 3: (due 2/11) P. 100 1.3, 1.4. P 105 2.4. P 114 3.5. P 115 3.9

Week 4

  • Lecture 7: Limits of Random Variables [.pdf]
  • Lecture 8: First passage and occupation times for random walk [.pdf]
  • Homework 4: (due 2/18) P. 130 4.1, 4.2, 4.5, 4.6, 4.10

Week 5

Week 6

  • Lecture 11: Return times for random walk [.pdf]
  • Lecture 12: Probability Generating Functions [.pdf]
  • Homework 6: (due 3/4) P 184, 8.4; P 195 9.4, 9.5, 9.7, 9.8

Week 7

  • Lecture 13: Branching processes [.pdf]
  • Lecture 14: Branching processes and Random Walks [.pdf]
  • No Homework. Midterm exam next week,

Week 8

  • Lecture 15: March 10: Midterm Review. Sample midterm exams: 2006 [.pdf] 2009 [.pdf]
  • Lecture 16: March 12: Midterm Exam. In class. Closed book. OK to bring one page (single side) of notes.
  • Homework 7: Provide solutions to all problems on the midterm.
  • Midterm: Problems only: [.pdf]. Problems and solutions: [.pdf].
  • Here are the raw scores on the midterm (47 scores):
    [48, 44, 43, 41, 38, 37, 37, 36, 36, 33, 33, 31, 31, 29, 28, 27, 27, 26, 26, 25, 24, 24, 23, 22, 22, 22, 21, 20, 20, 19, 19, 19, 18, 18, 17, 15, 14, 13, 13, 13, 12, 12, 11, 11, 10, 5, 5]
    Divide your score by 48 and multiply by 100 to get the value of "midterm" which will be used to compute your overall score according to:
    Larger of (0.1 * hwk + 0.4 * midterm + 0.5 * final) and (0.1 * hwk + 0.9 * final).

Week 9

  • Lecture 17: Long run behaviour of Markov chains. [.pdf] Lecture from a previous course: [.pdf]
  • Lecture 18: Long run behaviour of Markov chains: problems. [.pdf]
  • Homework 8: due 4/1: P 211 1.3 , P 214 1.13, p 256 4.3, P257 4.6, P258 4.8

SPRING BREAK

Week 10

  • Lecture 19: Stationary Markov Chains [.pdf]
  • Lecture 20: Markov Chains: Examples [.pdf]
  • Homework 9: due 4/8: P 296 3.6, P 297 3.8 , P 309 4.4, p 315 5.2 p 329 6.3

Week 11

  • Lecture 21: Poisson processes [.pdf]
  • Lecture 22: Continuous time Markov chains [.pdf]
  • Homework 10: due 4/15: P 343 1.7, p 354 2.1, p 365 3.1, p 376 ex 4.1, p 377 4.4

Week 12

  • Lecture 23: Continuous time Markov chains: continued. Notes from a previous year (some overlap with Lec 22)[.pdf]
  • Lecture 24: Queuing models [.pdf]
  • Homework 11: due 4/22: P 407 6.2, 6.3, 6.4 and P 556 2.4, 2.5

Week 13

  • Lecture 25: Renewal Theory [.pdf]
  • Lecture 26: Brownian motion [.pdf]
  • Homework 12: due 4/29: P 426 1.3. Deduce from this result the asymptotic equivalence of M(t) and t/E(X) as t tends to infinity, assuming to make the argument easy that F(T) = 1 for some finite T . P 436 3.4, P 456 5.1, P 457 5.4

Week 14

  • Lecture 27: Hitting Probabilities for Brownian Motion
    [.pdf]
  • Lecture 28: Brownian bridge
    [.pdf]
  • Homework 13: (not graded) Page 489 1.5, Page 497 2.1, Page 506 3.1, Page 522 4.2 and 4.3

Week 15

Final Exam: Thursday 5/13 3-6pm