What's the difference between probability and statistics?

In these notes I view Probability as a predictive science analogous to Physics.

The focus of Statistics (as an intellectual discipline) is on using quantitative data to try to answer questions you don't know yet know the answer to. For instance, Marin County (an affluent and liberal community north of San Francisco) has breast cancer rates about 15% higher than the rest of California. Is this due to some feature of the environment or lifestyle specific to Marin County, or is it just that people choosing to live there have more of the attributes known to increase breast cancer risk? No-one knows the answer, but no-one is going to find the answer without gathering and analyzing data, i.e. without statistics.

That kind of serious applied statistics isn't teachable via a site like this. Amongst popular science books, Senn's Dicing With Death gives the best glimpse of serious applied statistics. On the other hand most textbook theoretical statistics is just mathematical probability, and I use the word ``probability" to encompass such theoretical statistics.