Topics for (presumed) review:
Reference: SticiGui.
Bernoulli trial: two possible outcomes, success and failure. The probability of success is p; the probability of failure is 1−p. Recall the definition of independence for a collection of events and for a collection of random variables; the heuristic is that they are uninformative with respect to each other. Events A and B are independent if P(AB) = P(A)×P(B). A collection of events is independent if the probability of the intersection of every subcollection is equal to the product of the probabilities of the members of that subcollection. Two random variables are independent if every event determined by the first random variable is independent of every event determined by the second.
Consider a sequence of n independent Bernoulli trials with the same probability p of success in each trial. Probability distribution of the total number X of successes is binomial:
P(X=x) = _{n}C_{x} p^{x} (1−p)^{n−x}.
A simple random sample of size n from a finite population of N things is a random sample drawn without replacement in such a way that each of the _{N}C_{n} subsets of size n from the population is equally likely to be the sample. Consider drawing a simple random sample from a population of N objects of which G are good and N−G are bad. The probability distribution of the number X of good objects in the sample is hypergeometric:
P(X=x) = _{G}C_{x} × _{N−G}C_{n−x}/_{N}C_{n},
for max(0, n−(N−G)) ≤ x ≤ min(n, G).
Null and alternative hypotheses; type I and type II errors. Neyman-Pearson paradigm. Significance level, power, P-values.
Null and alternative hypotheses are competing theories about the world. The labeling sometimes seems arbitrary. The essential thing in deciding which hypothesis is the null is that one must be able to calculate the distribution of the test statistic when the null hypothesis is true. Type I error: erroneously rejecting the null hypothesis. Type II error: erroneously failing to reject the null hypothesis. Significance level: the chance of a type I error. Power against a given alternative: the chance of correctly rejecting the null when that alternative is true. If we have a family of tests of a given hypothesis, indexed by significance level, so that we can test the null hypothesis at any significance level, the P-value of the null hypothesis given a set of data is the smallest significance level for which all tests in the family with significance levels larger than P would reject the null hypothesis. That is,
P = inf { α : tests at level α' would reject the null whenever α' ≥ α}.
Note: the P-value is not the chance that the null hypothesis is true. The P-value is the probability of a particular event, calculated assuming that the null hypothesis is true. The null hypothesis is either true or false. We don't (usually) talk about the probability that hypotheses are true.
Parametric statistics: assume a functional form for the probability distribution of the observations; worry perhaps about some parameters in that function. Non-parametric statistics: fewer, weaker assumptions about the probability distribution of the observations; perhaps assume that the observations are i.i.d. or exchangeable, or that the underlying probability distribution is continuous (has a density). (Modern nonparametrics includes infinitely parametric problems, such as density estimation and function estimation, where specifying the unknown requires possibly infinitely many parameters.) Robust: assume a functional form for the probability distribution, but worry about whether the procedure is sensitive to "small" departures from that assumed form. The sense in which the departure is assumed to be small, and the kinds of departures that are permitted, are up to the practitioner.
Nonparametric methods based on randomization or ranks are about as robust as can be. They are our next topics.