The core of this lab centers on aiding the student in identifying and using the probability model in a hypothesis test. From a nontechnical description of an experiment and null hypothesis, the student is expected to be able to
At the start of the lab, after some brief instructions, the student is presented with a set of experiments in the Topics window . The topics window contains a table of contents, listing the experiments the student is expected to complete. In the example shown here the student is presented with two experiments. Each is accompanied by a short description of the problem. The instructor can choose any number of experiments for the lab, and they can be selected randomly for each student from a subset of those available for the lab. With random selection, students have less incentive to share results and they get a greater diversity of scenarios from which to learn the statistical abstractions. To change an experiment or to add a new experiment to the lab, the instructor makes the appropriate changes and additions to input files.
Consider the second experiment. The goal there is to determine whether one cloth manufacturer produces better fabric than another. One manufacturer has been used for several years and is taken as the standard. The other will be considered for a trial period, after which the decision whether to switch manufacturers needs to be made. Before proceeding with the lab, the student must first venture an opinion as to the strength of evidence needed for him to recommend switching manufacturers. This request is made to encourage the student to think about the problem before proceeding, to introduce the notion of setting a critical level in advance of experimentation, and to involve the student in the decision making process.
After the student makes his guess, he proceeds to an animation where he conducts the experiment. In this example, shipments of cloth leave the mill in trucks, and at the intersection the student chooses whether or not to send the shipment on to the jeans plant. When shipments arrive at the plant, they are examined and tallied as defective or not. After all shipments have been tallied, a test statistic is computed. The animation helps the student identify the source of randomness in the experimental process, and helps him understand how the data are created and compiled to form the test statistic. In other labs, such as the Stat Park, the student can and is encouraged to rerun the animation as often as he pleases to get a sense of the chance process in the amusement park game. Here, instead, he is restricted to one run of the experiment and the resulting test statistic.
After conducting the virtual experiment, the student proceeds to the Sampler, which is the centerpiece of the lab. Each experiment, regardless of its context, can be translated into a probability model using the Sampler. Here the student chooses ticket values and quantities for the pool, and determines the number of tickets to be drawn from the pool, whether draws are made with or without replacement, and the operation to perform on the tickets drawn, e.g. sum or average. The goal is to have the Sampler mimic the experiment. To do this the student uses the assumption that the new cloth manufacturer is no better than the standard and the chance process in the experiment. This lab is designed to follow the Stat Park lab mentioned above. In that lab the student first encounters the Sampler, and gains experinece in describing a chance process using the Sampler.
Once the Sampler is parameterized, the students runs an animation of the simulation that determines the null distribution of the test statistic. The animation helps the student see his test statistic as one of many possible outcomes from the experimental process, which in turn helps him in evaluating the significance of the observed statistic. The animation is performed in a cartoon-like way where an assistant draws tickets from the pool, computes the statistic, and repeats the process over and over hundreds of times (there is a fast forward version of the animation).
The student then proceeds to the Post Sampler where he is asked to consider the opinions offered by experts and to reconsider his original cut-off. He is encouraged to interpret the cut-off level that he and others provide in light of the null distribution. The experts' opinions are offered in a table. Each opinion is accompanied by either an p-value or critical-level. The student fills in the associated missing critical or p-value by reading tail probabilities from the histogram of the null distribution. To aid the student, tail areas are calculated interactively by moving a slider along the horizontal axis, and the tail areas are entered dynamically into the talbe by clicking on the appropriate cell.
The opinions offered by the experts include both traditional statistical statements concerning p-values and alpha-levels and expressions we have heard students give as their reasons or gut-feelings for selecting a particular cut-off. Also at this time, we ask the student to draw a conclusion, choosing between a statistically sound decision and an adhoc remark. These different sources of opinions and conclusions are purposely juxtaposed in hopes that students will eventually be able to discern between them.
|Main| |General| |Labs| |Tool Kit| |Tools|
This page was last updated on 09/26/98. Questions, comments, suggestions.