TILE - Design of Experiments

Pedagocial Issues | Lab Overview | Outline of Screens | Other Features

Pedagogical Issues:

The main goal of this lab is to teach the basic concepts in experimental design:

• Enrollment: issues of self-selection and generalization outside of test group
• Comparison groups (treatment and control)
• The benefit of randomization in assignment of subjects to groups
• The biases that may be eliminated via subject-blindness, evaluator-blindness, and placebo use

In addition, the student is expected to learn to intelligently read a newspaper article, picking out the most important facts about how the experiment was run.

Lab Overview:

The lab consists of 2 to 3 practice rounds and one challenge round. In each of the practice rounds, the student will reconstruct an experiment by reading a newspaper article and a summary of a journal article that describe the experiment. In the challenge round, the student will set up an experiment more or less from scratch, and from the experiment a newspaper article will be written. For each experiment in the practice round, the student will be provided an excerpt of a newspaper article as well as a summary of an original journal article about the experiment. They are expected to create an outline of the experiment based on the information provided. An outline consists of several action-units, which we call "boxes''. The type of action-units and their purpose are as follows:

• Enroll -- who is to be enrolled as an experimental subject?
• Split -- how are the subjects to be split into groups? We only allow 2 groups.
• Treat -- what treatment does the subject receive?
• Measure -- what response is measured on the subject?

The practice rounds introduce different types of experiments to the students such as:

• a simple experiment where one group receives the treatment and the other does not
• a matched pair design, where all subjects receive two treatments (i.e. treatment and placebo) and the order of the treatment is randomized

In the practice rounds, the student will be provided with the skeleton of the outline -- a diagram of empty boxes. Their job is to fill in the outline, by determining who was selected for enrollment, how the split was made, what the treatments were, and what was measured. To do this, they highlight the appropriate phrases in the text of the newspaper article and journal summary. The text will then turn into an icon and be added to one of the boxes in the diagram. The icons also appear in an icon glossary as they are identified.

When the student completes the outline, the experiment is "run" via an animation of "subjects" running across the boxes and piling up to form a histogram of responses. After the experiment is run, the student will answer quiz questions that are designed to point out the advantages and disadvantages of the design. The practice rounds can be completed in any order, and the student can switch between experiments in the practice round at will.

The challenge round is a reversal of the flow of the practice rounds. The student begins with a problem to solve and an icon glossary. Using the glossary, they make an outline for an experiment. Now they have control over the general outline of the experiment, including the order of the boxes. There is no single correct answer.

Once their outline is complete, it is built and run via an animation as before. However, depending on the outline, different results may be obtained. This includes catastrophic ones where the subjects run off the screen or splat into a heap. Finally, from the response histogram(s) and the design outline, the student answers quiz questions. The answers to these questions and the design outline are then incorporated into a newspaper article about the student's experiment.

Outline of Screens:

The Mission screen will appear first as the student begins the lab. The navigate bar will contain the buttons for the Mission, Practice1-3, and Challenge rounds. It may be that the instructor wishes to require only 1 or 2 of the 3 practice rounds. Each round has 4 screens:

• Glossary
• News
• Animation
• Quiz

Each practice-round begins by showing the News screen. After that, when the practice round is selected, the student will return to the screen that they were last working on in that round.

There is a natural order for completion of each screen in the practice round: News to Animation to Quiz. There is really no need to visit the Glossary screen. The buttons for all 4 of the screens within each round will be displayed as tabs along the side of the screen.

The natural order of completion in the Challenge round is: Glossary to Animation to Quiz to News. The animation results are dynamically created according to the design the student creates for the experiment. Unlike in the practice round, there is no single correct answer.

Once all the questions in the Quiz are answered and submitted, a newspaper article is created from the answers to the quiz questions.

Other Features -- available in all labs:

• Worker. The Worker is an icon of a head with a hard hat on it, the person who will build the experiment once the design is complete. The Worker is in the bottom right corner of the screen. When clicked on, the Worker offers brief advice. This advice will appear in dialog boxes that look like speech bubbles.
• Status Bar. A status bar is always present at the bottom of the screen. This is part of the general screen described by Duncan. Messages about active areas will appear in the status bar as well as error messages. Error messages in the status bar will be colored for a few moments to get the students attention and then fade to grey.
• Help. Help on how to run the lab, on statistical terms, and on the labs in general is available at all times.

|Main| |Purpose| |Sample Labs| |Tool Kit| |Evaluation| |Links|

This page was last updated on 02/27/98. Questions, comments, suggestions.