This material was copied from p172-180 of Sense and Nonsense of Statistical Inference , by Chamont Wang, Marcel Dekker (1993)

Vl. WOMEN AND LOVE: A CASE STUDY IN QUALITATIVE/QUANTITATIVE ANALYSIS

In 1972 a behavioral researcher, Shere Hite, initiated a large-scale study of female sexuality. The result was a monumental work considered by many the third great landmark in sex research, after The Kinsey Report (1948, 1953) and Human Sexual Responses (1966) by Masters and Johnson. In this section, we will examine scholarly debates over Hite's work. In addition, we will discuss the weakness of a common practice (used in Hite's work) that compares statistical breakdowns between the sample and the target population.

A motivation behind Hite's project on female sexuality was that (Hite, 1976):

Women have never been asked how they felt about sex. Researchers, looking for statistical "norms," have asked all the wrong questions for all the wrong reasons_ and al] too often wound up telling women how they should feel rather than asking them how they do feel.

The purpose of this project is to let women define their own sexuality_instead of doctors or other (usually male) authorities. Women are the real experts on their sexuality; they know how they feel and what they experience, without needing anyone to tell them.

For this purpose, Hite believed that:

a multiple-choice questionnaire was out of the question, because it would have implied preconceived categories of response, and thus, in a sense, would also have "told" the respondent that what the "allowable" or "normal" answers would be. (Hite, 1987)

As a consequence, Hite decided to use essay questionnaires and claimed that they "are not less 'scientific' than multiple-choice." By 1976, a book based on 3,000 women's responses to her essay questionnaires was published and soon became a huge success in both sales volume and the candid revelations of how women really feel about sex.

The study was replicated (and confirmed) in at least two other countries. The book and her second volume on male sexuality have been translated into 13 languages, and are used in courses at universities in this country and around the world. Because of her work, Hite was honored with a distinguished service award from the American Association of Sex Educators, Counselors and Therapists.

An important revelation in Hite's 1976 report is that women had been compelled to hide how they feel about lack of orgasm during intercourse. A natural conclusion (Hite, 1976) is that the traditional definition of sex is sexist and culturally linked:

Our whole society's definition of sex is sexist_sex for the overwhelming majority of people consists of foreplay, eventually followed by vaginal penetration and then by intercourse, ending eventually in male orgasm. This is a sexist definition of sex, oriented around male orgasm and the needs of reproduction. This definition is cultural, not biological.

In the early 1980s, Hite further sent out 100,000 questionnaires to explore how women were suffering in their love relationships with men. The response to this survey was a huge collection of anonymous letters from thousands of women disillusioned with love and marriage, and complaints about painful and infuriating attitudes on the part of men.

Hite's book, Women and Love (1987), thus provided a channel of release for women who experienced frequent degradation and ridicule by men. The book advocates another "cultural revolution" (instead of "sexual revolution") and was praised by some scholars:

The Hite report on Women and Love explodes the current myth of the "insecure women" who seek out destructive relationships. . . it documents a hidden, "socially acceptable" pattern of behavior in relationships which puts women's needs last.

Naomi Weisstein, Ph.D.
Professor of Psychology
SUNY, Buffalo

A massive autobiography of women today.

Catherine R. Stimpson
Dean of Graduate School
Rutgers University

The Hite Report trilogy will long be regarded as a landmark in the literature of human behavior. . . respect will continue to grow for these works in years to come.

Richard P. Halgin, Ph.D.
Department of Psychology
University of Massachusetts

The most important work on men and women since Simone de Beauvoir's The Second Sex.

Albert Ellis, Ph.D.

A distinguished report with scientific and scholarly authority.

Dr. Tore Haakinson
Wenner-Gren Center
Stockholm

By reading these reviews, one may conclude that the third Hite Report must be a masterpiece with unquestionable authority. But in fact the findings and the methodology used in the book received a thunderous rebuttal from statisticians, sociologists, public opinion pollsters, and psychologists (Time, October 12, 1987; Newsweek, November 23, 1987). In the following discussions, we will quote and discuss conflicting views on the Hite methodology.

When the first Hite Report (1976) was published, a medical writer for The New York Times critized the report as a "non-scientific survey." To fend off such criticisms, Hite (1987) enlisted a score of scholars to address the issue of scientific methods. Here is a representative sample of supports for the Hite methodology:

The Hite studies. . . represent "good" science, a model for future studies of natural human experience.... Hite is a serious, reliable scholar.

Gerald M. Phillips, Ph.D.
Pennsylvania State University
Editor, Communications
Quarterly Journal

Hite elicits from the populations she studies not only reliable scientific data, but also a wide spectrum of attitudes and beliefs about sex, love and who people are.

Jesse Lemisch
Professor, Department of History
SUNY, Buffalo

Hite has not generalized in a non-scholarly way. Many of the natural sciences worry a lot less about random samples.... And most of the work in the social sciences is not based on random samples either.

John L. Sullivan, Ph.D.
Professor of Political Science
University of Minnesota
Co-editor, American
Journal of Political Science
Editor, Quantitative Applications in the Social Sciences

It has been one of Hite's contributions to devise an excellent methodology. Hite's work has been erroneously criticized . . . as being "unscientific" because her respondents, though numerous, are not a "random sample." Hite has matched her sample
carefully to the U. S. population at large; the demographic breakdown of the sample population corresponds quite closely to that of the general U.S. population.

Gladys Engel Lang Professor of Communications,
Political Science and Sociology
University of Washington, Seattle

Hite herself was also confident about the methodology (1987, p. 777):

Sufficient effort was put into the various forms of distribution that the final statistical breakdown of those participating according to age, occupation, religion, and other variables known for the U. S. population at large in most cases quite closely mirrors that of the U. S. female population.

For instance, here are some comparisons between the study population and the U. S. female population:

By looking at these comparisons, it is tempting to conclude that the study population represents well the U.S. population. But this is not the case, and unfortunately this is the position taken by Hite and by those who praised her work as a scientific survey.

For example, Hite was confident enough to refute both the percentage and methodology of The Kinsey Report:

In 1953, Alfred Kinsey found that 26 percent of women were having sex outside of their marriages. In this study, 70 percent of women married five years or more are having sex outside their marriage. This is an enormous increase. Women's rate of extramarital sex has almost tripled in thirty-five years. Kinsey~s figures were just rather low because he was conducting face to face interviews.... This may have inhibited reporting on the part of some interviewees. The present study is based on anonymous questionnaires, so that women would have had no reason to conceal anything.

But Time (1987) pointed out that "more recent studies, including the Redbook poll, have shown little change [in the infidelity rates]" from Kinsey's original 26%. Another random sample of 2,000 people (Patterson and Kim, 1991) shows that 31% of married people are having or have had an affair, drastically different from Hite's 70% figure.

In one chapter entitled "Ten Women Describe Their Marriages," Hite (1987, p. 346) wrote: "The women in this chapter represent the more than 2,000 married women participating in this study. Their responses were chosen because they represent well the thoughts and feelings of other married women." And her conclusion about marriage is as follows:

Feminists have raised a cry against the many injustices of marriage_exploitation of women financially, physically, sexually, and emotionally. This outcry has been just and accurate.

However, a 1987 Harris poll of 3,000 people found that family life is a source of great satisfaction to both men and women, with 89% saying their relationship with their partner is satisfying.

Some of the questions in Hite's survey appeared to solicit men-bashing. For instance, 69% of women in her study say that men seem confused by falling in love, and 87% say it is difficult to meet men they admire and respect. Such findings, although true in certain cases, are at odds with the Harris poll mentioned above.

In another instance, Hite reported that 98% of women want to make basic changes in their relationships. Well, everybody would agree that things in love relationships cannot be 100% perfect. But as warned by Tom Smith of the National Opinion Research Center: "Any question you asked that got 98% is either a wrong question or wrongly phrased" (Time, October 12, 1987).

Some scholars asserted that the Hite survey was designed to support her preconceived feminist notions. And perhaps the most hard-edged comment was from Ellen Goodman, a Pulitzer-prize-winning columnist: "She goes in with a prejudice and comes out with a statistic" (Time, October 12, 1987).

Note that the sample size in Hite's report was 4,500, a lot larger than most of the Gallup Polls. So what went wrong? For one thing, the response rate of Hite's survey is 4.5 %. Just by common sense, one may wonder about the other 95.5%. Those women may not have a problem with men and thus did not bother to respond. According to chairman Donald Rubin of the Harvard statistics department (Newsweek, November 23, 1987), one should look for a response rate of 70 to 80 percent in order to draw valid conclusions in this type of study.

Upon further reflection, one may wonder about the reasons of those who chose not to respond. Here are some possible answers: It is hopeless to try to change the way men treat women; it is time-consuming to answer a long questionnaire; "I" am only one of millions who suffer, and it won't make any difference if "I" do not respond.

We believe that the actual number of women disillusioned with marriage and love would be a lot higher than the 4.5% who responded to the Hite survey. Assume that there are 70 million women in this country and only 10% are so unlucky as to have chosen the "wrong man," then there would be 7 million people involved in the study, more than the number of patients of any single disease in modern society.

For this reason, in this subsection, we will turn our discussion to the insight and the contribution of the Hite Reports. After all, her first report was ground-breaking and deserves a place in history. Her third report, although seriously muddled, addresses issues tied to real and pressing needs of human beings.

In the third Hite Report (Women and Love, 1987, p. 246), a woman wrote with self-awareness and great passion:

Love relationships with men aren't fair; but so what, I still want one. I just want to know what I have to do to get one to work.

We believe that many women share the same attitude for love. In public, every educated person would agree human beings are created equal; but in personal relationships, many women still assume (or are forced to assume) the role of "caretakers," while men remain the "taker" (Hite, 1987, p. 771). With increasing numbers of women entering the work force and sharing the financial burden of modern life, it is expected that more women must balance husband, children, and career. It is also expected that fairness and equality are only mirages in many relationships.

The third Hite Report, Women and Love, is in many ways a moving tale of repression and humiliation. Page after page, one is struck by the despair, alienation, and rage that certainly exist in many corners of our society. Professor Joseph Fell of Bucknell University summarized what has been revealed in Hite's books: "I find in the Hite Reports a massive cry of the human heart, a cry for the open recognition of the fundamental needs."

But such open recognition was seldom revealed in depth by the traditional methods that use random sample and multiple- choice questionnaires. This fact is complimented by scholars who are tired of dominant "objective" statistical methods (Hite, 1987):

[Hite's survey used] questionnaires that invited long and detailed replies rather than brief, easily codable ones.... And from these responses, presented in rich, raw detail, deep truths emerge_truths which, in many cases were probably never revealed to anyone else before.

Robert L. Carneiro, Ph.D.
Curator of Anthropology,
The American Museum ofNatural History, New York

Statistical representativeness is only one criterion for assessing the adequacy of empirical data. . . other criteria are particularly pertinent when looking at qualitative data. This is primarily the situation with Hite's research.

Robert M. Emerson, Ph.D.
Professor of Sociology, UCLA
Editor, Urban Life: A Journal of Ethnographic Research

Hite's research, as with all research dealing with thoughts and feelings, cannot be expected to be analyzed with the same techniques as those that tell us what doses of what drugs give what results in what kinds of patients.... We are a scientifically illiterate people, and honest scientists such as Hite are bound to suffer as a result.

Mary S. Calderone, M.D., MPH
Founder, Sex Education and Information Council of the U.S.

The Hite Reports are part of an international trend in the social sciences expressing
dissatisfaction with the adequacy of simple quantitative methods as a way of
exploring people's attitudes.

Jesse Lemisch Professor
Department of History SUNY, Buffalo

We basically agree with these scholars' support of the Hite methodology in that her methods are excellent instruments to penetrate into profound realities that had been previously unexplored. Much of the behavioral research we have come across (see, e.g., Chap. 1, Sec. I, Chap. 3, Sec. I, and Chap. 5, Sec. II) is loaded with statistical analysis but exhibits little intellectual content. Hite's approach, albeit naive in statistical methods, produced a landmark report and revealed much of the dark side of male-female relationships.

Nevertheless, we are appalled that the "Hite methodology" was used (and praised) in her statistical generalization from a sample to a population. We believe that what Hite should have done is a two-step procedure: (1) use essay questionnaires to explore depth and variety of responses, and (2) use a random-sample (and multiple-choice questions) to confirm statistical issues unearthed by EDA (exploratory data analysis) in step (1).

This second step is called CDA (confirmatory data analysis) in statistical literature. In Hite's case, the CDA can be easily done by hiring a pollster to confirm or refute the original findings.

For instance, 76% of the single women in Hite's study say that they frequently have sex on their "first date." The opposite of this number (by which we mean 24%) would be more likely, and a random sample should be conducted to assess the accuracy of the finding.

Hite (1987) reported that "to go from essay statements to mixed quantitative/qualitative data is a long and intricate process." She estimated that "in this study, there were over 40,000 woman-hours involved in analyzing the answers, plus at least 20,000 put in by the women who answered the questionnaire. This of course does not include the time and effort needed to turn the resulting compilation of data into a book."

However, more woman-hours do not automatically lead to a more scientific conclusion, and the two-step procedure might have reduced the 40,000 hours Hite used to analyze the answers to 30,000 or less. '

In sum, EDA often gives us more insights; but findings from EDA can also give us less than they seem to. Research workers who are not equipped to tell the difference between EDA and CDA may soon find themselves laughing stocks just like Hite (and scholars who praised Hite's "scientific" results).

An interesting fact is that Hite used the same methodology to produce three books (female sexuality, male sexuality, and women and love), but only the third book received a glare of public scrutiny (and ridicule). In her first report, Hite correctly pointed out that the traditional definition of female sexuality is cultural, not biological. Her findings on female orgasm, however, are biological, not cultural. The main point here is that many biological phenomena (such as hunger, thirst, and desire) are relatively universal. Without randomization, Hite's sample incidentally matched the population at large.

The third Hite Report (Women and Love), however, deals with attitudes and emotions, which are not all that universal any more. Without a random sample, generalization from a sample to a population can result in a collapse of credibility.

As an analogy to the first and the third Hite Reports, assume that you want one cup of homogenized milk and one cup of orange juice from two big bottles of both. For the cup of milk, you just pour it from the big bottle. But for the cup of orange juice, you have to shake the bottle.

Professor John Sullivan at the University of Minnesota (and editor of two academic journals) pointed out that "most of the work in the social sciences is not based on random samples." He further wrote (see Hite, 1987):

in fact, many if not most of the articles in psychology journals are based on data from college students, and then generalized. Interestingly, they are not criticized in the same way Hite has been.

This is true. And this is why most findings in psychology journals (and other soft sciences as well) have to be looked at with caution. Otherwise, these so called "scientific" findings may be a source of more riddles than clues.