Null and Vetoed: "Chance Coincidence"?

Philip B. Stark, 3 November 2009 (revised 8 February 2010; published in Chance, November 2010, v23(4), 43–46.)

Abstract

In late October, 2009, California Governor Arnold Schwarzenegger vetoed Assembly Bill 1176. The first letters of the third through ninth lines spell an expletive. Could this be chance coincidence? The probability that the acrostic would occur ``accidentally'' is very sensitive to the hypothesized probability model for generating the veto: Answers vary by many orders of magnitude. It is hard to find a compelling model. Multiplicity must also be considered. Common sense might be more persuasive than calculations.

Governor Schwarzenegger's Acrostic Veto

In late October, 2009, California Governor Arnold Schwarzenegger vetoed Assembly Bill 1176 (Ammiano). gov.ca.gov/pdf/press/2009bills/AB1176_Ammiano_Veto_Message.pdf (last accessed 5 February 2010). The veto reads as follows:

   To the Members of the California State Assembly:

   I am returning Assembly Bill 1176 without my signature.

   For some time now I have lamented the fact that major issues are overlooked while many 
   unnecessary bills come to me for consideration. Water reform, prison reform, and health 
   care are major issues my Administration has brought to the table, but the Legislature just 
   kicks the can down the alley.

   Yet another legislative year has come and gone without the major reforms Californians 
   overwhelmingly deserve. In light of this, and after careful consideration, I believe it is 
   unnecessary to sign this measure at this time.
 
   Sincerely,
   Arnold Schwarzenegger

The first letters of the third through ninth lines comprise an expletive acrostic. The news media have wondered whether this could be accidental.

The governor's office has claimed that the acrostic is "a wild coincidence" (http://www.huffingtonpost.com/2009/10/27/schwarzenegger-sends-lawm_n_336319.html, last accessed 5 February 2010). Some have claimed that the chance the acrostic would occur accidentally is 1 in 10 million (http://www.huffingtonpost.com/2009/10/30/schwarzenegger-f-bomb-in_n_340579.html, last accessed 5 February 2010), but that calculation depends on rather contrived assumptions. I believe there is no real answer to the question "what's the chance the acrostic would occur accidentally?" because there is no single sensible chance model for the wording of a veto. The issue is the framing of an appropriate "null hypothesis" under which to compute the chance.

While I am sure that Governor Schwarzenegger chooses his words carefully, this colorful example invites comparing a variety of null models for how the veto might have been worded "at random." To put the question in a statistical framework, we ask "if a seven-line message had been worded randomly, what is the chance that that particular acrostic would have occurred?" (We ignore for the moment the fact that many other acrostics would have triggered a similar media response, e.g., b-u-g-g-e-r--o-f-f. We also ignore for now the fact that the Governor vetoes many bills. Multiplicity issues such as these could greatly increase the calculated probabilities.)

"Randomly" does not mean much by itself. We could concoct any number of models for wording the message at random. In this note, I consider six. They give probabilities that span more than 8 orders of magnitude.

Monkeys banging on typewriters

If seven lines were typed with each character chosen at random, independently, from the 26 letters of the alphabet (ignoring case, spaces, numbers, and punctuation), the chance that the first letter of those seven lines would spell the acrostic is

(1/26)7 = 1.245e-10,

i.e., 1 in 8,031,810,176.

Random words from the Project Gutenberg corpus

Presumably, the governor was constrained to use English words in his veto. The frequency of initial letters of English words is not uniform. According to Wikipedia (http://en.wikipedia.org/wiki/Letter_frequency, last accessed 3 November 2009), the relative frequencies of the relevant initial letters in the Project Gutenberg corpus (http://www.gutenberg.org/wiki/Main_Page, last accessed 5 February 2010) are as follows:

letter frequency
c 0.03511
f 0.03779
k 0.00690
o 0.06264
u 0.01487
y 0.01620

If seven words were chosen from the Gutenberg corpus at random, independently, to start the seven lines, the chance their initial letters would comprise the acrostic is

0.03779 × 0.01487 × 0.03511 × 0.00690 × 0.01620 × 0.06264 × 0.01487 = 2.054e-12,

i.e., 1 in 486,804,391,348.

Permuting the lines

Suppose that the seven lines were given, and the order were shuffled. What is the chance that the letters would come out in an order that comprises the acrostic? Two of the lines start with "u" and the rest start with distinct letters. The chance is thus

1/(7C2 × 5 × 4 × 3 × 2) = 0.0004,

or 1 in 2,520.

Permuting the words

Suppose we hold the Governor to his words, but not to their order. That is, suppose we took the 85 words that comprise the seven lines, wrote them each on a card, shuffled the cards well, then dealt them sequentially to put them in a random order. We keep the number of words in each line fixed, so that there are linebreaks after the 16th word, the 29th word, etc. What would the chance be that the initial letter on each line would comprise the acrostic?

Those 85 words include the relevant initial letters with the following counts:

letter count
c 8
f 3
k 1
o 3
u 2
y 2

There are 85! sequences in which the words can be dealt (not all are distinguishable, because some words occur more than once). For the order to spell the acrostic, the first word must be one of the three words that start with "f," the 17th word must be one of the two words that start with "u," etc. Once those seven words are specified, the remaining 78 can occur in any order. Thus the probability of drawing the 85 words in an order that gives the acrostic is

3×2×8×1×2×3×1×78!/85! = 1.158e-11,

or 1 in 86,377,328,100.

Permuting words within lines

We might hold the set of words in each line fixed (acknowledging that the ideas expressed by the words might have a required order), and permute the order of the words within lines randomly. Conceptually, each word on a line is written on a card, then those cards are shuffled well and dealt in a sequence. The number of words on each line and the number that start with the requisite letter are in the table below.

linewordslettercount
116f 2
213u 1
315c 1
46k 1
513y 2
614o 1
78u 1

The number of ways the lines could be internally reordered is

16! × 13! × 15! × 6! × 13! × 14! × 8!.

Not all of those are distinguishable, because some words occur more than once on a single line. To have the acrostic, one of the correct letters must be in the first position on each line. The number of orderings that give the acrostic is thus

2 × 15! × 12! × 14! × 5! × 2 × 12! × 13! × 7!.

Under this null model of shuffling the words in each line, the probability of generating the acrostic is

(2 × 15! × 12! × 14! × 5! × 2 × 12! × 13! × 7!)/(16! × 13! × 15! × 6! × 13! × 14! × 8!) = 4/(16 × 13 × 15 × 6 × 13 × 14 × 8) = 1.468e-07,

or 1 in 6,814,080.

Random linebreaks

Suppose that the governor's words were kept in the order in which they actually occurred, but that the 85 words were divided by linebreaks into seven lines, each with at least one word. Suppose that every way of breaking the lines was equally likely. What is the chance that the first letters on the seven resulting lines would spell the acrostic?

To end up with seven lines, six linebreaks must be inserted. A linebreak may be inserted before the second word, the third word, …, or before the 85th word. There are thus 84 places into which 6 linebreaks must be inserted, 84C6 equally likely partitions. Of those, only 12 produce the acrostic (the break to produce "u" at the beginning of the second line can be in only one place; the break to produce "c" at the beginning of the third line can happen in any of 3 places; the break for the "k" can be in only one place, etc.). Hence, the chance that this randomization scheme would produce the acrostic is

12/84C6 = 2.952e-08,

or 1 in 33,873,462.

Other models

We might keep phrases together, and shuffle them, and see where the new linebreaks fall. We might allow some words or phrases to substitute for others that express the same idea. There are countless null models we could postulate. None is compelling.

Re-cap and Discussion

The null hypothesis for testing "coincidences" matters. In this example, it is easy to get a wide spectrum of values for the "probability" of a coincidence. In the six calculations here, the probability ranges from about one in a couple of thousand to one in 487 billion: a factor of nearly 200 million—more than 8 orders of magnitude. News consumers should be wary of calculations of the "chance" of a coincidence, regardless of the context.

Bayesian approaches

These calculations all give the probability that the acrostic would occur, if it is an accident—for different ways of generating the veto at random. Some might prefer to know the chance that the acrostic is an accident. I don't think there's any good way to answer that question. An answer would require a prior probability that the acrostic is accidental, as well as the probability that the acrostic would occur if the veto were generated innocently. We have already seen that the second probability is a problem. The first is, too.

For instance, suppose we think there's a 10% chance the Governor would do this sort of thing on purpose. Suppose we also think that if the Governor did not do this on purpose, the biggest chance that the acrostic would come up accidentally is 1 in 2,520. Then the conditional probability that the Governor did not do this on purpose given that the acrostic occurred is

P(accident | acrostic) = P(accident and acrostic)/P(acrostic) ≤ (0.9/2520)/(0.1+0.9/2520) = 0.4%.

On the other hand, if we think there's only a 1 in 1,000 chance that the Governor would do this sort of thing on purpose, then

P(accident | acrostic) ≤ (0.999/2520)/(0.001+0.999/2520) = 28.4%.

There is no objective way to get a prior probability, nor is there a good way to upper bound the chance the acrostic would happen accidentally. Different values give different conclusions.

Questioning the assumptions

I don't think any of the sets of assumptions behind the numbers presented above is realistic. None takes into account the fact that the message must be composed of sentences, sentences that have to make sense in the context of a veto. Perhaps a better "null model" would pull full sentences at random from Governor Schwartzenegger's other vetoes, string them together, and see where the linebreaks fell. Moreover, the computations presented in this note take the number of lines to be fixed, equal to seven, but the veto could have had more or fewer.

The probability calculations do not take into account the number of vetoes the Governor has written in all. The larger the number, the greater the chance this could happen accidentally. Nor do the calculations consider other acrostic expletives that might have occurred. For instance, here are the same 85 words in a different order:

   Prison reform, water reform, and health care are major issues the Legislature has overlooked,
   issues my Administration brought to the table. But while I have lamented of unnecessary reforms for 
   some time now, yet another legislative year has come and gone, and without major bills to
   sign.  I believe this after careful consideration.

   Overwhelmingly, Californians deserve this consideration--the major light that can, in 
   fact, measure the unnecessary. Many are come to me down the alley at this time. It is just 
   for the kicks.

Strained, yes. But it parses. And here are the 85 words, with one substitution ("now" → "sadly"):

   Yet another legislative year has come and gone without the major reforms Californians 
   overwhelmingly deserve. In light of this, and after careful consideration, I believe it is 
   unnecessary to sign this measure at this time.

   Sadly, I have for some time lamented the fact that major issues are overlooked while many 
   unnecessary bills come to me for consideration.  Water reform, prison reform, and health 
   care are major issues my Administration has brought to the table, but the Legislature just 
   kicks the can down the alley.

Either of those re-writes might have triggered media attention like that the actual veto received, but such alternative acrostics are not contemplated in the calculations above. However, they increase the chance of expletives under most of these models. Similarly, the calculations do not consider acrostics formed from the last letter in each line, etc.

Is there a better approach?

It might make more sense to look at the actual phrases in the veto to see whether they seem natural or strained. For instance, "kicks the can down the alley" is unusual (I'm not quite sure what it means)—and it provides the rarest of the initial letters, the k. Has the Governor used that phrase elsewhere? "Overwhelmingly deserve" also strikes me as odd. In the construction, "Water reform, prison reform, and health care" it seems more natural and parallel to say "health care reform," but that would move the next linebreak so that the (rare) k would not be in the first position. In my opinion, such "forensic text analysis" is more persuasive than the probability calculations, which are at least as contrived as the veto!

Conclusion

In summary, you don't get probability out without putting probability in. You are, in effect, making up the probability of a coincidence by inventing the null model for producing the coincidence. Gubernatorial vetoes are not written by choosing letters, words, or sentences at random.

Acknowledgments. I am grateful to Yoav Benjamini, Ronald Rivest, and Hadley Wickham for comments on an earlier draft.

© 2009, 2010, 2011 P.B. Stark. Last modified 17 July 2011.

http://statistics.berkeley.edu/Preprints/acrosticVeto09.htm