"Just chance" vs "better on the day"
Popularized in Taleb's
Black Swan under the name narrative fallacy is the idea that recounting past events as a sequential story
"causes us to see past events as more predictable, more expected, and less random than they actually were".
Here is an example to consider.
The sports context
Let me start by contrasting two extreme cases.
If two people play roulette, betting the same amounts on different numbers, then one will
do better than the other, but readers will agree with me this is "just chance".
In contrast, imagine two top contenders in an Olympic sport like the marathon or javelin.
The winner is unpredictable -- before the contest one could find betting odds.
It's possible that some specific chance event occurs -- one contestant gets injured,
for instance. But absent some very specific such event, we would never say that the
winner won "just by chance" -- instead, we would say they were "better on the day".
What about golf?
PGA players make 8 foot putts about 50% of the time.
Suppose for player A it's 60% and for player B it's 40%, so that's a difference in skill.
Then suppose this situation arises one day, and player B makes the putt but player A doesn't.
Do we say "a chance event of probability 16% happened" or do we say "B was better on that hole"?
Most of us would say the latter -- unless B's ball first teetered on the edge of the hole, in
which case we would perceive a very specific lucky event.
What about a team sports match, where gambling odds give team C a 2/3 chance to beat team D, but team D wins?
Reading a journalist's report of such a match,
you usually find many more references to skill or errors than to chance or luck.
Randomness as unpredictability
Recognizing that something is unpredictable and assigning a probability does
not mean that we think it is anything "pure chance" like a die.
Rather, many future events in the human world are like a team sports match, in that there is some
complicated chain of cause-and-effect which is impossible to predict in any detail
but where we use overall knowledge of past similar events to guess a probability.
The fact that we retrospectively see specific cause-effect relations within the
process leading to the event is irrelevant, to the extent that
the causes are unpredictable.
The daily stock market rise or fall provides a similar example.
Retrospectively, commentators will assign specific causes to the rise or fall, but
(even if true) they are irrelevant, in the sense that these causes are unpredictable.
That's my take on the narrative fallacy.
Part of the conceptual issue here is that we don't have a specific word for this common
setting of unpredictability, and we are reluctant to say "random".
Hence my quixotic campaign to
retire "dice" as the icon for randomness.
When does the narrative fallacy come into play?
Of course there are many everyday events that we do perceive as chance.
As noted elsewhere
a surprising proportion of
references to chance in blogs
involves "commentary" or "casual curiosity" about specific events perceived as chance.
The narrative fallacy therefore seems to come into play in more complicated contexts where we
expect skill or planning to dominate.
Consider team sports in the context of one season, rather than one match.
We recognize that the result of a single match is unpredictable, but
we perceive that the winner of last year's league or tournament was actually the best team,
rather than arising from a combination of skill and chance.
In other words, when we look ahead -- e.g. the
(July 2018) probabilities for 2019 Superbowl winner
-- we seek to assess which will be the most skillful team, and the uncertainty
involved is perceived as the uncertainty in that assessment rather than the
uncertainty in individual match results.
I have no idea how to find data relevant to these "perception" issues.
Loosely related are some modeling results which give a glimpse
of the "skill versus chance" issue here.
In a Premier League type context,
a change of skills, from one season to the next, in which a typical "A beats B" probability
changes by 10%,
would be hard to detect as different from the "unchanged skills" possibility.
See discussion around
Figure 2 of this paper; Figure 1 shows the effect of skill on a tournament setting.