Sunday, 1 July 2012

Coincidences surprising and inevitable

Kitler photographed by Amy Halligan

Why is it that we might have no problem suspending our disbelief when watching a movie about an alien invasion, but we would find it ridiculous if the writers of a realistic soap opera suddenly started a storyline involving aliens?

This is actually a question that has interested me for some time, even if I seldom gave it much thought.

I will be getting back to the lottery paradox I posed in my last post, but first I have some things to say about the nature of coincidences and improbable events, and this is one of the questions I will attempt to answer.

Plausibility of fictional coincidences

The reason why certain bizarre occurrences are acceptable in some television shows and not in others is in my opinion down to something about the nature of coincidences. It's all about the reason the events are brought to our attention.

Consider a television show such as the X-Files. Imagine that in series five the X-Files division is closed down by the higher-ups, but Mulder suddenly wins the national lottery and uses it to finance an independent paranormal investigation team. I imagine most viewers would find this preposterous and would lose interest.

But now consider the situation where this is the initial premise of a show. This is unlikely to pose such a problem. We are no longer being asked to believe in a staggeringly improbable coincidence happening to somebody we know for other reasons - instead we are considering the implications of a staggeringly improbable coincidence which happened to a stranger who was brought to our attention for this precise reason.

It seems that coincidences are perfectly fine if they are part of the premise of the show, but they can have a negative impact otherwise. And this makes sense. Coincidences are rare, but interesting coincidences are likely to be brought to our attention, and so it makes sense to make a television show based on an unlikely premise.

Even situations we would not normally regard as coincidences can be thought of in this light. If there was a long-running TV show following the career of an economist named Jeb Bartlet, we might think it had 'jumped the shark' when he is elected president of the United States. However nobody thinks there is anything wrong with having a show with Bartlet's presidency as its premise.

This is why it would not be a popular move for the writers of a realistic soap opera to suddenly introduce an alien abduction storyline, even though much of the audience may enjoy series about aliens. Even if we suspend our disbelief in them, alien abductions are certainly tremendously rare events. If a realistic story is going to include them, then they had better be the focus of that story in some way.

Over-reliance on coincidence to resolve plot holes or generate drama is a sign of bad writing. A story which is entirely built around a coincidence or series of coincidences is another matter - the story can be notable, beautiful and interesting because of the coincidence rather than despite it. The coincidence or rare event is the reason the story is brought to our attention.

Inevitable coincidences and type I errors

Coincidences may be surprising to us, but at the same time, they are inevitable. Thousands of unremarkable events happen to us every day. It is to be expected that coincidences will crop up from time to time. One of Richard Feynman's favourite jokes was to say 'You'll never guess what happened to me today! Nothing!'

The frequency of coincidences varies according to how impressive something has to be before you call it a coincidence.

For example, if we set a low bar, we should find coincidental events happen to us every day. Today, I had just that minute finished a job when I got the first call in a few days asking about the progress of that job. This is the best thing I could think of after searching my memory of the day for coincidences. I imagine on most other days I could come up with something similar.

However if you wait longer periods of time, coincidences will be both more numerous and impressive. In the course of a life, you can expect to experience some pretty profound experiences at one point or another.

It's natural to be surprised by and remark upon these coincidences when they happen, but it's all too tempting to jump to the conclusion that there must be some other explanation than pure chance. This kind of error is called a type I error in statistics. For those who are biased in favour of certain explanations, whether religious, supernatural, paranormal or pseudoscientific, type I errors are particularly common. Often, the supposed explanation is more preposterous that the observations are the result of chance.

For example, if a vague image of something that could be construed as looking like a bearded face appears on a piece of burnt toast, one explanation is that the facial resemblance is coincidental. Another explanation is that the coincidence is too improbable, ergo this must be a manifestation of Jesus Christ performing a miracle in order to demonstrate his existence to us.

This is irrational on a number of levels. It is improbable that Jesus has a contemporary existence. Even given the existence of Jesus, it is surely improbable that he would manifest in such a way. Whatever the probability that the pattern could have arisen by chance, the probability that this is a true divine manifestation is orders of magnitude more remote.

However impressive the coincidences might be that are experienced in the typical life, how much more incredible must be the greatest of the coincidences experienced in the world at large. Given the tremendous number of people living in the world, we should expect some truly impressive coincidences to occur. As such, skeptics will generally not believe extraordinary claims based on anecdotes which can be ascribed to coincidence.

Occasional coincidences happening to strangers is what we should predict and it is what we observe. If a coincidence has occurred to a stranger (and particularly if this stranger is only brought to our attention because of the coincidence), we should not be surprised unless the event is so improbable that nothing so impressive should be likely to happen to anybody in the entire population of the planet within the time frame of interest.

Surprising coincidences and type II errors

In statistics, a type II error is where a real phenomenon is dismissed due to insufficient evidence. It may be that we assume that any evidence in support of the phenomenon is a result of coincidence. An over-eagerness to ascribe remarkable occurrences to coincidence may make type II errors more common. The fact that skeptics like me are educated enough to realise that coincidences are inevitable may mean that some of us are more prone to these types of errors.

Again, the reason that a coincidental event is brought to our attention and how large the "category" that describes the person who experienced it affects how impressive a coincidence needs to be before we find it remarkable. We should expect to find more impressive coincidental anecdotes reported by the set of people who are strangers than by the set of people who are acquaintances. We should expect acquaintances to report more impressive coincidences than family, and we should expect more impressive coincidences to happen to family members than happen to us. It's a numbers game. The larger your sample size, the more impressive the most impressive coincidence is likely to be. If someone is brought to your attention specifically because of a coincidence, then the set of people you need to consider is the set of all people who might have been brought to your attention if this coincidence had happened to them.

This explains why we should be surprised to find that we have won the lottery, but less surprised to hear that a complete stranger has won. This does not represent a logical error, mistaking our subjective view for the objective one which holds all people equally likely to win. The odds of any individual person winning are remote (sample size of one), but the odds of anybody winning are relatively high (sample size of millions).

If we have a set of people we care about, and a set of people we don't care about, the set of people we care about is surely many orders of magnitude smaller. As such, it really is a cause for rational surprise if one of the people we care about wins the lottery.

In my next post, I will try to tie these thoughts into answering the question posed in my last one.

No comments:

Post a Comment