Humans have an evolutionary incentive to recognize patterns. The same mechanisms that helped early humans avoid snakes and other predators are still part of our constitution today. One of our in-built biases is the human tendency to see patterns where none exist. Clustering illusion is our bias towards under-predicting the variability of random – or near-random – data.
Clustering Illusion and Pattern Recognition
It's not that patterns are wrong, or don't exist. If you, for example, noticed a traffic stop backs up daily around the same time, you can save a significant amount of time on an alternate route.
Every part of our lives is subject to ingrained patterns and pre-existing habits that can benefit – or harm – us. Finding patterns where they exist can significantly help your happiness, dating life, finances, health, education, weight-loss, and many other things.
For all those reasons and more, it's a good thing to be sensitive to patterns in life.
Of course... we may overdo it in many cases. Due to this inbuilt tendency, humans recognize patterns using data that has no underlying correlation. In these cases, we fool ourselves that there is a relationship where one does not exist.
Clustering Illusion Examples
Humans have a tendency to recognize patterns in frivolous details that don't actually exist.
Take celebrity deaths. In the United States there is a meme that "Celebrity deaths happen in threes."
Of course, celebrity deaths are (unfortunately) common. They're so common, in fact, we almost become numb to them - it's only when there is a cluster that we remember, and start to put together spurious conclusions.
The hot-hand fallacy and the gambler's fallacy are two closely linked fallacies. In both, you make predictive or causative assumptions from what are, effectively, random events.
With the gambler's fallacy, you believe past random events can predict future future random events. For example, if you roll black eight times straight in roulette, you might believe the next one will be black again. Truth is, the next roll is completely divorced from the history of the previous eight.
Countering the Fallacies
How do you counteract these fallacies?
Always maintain a healthy dose of skepticism. You should assume that, generally, there is no correlation between two events. While you might change that view, make sure the evidence is robust – be rigorous!
Since you don't physically have the time to research every subject this is quite the lift. Finding the balance between a healthy skepticism and your pattern recognition is hard, but it's something you'll want to be mindful you're doing.
Finding No Relationship Can Help Defeat Clustering Illusion
Some websites famously point to fake correlations in a humorous way. Today we're going to sing the praises of finding no evidence.
Truth is, we should equally be excited about not finding a relationship between events.
(That's not very sexy though. Imagine the headline "Data Still Noisy on Caffeine's Impact on Anxiety". )
The other issue: remaining skeptical might make you appear non-committal or lack confidence. Sometimes, when you are skeptical of negative information, denial looks like the Pollyanna principle or the Ostrich effect.
The best thing you can do? Be skeptical, yet be flexible in your assumptions. Don't ever be too committed to one way of thought.
As the saying goes: have strong opinions, loosely held.
