The Survivorship Bias (sometimes styled Survivor Bias) is the human tendency to value the seen or available at the expense of the unseen.
Data, groups, things, and people that exist today receive the bulk of our attention. We discount and ignore historical things or people or failures.
Survivorship Bias is often a form of selection bias, and sometimes further of cherry picking.
Survivorship Bias, War, and Abraham Wald's Observation
In one of the most famous studies designed to counter survivorship bias, Abraham Wald set out to determine the weaknesses of airplanes in World War II for the Center of Naval Analyses' Statistical Research Group (SRG). His report, "A Method of Estimating Plane Vulnerability Based on Damage of Survivors" (PDF reprint here) is a clever way of reacting to incomplete and missing data on aircraft survival.
Noting the quantity and location of damage to aircraft that returned from battle, his colleagues had suggested adding armor in those areas. Wald pointed out that they were only seeing the surviving aircraft.
All of the downed aircraft weren't being studied - the damage they were seeing was, therefore, damage that the planes could survive:
Contra his coworkers, any additional armor or engineering should focus on the areas of the planes not showing damage. Those were the areas that couldn't sustain a hit.
Survivorship Bias and Your Finances and Career
Survivorship bias also rears its ugly head in finance and business. Because of the nature of data, it's easier to find records of things that exist than no longer exist. You need to be vigilant and watch out for situations where survivorship bias skews the data.
Survivorship Bias in Indexing
Consider an equity index such as the S&P 500. If you aggregated all of the current members of the S&P 500, you'd find something curious - that they outperformed the index itself.
Think about this for a second though... of course it should be the case. Indices can only include companies that currently exist but have the burden of the performance of companies that no longer do. There is a minimum trading price on many indices as well - indices literally remove companies for poor performance.
When companies fall from the index their past contribution to the index isn't scrubbed. Poor past performance remains baked into an index's history. When a new company joins the index, there's no provision to backdate statistics.
Indices only benefit from a new firm going forward.
And that's all it takes for that phenomenon to prove true - a bit of memory combined with the effects of poorly performing and successful companies. When you are aggregating performance outside an index, take care to include performance data from previously existing firms as well.
Survivorship Bias in Investing
Viewing a friend's portfolio holdings or an investment company's list of funds usually shows a similar effect. What's presented to you, an individual investor, is a snapshot of successful picks and funds. What you see is often beating some index's performance over a fairly long period.
Yet again, we see the survivorship bias at work. Your friend may have sold poorly performing picks, leaving only long-running market beaters. Financial companies continuously cull or merge poorly performing funds and syndicates, leaving only high flyers with high annual returns around for their sales pitches.
Don't get caught up in hyped market-beating picks and funds or sales pitches and bragging sessions. The truth is, you are witnessing only a snapshot... poor performers left the sales deck a long time ago. You can't invest in a bad fund from the 1990s, but you'll surely see a few funds with long histories that did perform well.
And how much is the bias? In 1996, Elton, Gruber, and Blake studied the survivorship bias that comes from the attrition of mutual funds [PDF]. They found a 0.9% a year bias due to the lack of effect of missing under-performers. Remember, that's a year... the effect compounds for longer time periods.
Survivorship Bias in the Workforce and Business
Not to pick on Business Insider, but their article concluding valedictorians are less likely to become millionaires because millionaires in some survey had a 2.9 average GPA finely illustrates the bias.
Ignore whether it's true - you can't compare 81 valedictorians in a longitudinal study to millionaires ex post facto.
Valedictorians are a reasonably well distributed sample - if not perfectly random. However, the millionaires are already millionaires - working backwards from outliers causes the post to draw the incorrect conclusions. While the authors found the valedictorians were very high performers, the other referenced study doesn't prove C students become millionaires more often.
A proper comparison would take a selection of students with a 2.9 average GPA and track outcomes.
Don't believe me? Consider the lottery.
If I interviewed 500 lottery winners, one common factor will show up - everyone in my sample gambled. If I'm not careful, I'll believe that the path to wealth leads through a lottery habit. Because of poor sampling, I wouldn't see the major downsides of the lottery.
And downsides to the lottery are legion. To wit: financial woes, marriage troubles, ruined friendships - you'd miss all those downfalls if you only looked at lottery winners.
Beware the Survivorship Bias
The Survivorship Bias also has effects on humanity. Genes propagate when people survive to reproductive age. Nature exhibits its own survivorship bias towards the fecund and the strong - and not necessarily the most intelligent. (As so brilliantly lampooned in Idiocracy.)
And even if that weren't the case (yes, we're sure it is), how would we know?
Only survivors write, so yes: history is written by the victors.
- What knowledge have we lost to history?
- How many profound (yet lost) insights into the human condition have humans made and forgotten?
- What commonly known events do we have all wrong?
Diogenes and Survivorship Bias
Diagoras approached the philosopher Diogenes with a painting of survivors of a shipwreck. "Look, you who think the Gods have no care of human things, what do you say to so many persons preserved from death by their especial favour?"
Diogenes didn't miss a beat. "Why, I say that their pictures are not here who were cast away, which were by much the greater number."
Indeed. As shipwreck survivor paintings only show survivors and returning aircraft only show survivable damage - you must be very careful with the survivorship bias.
