Survivorship bias

Survivorship bias is one of the most common selection biases, and can be extremely detrimental when assessing whether a new idea or innovation will work.

Survivorship bias is the tendency for humans to be aware of an focus on thins (be it people or products) which made it past some selection process, and not pay attention to the other examples which did not make it through.

Essentially, survivorship bias says that we focus on the success stories and ignore all the other attempts which did not work.

One of the most important examples of survivorship bias was unearthed by Abraham Wald, when looking at bullet holes on returning military aircraft which had been shot at by enemies.

This analysis resulted in the following mapping of damage and bullet holes on the returning aircraft:

The question was then, how do we use this data to make the planes safer to fly, by adding armour where it was needed to ensure that they were more likely to return?

The obvious answer that most people would come up with is to add armour to the parts of the plane which seem to be shot the most (in this case the wings and tail).

What would you have said?

What most people don’t realise though is that they have chosen to reinforce the parts of the plane which could be studied, because they managed to return.

There were many more planes which were shot, and did not return. These pilots were the ones who suffered the most and often lost their lives.

So looking at the image again, is there a pattern to the parts of the plane which were not full of bullet holes, which may have allowed the pilots to return home safely? If you look at the image, the answer is yes, in the two motors and cockpit which have almost no damage. This allowed those pilots to survive.

So while it may go against our first assumption, the right place to add armour was actually where no damage was seen in the survivors. By reinforcing the engines and cockpit, you make it more likely the pilot will survive.

This mindset can also be extremely damaging when assessing or developing new innovative ideas, for two main reasons:

Comparing to what has previously survived (the Status Quo): Many new ideas fail to make it past review stages, especially when needing funding to scale, because the decision makers will compare this new, risky innovation to what they know works (what has previously survived). These decision makers will then see the new innovation as risky because of how different it is.
Looking for examples of other innovations (and people) which have succeeded and ignoring those who tried the same thing but failed: Facebook and Microsoft both have founders who dropped out of college to develop their software companies, and became billionaires as a result. But can we name any of the thousands of CEOs who were inspired by these stories, also decided to drop out of college, but couldn’t scale their own software companies? No we can’t, because their stories are not well known. Yet they are much more likely to happen than the small number of success stories. Don’t confuse correlation (dropped out of college and then because successful) and causation (were successful because they dropped out of college).

it is important to be aware of survivorship bias, and to always ask ourself whether the success stories we know of were successful because they really were better than everyone else, or also because:

They are actually in the minority, and we don’t know examples of all the failures
They had more luck than others who were trying the same thing
There is a reason why their success is more well known (for example, they had a team of producers and publishers to make their story famous)
They were in the right place at the right time
We are confusing correlation and causation

Be aware of your biases, and they will help you make better decisions when developing your own new ideas.

Bio
Latest Posts

Nick Skillicorn

Chief Editor, Founder & CEO at Improvides Innovation Consulting

Creativity & Innovation expert: I help individuals and companies build their creativity and innovation capabilities, so you can develop the next breakthrough idea which customers love. Chief Editor of Ideatovalue.com and Founder / CEO of Improvides Innovation Consulting. Coach / Speaker / Author / TEDx Speaker / Voted as one of the most influential innovation bloggers.