Survivorship Bias and It's Impact On Consultants
It's all about the denominator...
If you’re a consultant, it’s likely at some time or another you’ve fallen victim to survivorship bias.
Survivorship bias is the fallacy of only paying attention to the survivors instead of the participants in any given activity.
For example, 80% of restaurants fail and quietly disappear. Yet, because most people only hear about success stories, they feel their odds of success are much higher.
It’s important consultants understand how survivorship bias can impact both their recommendations for clients and how they try to build their own practice.
The Three Problems With Survivorship Bias
You don’t have to look far to see people sharing stories of personal success and how they achieved it. What you will rarely hear about in these stories is how many people failed repeating exactly what they did.
The problem with these success stories is three-fold:
Those who succeed attribute their success to skill as opposed to luck. Sometimes people just get lucky. They are the right person at the right time and in the right place. They can succeed while someone else in a slightly different time and place fails.
Those who succeed struggle to identify the causes of their success. They will struggle to figure out whether they succeeded in spite of what they did or because of what they did. What did they do that people who failed didn’t do? What was their stroke of genius or insight which others didn’t have? Are they even aware of the failures?
It distorts the perceived odds of success. Because the people who succeed are eager to discuss their successes (and those who fail typically don’t), we only hear about the successes. Worse yet, the number of successful people will grow over time creating further proof that this is a good idea. This is probably the biggest problem of the three. Even really bad ideas often have successful examples.
We can perhaps best illustrate this with dropshippers.
Why Dropshipping Is A Terrible Business
Over a decade ago, Tim Ferriss inspires thousands (potentially millions) of people to start dropshipping businesses.
The idea is simple enough. You create a website to sell a product. You run ads or use clever marketing to attract customers. They decide to buy from you and you process the orders. The magic is what happens next. You don’t keep any stock on hand yourself. Instead, that order goes to a fulfilment company that ships the product to the customer. You charge between 15% to 20% markup in the process.
The appeal of dropshipping is obvious. You can create an automated system. You can process the orders on a beach and watch the cash roll in. Hence the four-hour workweek.
The problem is also obvious. Why wouldn’t the company just purchase the product directly and cut you out of the process? Either the customer can earn a 15% - 20% discount or the fulfilment company can earn higher margins.
Successful dropshippers claim the additional margin reflects the increased value they offer like better service. The truth is once customers realise they can buy directly, they do it (and better customer service is only really relevant for high-frequency buyers).
Yet because the idea of dropshipping is so alluring and there are a large number of people who can share their success stories, you can find conferences packed to the rafters trying to pursue an idea which will almost certainly fail.
So, what’s happening?
Illustrating The Survivorship Bias
It’s best to illustrate the problem using this diagram.
Let’s imagine 10 people try dropshipping and two succeed. Those two people will talk about it and inspire others. Maybe 15% of these followers will succeed. Now there are four people sharing their success, even though the odds of success have dropped.
So the number of successes grows - typically while the probability of success shrinks.
(Aside, the other problem is both successes and failures are prone to fib about their level of success).
Do You Fall Victim To Common Examples Of Survivorship Bias?
When we’re more familiar with the success rate, we can provide good advice. If a friend told us they intended to become a professional actor or musician, we might be polite and encourage them to give it a go…but have something to fall back on. Or we might just say that’s a terrible idea and they will almost certainly fail.
Most of us know about survivorship bias, but we still feel our odds of success are higher than the average person. For example…
Many of us have started blogs and dream of gaining an audience of thousands. Yet the number of blogs which attract more than a few dozen readers is close to 1%. That’s a 99% failure rate (if this blog attracts 1k readers, that would be a remarkable achievement).
Many of us have started YouTube channels. Yet, the odds of attracting more than 1k subscribers is around 15%. That’s an 85% failure rate!
And if you’re planning to write a book, you should expect to sell a few hundred copies at most. That’s it.
A friend of mine who published a book two years ago. He estimated, given the size of his audience, relationships, and reach, he would sell around 10k books. I estimated he would sell about 700 to 800 copies.
When I last checked in, he had sold closer to 400 copies (including 100 copies in a bulk deal).
(Another aside, the odds of succeeding as a consultant are no different. Depending on how you define success (and let’s, for now, define success as matching what you earned in your previous role), your odds are probably around 20% to 30%.)
In the moment, it’s really hard to think about the odds of success instead of the examples of success you’ve seen.
Would You Recommend A Strategy/Tactic With A 80% Failure Rate?
This is equally important to consider when we make suggestions to clients. we make suggestions and recommendations.
We hear about a tactic or idea that worked well and then try or recommend it to clients. The key step is to understand the denominator.
The success story you’re hearing about is set against how many failures?
Do you even look for failures? If a tactic or strategy had a known 80% failure rate would you recommend it to a client?
Probably not.
But unless you’re looking at the odds of success instead of the number of successes there’s a very strong chance you’ve done that. In my niche, I can think of numerous examples of tactics which almost always fail, but where people can point to successful examples.
Before recommending a tactic or strategy, you need to have some idea of the odds of success. How many organisations have tried this before? What is the success rate? Did the successes do anything differently from the failures?
How To Defy The Odds
Most people who know the odds think they will defy the odds. The reality is they won’t. The majority of us are average (or very close to average). But that doesn’t mean it’s impossible to defy the odds.
There are very specific things you can do to defy the odds. But none of these things means trying harder or believing your minor advantages are game-changers (they probably aren’t).
There are several ways to defy the odds in almost any situation
Leverage a unique advantage that others can’t match.
For consultants, having a mailing list of 20k+ people or strong relationships with senior leaders at major companies is an absolute game-changer. Another game-changer might be money. If you can invest more in your website, content, and host unique in-person events, that could also be a game-changer.
Your leverage must a) very few people have and b) immediately translates into more business.
Likewise, if you’re embracing a tactic or recommending a strategy to a client which has a high failure rate, what is the unique advantage they will leverage to succeed where most brands and organisations fail? This must be so obviously an advantage that matters that few would dispute it.
Don’t be average.
The denominator is always the sum of the average. If you’re not approaching the action in an average way, the odds might be very different. Most blogs might fail, but what if you write really short posts people can consume like a snack or far longer posts going into more depth than anyone else? What if you did videos or podcasts in a totally different way? What if you dive deeper into an area that anyone has ever dared to before? Then the odds become unknowable (which may be better or worse than bad).
This isn’t as simple as ‘find your niche’. Finding a smaller topic within a niche doesn’t massively increase your odds of success. But approaching the whole activity in a completely different way changes the odds. Miss Excel is one of my favourite examples. Not many people can dance while giving Excel tips to make a living, but it definitely changes the odds.
Increase your attempts
You might not be able to change the denominator, but you can change the numerator. The more shots you take, the more likely one will succeed. Even a slim chance of success tilts in your favour with every unique attempt you make. Better yet, if you learn from each failure, the odds of succeeding at the next attempt improve significantly. Persistence, learning over time, and multiple attempts helps you defy the odds.
Fight The Urge To Follow The Herd
Ultimately, it’s incredibly hard to recommend against a popular idea because the odds of success are low. Even when a common sense check knows it’s a bad idea (dropshipping!), people still pursue it with gusto. The allure of it and the mental visualisation of what success would mean are powerful.
Yet, recommending against following a trend is one of the most important things a consultant can do. A consultant is being paid to take an analytical approach to a situation and fight precisely these kinds of biases. It might not always be a popular service, but it’s an invaluable one.