Statistical Validation of Disruption Theory

There has been a bit of discussion on the web about the validity of Clayton Christensen’s Disruption theory. I hope this article by Thomas Thurston will put it all to rest; “Christensen Vs. Lepore: A Matter Of Fact”.

Most people don’t know this, but it turns out Disruption Theory is the foundation of the most accurate, thoroughly vetted, quantitative prediction models of new business survival or failure in the world today.

I did my best to reduce his theory to falsifiable yes/no logic using published research. Even so, in the first round these relatively crude rules based on Disruption Theory blindly predicted if new businesses would survive or fail with 94 percent accuracy and over 99 percent statistical confidence. Holy crap.

So statistically, disruption theory is pretty good a predictions.

How often does it get it wrong?

A lot of people point to examples of when Disruption Theory, or Christensen, was wrong. It was wrong about the iPhone. Tesla. Ralph Lauren. In fact, it’s been wrong over 7,500 times by my count (remember it has a 33 percent error rate when predicting winners). Keep in mind, however, it’s 66 percent right while everything else is stuck at 25 percent. Improvement, not perfection, is the standard.

Well, it doesn’t get it right 100% of the time. Does that mean the theory isn’t valid? Not at all.

Many bloggers dismissed or attempted to modify Disruption Theory because it got Apple wrong. These people don’t understand statistics or don’t understand how to use statistics to validate or invalidate an argument. I could list them here, but I won’t.

The method used here is very standard. For example, every new drug that comes to the market is tested for effectiveness using the same statistical methods. Drugs sometimes work, but often they don’t. All they have to do is work a bit better, a bit more often. If we rejected every drug that ever failed to work for certain patients, there would be no drugs.

I especially like Thomas’ closing sentence. It’s the standard way of statistically testing which theory is correct, which Thomas uses to test Lepore’s theory against Clay’s. Anybody who doesn’t fully get it shouldn’t be talking about how theories are validated.

Lepore could be right about Disruption Theory, but the odds are literally over 500,000 times greater that, as a matter of fact, she’s just plain wrong.

P.S.

Although I am appalled that a large number of people who discussed Disruption Theory on the web didn’t seem to understand the basic principles of statistical testing, I understand where they came from.

Statistics simply isn’t taught enough at schools. Statistics, in my view, is one of the most important branches of mathematics and is relevant even for people who won’t touch maths ever in their professional careers. We have to understand statistics so that we, in democratic nations, can correctly assess the accuracy of claims made by politicians, and so much more.

This is a real shame.