Fail Early, Fail Often, Succeed

One of the mantras of the lean startup movement - and, as I have written before, of the agile software development movement - is "fail early, fail often." The logic works something like this: you never know for sure, in advance, when and how you will fail, but you know you will make some mistakes, perhaps serious ones, and have some failures, perhaps major ones. Instead of building around getting it all right the first time across all parts, find ways to test, try and succeed or fail in very small ways very often. The more you make small mistakes or failures and correct them, the more resilient the whole system will be, the quicker (and cheaper) it will be to get to big successes.

Amazon Web Services (and Google's infrastructure) are designed around a similar principle, but across space, rather than time. Rather than building a huge complex and very expensive system that will reduce the probability of failure to infinitesimal (but still not zero), build around the ability to handle failures in many small cheap parts, survive them, replace them, and move on.

I had the opportunity this week to hear two stories that confirmed how crucial this skill is.

The Pharma Startup

A friend of mine is the CEO of a pharma startup, developing new drugs for use in humans. As most people know by now, once the drug appears to be useful, safe and effective in theory and simulations, there are ever-increasing tiers of testing to get to acceptance for human usage. The first stage is in vitro, i.e. test tube experiments, followed by in vivo, or "in life", i.e. in live animals, followed by higher-order animals (normally simians biologically similar to humans), then humans.

This pharma has a promising drug that was showing good initial results in vitro, or the first stage. In vitro testing was far from complete, would likely take another $1MM to get there. Nonetheless, the CEO pushed, against the desires of the executive team and the Board, to immediately move to the next stage of in vivo testing.

Why, when the first stage isn't complete, would he want to move to the next stage? After all, wouldn't it mean wasting some money and later coming back to the first stage to complete testing?

His answer: fail early, fail often.

Neither he nor anyone at the company knew if second stage testing would show any results. First stage could look great, second stage could fizzle, or it could also look great. But he wanted to get some feedback on the second stage before spending $1MM more on the first stage. If it is going to fail in second stage, let's find out right now. And so, against everyone's wishes, he pushed to start second stage.

It failed. Second stage testing showed absolutely no benefit! It convinced the company to shut down that drug in the pipeline.

Put in other terms, the CEO saved the company $1MM! This is a war chest to live and fight another day; he probably saved the company.

The Cultural Startup

A friend of mine who lives in a country in Asia recently left a startup. I asked him what his biggest challenge was when running engineering at the startup. I expected to hear about sales & marketing, or perhaps talent knowledge and education, or project management. Nope. His number one challenge was hiring people who were willing to fail. In this location, although it was hard enough to find talented, skilled and smart people, the culture was so risk-averse and so strongly mitigated against failure, that people were unwilling to work on anything that might fail.

Now, this isn't just big failure. This VP told me of cases where they actually followed proper "fail early, fail often". They would come up with a new feature or design, and rather than throwing lots of resources over an extended period of time, only to see it fail, they would take 1-2 people for 2 weeks, whip up a basic version, and then A/B test it on the site. Sometimes, you would get positive results; others, the new feature or product would be neutral or even negative.

Either way, this is a success. You discover what works and what doesn't at minimal cost.

However, the engineers and designers who worked on these products felt it was a failure. "I worked on this, and no one wanted it, and we cancelled it, so I am part of a failure." The brightest engineer with this type of mindset will be hard-pressed to succeed at a startup.

In startups especially, but in all businesses, failures can be costly, yet failures are all but guaranteed. "Fail early, fail often" is the best way to minimize the cost of failure and get to success.