Whence Serverless Cloud? It's About the Market.

By Avi Deitcher

I love tech. Despite an MBA and a decade of consulting and running a start-up or two, deep down, I always will be an engineer.

One of the most important lessons I learned as a young engineer 20 years ago at Morgan Stanley - courtesy of Guy Chiarello - is that the technology is only the means, not the end. Understand the finances, the market, even the politics if you want to do something with technology, even just inside a company, let alone outside.

What does all of this have to do with serverless cloud?

Several months ago, Joe Emison published an article in New Stack called, "How Amazon Web Services Isn't Winning." I am not sure I agree with the premise - and the headline probably is intended to be provocative - but he had some interesting thoughts that have aligned with my extensive use of cloud services, both on behalf of clients as well as in my own ventures.

Let's start with the obvious: AWS is the elephant in the room. Just as in the 90s everyone wanted to go after Microsoft, in the second decade of the 21st century, AWS is the dominant player, larger than everyone else combined, and many are going after them.

The major direct competitors - Google Compute and Microsoft Azure - are competing head-on. The smaller players, those without the massive cash to spend on infrastructure and marketing necessary to compete directly, are aiming for areas wherein AWS cannot compete due to business model or structure.

Competitors are using two primary avenues of attack:

  1. Independence
  2. Abstraction

Independence

Running any part of AWS requires the entire AWS. It isn't possible to run your own EC2 cloud, and I am skeptical Amazon would (for business model reasons) or could (for architectural reasons) offer a "run it yourself" version of EC2, S3 or any other major component.

This means you are tied to Amazon through and through. Rumours have abounded for some time that despite (or perhaps because of) the major engineering Amazon's most well-known customer has performed to run in AWS, Netflix would love to get out of AWS... if it could. The sheer scale of Netflix gives it much better pricing from Amazon, but also, likely, sufficient scale to go it alone.

If you do not want to run in AWS, whether today because you do not trust them as a competitor - how likely is Barnes & Noble or any retailer to want to depend on Amazon? - or because you want to hedge your bets, you are stuck. It matters not if you want to start in AWS and then move out, or start day one on your own AWS-services cloud, it simply isn't available.

I do not address whether or not running it on your own infrastructure makes sense, and what the market will look like. For that, I defer to much wiser cloud visionaries like Simon Wardley. I simply address what companies who want to be independent have available.

Enter companies like Joyent. First, are they not in the retail space, and thus not a competitor in one division (amazon.com) for the services of a customer in another (aws.amazon.com). Second, and more importantly, they offer the ability to run their entire cloud service - IaaS, Storage, Process-on-Demand - on their servers or yours, or even on AWS!

Actually, the entire Joyent stack is open source, meaning you could run it yourself without using them at all. If they go belly-up, everything is there for the taking.

I cannot predict if this approach is sufficient to up-end AWS, or draw sufficiently large market. The important point, though, is that they are attacking from a direction that Amazon is unlikely to be able to respond: complete independence.

Abstraction

The entire AWS model is built around an array of complex interconnected services. They assume you need servers for compute, S3 or EBS for storage, IAM for permissions control, etc.

Amazon did this by, as Emison says, circumventing the IT people and going straight to the developers.

In 2008, Amazon went after the data center and the IT staff who controlled machine and VM deployment, and allowed developers to do all of those jobs, using APIs.

in Wardley's terms, Amazon saw that developers depended on IT people for providing a product - servers and storage - further down the value chain. Amazon made it into a utility, available on-demand, and bypassed the people who made a living from deploying and managing the product.

However, Amazon really went after the server software developers. These people are great... but needed only to provide outputs, primarily to front-end interfaces. The output might go through multiple stages, but in the end, the purpose is to provide a real person with real functionality.

Enter the "serverless" offerings, primarily Parse (owned by Facebook) and Firebase (owned by Google), as well as Manta (Joyent) and, now, Lambda (AWS). These offerings go after the next layer up, eliminating one more element of complexity.

Look at the stack of offerings we used to manage on our own, and where they went. I begin with the bottom:

  1. Real-estate ➝ Colocation providers
  2. Power & cooling ➝ Hosting providers
  3. Servers & storage ➝ IaaS
  4. Server processes ➝ ??

Amazon entered to "utilify" (as in, turn into a utility) #3 above. Serverless aims to utilify #4. One can make an argument - I am convinced it has some real merit - that #4 will not be fully abstracted. Indeed, even with AWS, GCE, Azure, plenty of companies still use Rackspace and Equinix. But it definitely will be, and already is, abstracted in many use cases. Amazon's decision to offer Lambda is proof of that.

Is It Enough?

As with the Independence avenue of attack, one can ask whether the Abstraction avenue is sufficient to undermine Amazon's dominance. I do not have a crystal ball to the future. I do think that Amazon's "weave lots of complicated parts together" approach to serverless computing demonstrated in Joe's article, reflecting their roots as "data centre via API", combined with their non-container-native approach to containers (Docker on EC2 instances you have to spin up first) and the heaviness of IAM (built for corporate employee permissions), show a potential Achilles heel.

Summary

The overriding factor for your offering isn't the technology, or even the legacy, but who you believe your customers are and how you offer them. Corporate hardware and colocation providers are hurt by AWS not because Amazon had better technology, but because Amazon found the true customers of those servers one layer up and made the lower level less relevant.

Would you rather be selling the best technology to the customer? Or the one that makes those customers irrelevant?

The inevitable questions are:

  1. Are those processes are needed as well, or can be offered as a service?
  2. Is Amazon financially, architecturally and customer-wise oriented to dominate here?