How the tech boom is bad for innovation

By Felix Salmon

October 14, 2011

Jon Stokes has a fascinating column making a credible case that the VC and tech bubble is hampering development of the cloud. "
data-share-img=""
data-share="twitter,facebook,linkedin,reddit,google,mail"
data-share-count="false">

Tags:

Jon Stokes has a fascinating column making a credible case that the VC and tech bubble is hampering development of the cloud.

I recently had a sit-down chat with Ping Li, a venture capitalist at Accel Partners who does investments across the layers of the cloud stack… he explained that the talent shortage is stifling fundamental innovation in the cloud space.

To do really fundamental engineering innovation of the kind that was done, say, in the early days of Google and VMware, you need to hire and retain teams of talented engineers. But in today’s go-go funding environment, top engineers are being enticed with truckloads of money to break off and form two- and three-person startups. This phenomenon, explains Li, is why “many of the really big innovations happen in less frothy times.” He did go on to clarify that “some great companies do get created in these times (like Amazon in the last bubble). It’s just harder given talent shortage.”

I asked Jon if this meant that real innovation in the cloud requires pretty big teams, and can’t be done by smaller startups. And the answer there is absolutely yes — if you’re looking for something huge like Amazon’s AWS, which required the full focus of Amazon’s large technical staff over a multi-year timeframe. Scalable websites can, thanks to Amazon’s cloud, now be launched with a handful of employees. But to develop the cloud itself takes serious resources — to the point at which it’s now conventional wisdom that you need to be Amazon, Apple, Google, or Microsoft to even play in that game. And even they’re starving for talent.

There’s the email I got a few months ago from a friend of mine and product manager at Apple, who was wondering if I knew any cloud computing hackers that they could hire. When we get to the point where Apple product managers on the client side are reaching out to their personal networks in search of cloud coding talent for the world’s largest tech company, you know it’s bad out there.

Jon frames the problem as one of supply and demand:

The current crisis in the cloud is the product of too many dollars and transistors chasing too few coders and sysadmins. It will take a while for the latter to catch up with the former… unless, of course, another major downturn strikes. It seems ironic that less money could equal more innovation, but it wouldn’t be the first time that a wave of downsizing and tight money boosted productivity.

I asked him whether looser skilled-immigration policies might help, and he said probably not:

I think that the root problem isn’t one of geography–it’s that this stuff is happening so fast (i.e. Moore’s Law and my cheap transistors argument) that the hardware build-out is outstripping programmer education. And by “programmer education” I mean not only the number of programmers being trained in aggregate across the world, but also programming as a discipline’s ability to empower ordinary mortal to develop and deploy software on these massively parallel systems. The cloud has to be “de-ninjafied”, so to speak. Getting max productivity out of the cloud has to be brought within the grasp of non-ninjas, the way that, say, VisualBasic from MSFT brought building a relatively complex custom relational database application within the grasp of the average local technical college graduate.

This rings true to me. The cloud is not located in any particular country, and if there were great engineers who could be hired to work on it from Beijing or Bangalore, I’m sure that Apple and Amazon are more than capable of doing that. What needs to be done here is basically cloud-development grunt work: taking a young and complex technology, and building the tools which can bring it to the masses. There’s not a lot of glory in that — while companies which live in the cloud, like Airbnb, can find themselves worth billions, the engineers who work on the cloud are more like the utility workers of the internet. And it’s easy to see why they might be finding more attractive opportunities, right now, elsewhere.

Here’s how Jon puts it:

In order to move the cloud itself forward in a major way by solving large batches of related fundamental technical problems you need longer timeframes. You can fiddle around in the guts of the cloud, smoothing out this and optimizing that, and adding features and bells and whistles. But to do the big projects, you need time.

Now, there are shorter-term innovations that can and will get done in the cloud, so VCs have plenty to fund. But to shift the tectonic plates, you need time and resources.

This isn’t like sustainable energy — it’s not something that the government can or should be stepping in to fund. More money pouring into the tech space would only exacerbate the current problems.

There’s a case to be made that AWS is the result of what happened when Amazon, after the dot-com bust, found itself with an unusual degree of access to the time and talent of a large number of engineers. The cloud is young; it could do with a lot more development along those lines. But as Jon says, we’re unlikely to see such fundamental evolution in cloud architecture for a while. Because for the time being, smaller, lighter, and riskier projects look much more attractive.