Does Ouya point to a Kickstarter tech bubble?

I recently expressed great scepticism over the massively funded Ouya game console project, which has grabbed more than $5 million (£3.2 million) in donations so far.

In my column, I said Kickstarter is a lousy way to pre-sell consumer technology products because the overall rate of failure in consumer tech is just way too high. The outcry from Kickstarter fans made me want to look much deeper into the issue.

The projects on Kickstarter seem awesome. I want to see these films. I want to play these games. I want to watch these shows. So I think it's critically relevant to ask whether the games, films, dances and shows actually get made.

I decided to put my research where my mouth is and studied 312 of Kickstarter's "most funded" projects so far, the 24 most funded projects in each one of Kickstarter's 13 categories. (Ouya isn't among them, because its Kickstarter hasn't ended yet.)

Figuring out whether the projects have actually accomplished their goals is more difficult than it seems. Some creators stop providing updates on Kickstarter as soon as they get the money. In those cases, I read the Kickstarter comments for the project and did web and shopping searches for products or events to see if they ever went on sale. I tried to stay strict on this: A project only counted as "delivered" if a final product was in a backer's hands (or a performance carried out and performed) by the 19th of July, 2012. Pre-orders and promises don't count.

In this I'm following up the excellent work done recently by AppsBlogger and Ethan Mollick from the University of Pennsylvania, who estimated together (in a must-read article full of compelling stats) that only 25 per cent of design and technology projects delivered on time, although 75 per cent would complete after an eight-month delay.

I wish I could make you guys an infographic, but frankly I have no idea how to make an infographic. I can, however, make extremely bad Excel charts, so that's what you’re going to get.

Here's what I've discovered.

Quick summary of main points:

Design, Games and Technology tend to have far bigger dollar amounts on their top projects than other Kickstarter categories.

A lot of that money just suddenly flooded in recently.

Older projects have a good record of success.

But the $500,000+ mega-projects are so much bigger than the average Kickstarter project that they seem like a completely different class of business.

These mega-projects don't have much of a track record yet.

A few major projects distort Kickstarter's funding curve

The vast majority of Kickstarter projects, even the most funded ones, are pretty modest. Among my 312 "most funded" projects, the median cash involved was $61,150 (£39,300).

The scale you need to work on is different, too. Once you’re talking millions, you get beyond the point where you can hand-check every device, and to the level where you need a reliable supply chain. Tech project Pen Type-A has had trouble delivering its 4,000 units. Ouya has more than 40,000 backers/pre-purchasers so far.

Furthermore, a huge individual project can really drive the public perception of Kickstarter. Of the 30,900 Google News hits for "kickstarter" in the past month, a third have mentioned Ouya. That puts Kickstarter's reputation on the line with these big gambles.

“Geek” projects involve a lot more money than others

Kickstarter's three "geek" categories – Games, Design, and Tech – attract a lot more money than the other categories. I'm just going to summarise this one as a chart:

AppsBlogger brings another useful data point: Of the 33 large projects it found that received over 10 times their respective funding goal, all except one were in the geek categories. (That other one was an Amanda Palmer album, and she and her husband Neil Gaiman are epic online celebrities.)

Kickstarter is a mature funding source for art, but not yet for tech

Of the 24 most funded projects in Art, 16 were funded before 2012. Of the 24 most funded projects in Dance, it's 12. In Music, 12. Kickstarter has been used for a few years in these communities and it's been ramping up.

Of the 24 most funded projects in the tech-heavy Design category, 18 are from this year. Ditto for Technology. In Comics, it's 19. In video games, every single big-ticket project was funded this year. So it's no wonder projects in these categories haven't seen as much success as in other categories. They haven't had enough time.

That's what I mean by a "Kickstarter bubble." In certain project categories, there's been a sudden flood of money into the system before backers can see whether it's a functional way of producing actual products in that category.

Most of the top Kickstarter projects deliver

I'm willing to give projects that have just closed a pass for not coming through – most of them are using the money to try to realise their dreams right now. Of the 198 projects I studied which funded in 2012, 63 have delivered. Looking at older projects, nine categories have an 80 per cent or better completion rate for pre-2012 projects: Art, Comics, Dance, Design, Fashion, Food, Photography, Technology and Theatre. Only 21 of the 98 projects from 2011 haven't delivered, and two of the projects from 2010 still haven't delivered.

(Yes, there's a gap in the Games category because none of the top 24 gaming projects were funded more than six months ago.)

In the geek categories, 10 out of 12 older projects have succeeded so far. Who's out? The Eyez heads-up camera glasses, which just sounds like a mess from the creators' updates, and the Hexbright flashlight. Pen Type-A, a high-end metal pen, has delivered some units but seems to be having trouble making it to scale.

But few of the mega-projects have delivered.

Of the 312 projects I studied, 30 have received more than $500,000. Let's call those the mega-projects. Of those 30, only six have delivered products by now – a 20 per cent success rate.

That's because mega-projects are generally new for Kickstarter. Of the 30 mega-projects, 28 were funded in 2012. Ten of them have been funded in June and July alone.

It's not that Kickstarter is a bad way to fund these high-dollar projects. We simply don't know. The money has been flowing in too fast to produce results by which we can judge.

The track record for video games on Kickstarter is awful

Video games are actually the most expensive category I studied, with an average funding level of $779,009 (£502,000) and a median level of $547,714 (£352,000). ("Design" has a higher average, but that's distorted by Pebble's massive multi-million haul.) Every single one of the top 24 funded games was funded this year, 13 of them in the last 60 days. There's a lot of hot air in that balloon right now. As you'd expect from a category so new, only four of the 24 projects have actually resulted in games.

I'm actually hoping I'm wrong with this conclusion, but I searched the web, read the pages, and can't find most of these games. Maybe some gamers reading this can explain why people are pouring money into this category without seeing results.

I'm still suspicious of Ouya

Sorry, Ouya fans, you still haven't won me over. Ouya is a brand new company with no history of delivering complex hardware products. It's backed by Yves Behar, a design consultant rather than a manufacturer, and Julie Uhrman, an exec with a long history in software and none in hardware. Behar's projects have always looked good, but they've succeeded based on who his partners are and their manufacturing experience.

Just yesterday, after I wrote my last column, they revealed that they're employing Amazon's Muffi Ghadiali, who says he does have a history building hardware. That's good, but the company is still running more on "please have faith in us," as opposed to, "here's why we can do this."

I understand that gamers are frustrated with the oligarchic state of the gaming marketplace, but that doesn't mean these guys can pull it off. Ouya's dream is interesting, but its PR so far hasn't proven to me that it's being executed by people who can bring a major hardware project to fruition on time and on budget.

So what does all this mean?

Someone once told me: "Be a pessimist; that way your life is full of pleasant surprises."

I've come away from this analysis much more enthusiastic about Kickstarter than when I started. Kickstarter is a terrific funding platform for the arts, and it has a pretty good track record of delivering projects in the art, dance, comics, food, and theatre categories. Most of the biggest projects in those categories, given enough time, have delivered on their promises.

For tech geeks, though, it's best to think of Kickstarter as a “reality show” or a “new kind of entertainment,” as Ian Bogost at Fast Company says. Money has been flowing into those categories very swiftly recently, and when we're talking about video games, there's no track record at all when it comes to the largest projects. It's too soon to tell if Kickstarter is attracting viable, large-scale electronics or gaming projects.

Kickstarter supporters have warned me that this shouldn't be seen as a new way to shop – it's a way to show support for ideas you like. I think that's a really healthy approach. Pledge something small. You'll probably get the sticker or T-shirt you're promised, and you'll show support for an idea. But I'd still hold off from pre-ordering non-existent hardware or software through this system.