Vetting a startup (or two): The systematic birth of @WPEngine

Let me dispose of any lingering ideas that WPEngine was a great flash of insight, born perfect from conception like a Greek God emerging from the ocean, a once-per-decade occurrence sourced from unsystematic inspiration. Let me also dispel your doubts that you can’t produce similar ideas with similar results using a similar process. You can, because there’s nothing magical in how I got to this point.

Inspiration and luck play important roles at the beginning, but turning a bright idea into a real business where strangers come to a website and part with money is an intentional process. A process you can learn and get better at.

This story is one shared by (in my experience) the majority of successful startups — an initial idea inspired by travails from my own life, an idea which turns out to be unworkable, but which ultimately leads to truly good ideas through both incremental and discontinuous transformations. I’ve written about that path before, both in my own companies and in others.

I didn’t do this properly at Smart Bear; that just happened through a combination of luck and perseverance. This time, though, it was intentional. As a good student of startup theory, especially modern theories of customer development, this time I was methodical and purposeful.

And it worked.

Here’s how to do it.

The first startup idea came from a pain I solved at Smart Bear. How do you measure the efficacy of ﻿30 simultaneous marketing campaigns, particularly when offline (e.g. print ads or trade shows)? At Smart Bear I built a system that worked incredibly well — the data were so accurate I could tell you the day an issue of (now defunct) Dr. Dobb’s Magazine hit people’s desks — precision unheard of with offline advertising and still pretty impressive online. (I’ll tell the full story of that system in another post; a lot of the tricks you can use yourself.)

Problem was, this system was hacked together over two years: a mish-mash of spreadsheets, Perl scripts, API-calls, manual exports, Excel spreadsheets, and Access databases. ﻿It was an inscrutable, tedious process that only I could execute, even then only when guided by interminable checklists. A stranger presented with this “system” would conclude that it was intentionally designed to be impossible for anyone but the creator to run, and furthermore intended as a peculiar form of self-torture.

A year after leaving Smart Bear, having talked with dozens of startup founders about their marketing woes, I realized 100% of them needed the marketing-measurement engine I built at Smart Bear.

There’s the magic moment! Inspiration sparked by need. A real solution to a real problem shared by thousands, if not millions. I have the know-how to built it and even a blog to promote it.

I got meetings with everyone I could find. Startup founders in my town of Austin were the most obvious target because it was easy to take them to lunch. I’d been networking for a while so I had a nice list to call on. Specifically, I had at least peripherally helped a bunch of other startups over email and at various meet-ups and co-working spaces, and now I called in the favors.

(At this moment many of you will protest: “But what if I don’t have an extensive network?” Then find local meet-ups of the type of people who would use your product and go, maybe even sponsor something. But what if you don’t live in a place where such things happen? Then you should be working on your online presence. But what if you don’t want to do that either? Then you can try to find people by dragging them in with Google Ads and asking them to sign up. But what if you can’t do that or don’t want to spend that money?

Then you’re saying you don’t want to communicate with anyone in any form and don’t want to spend any money. So I guess you’re not cut out to run a company, or do anything else requiring substantial social interaction!)

For me it comes down to this: Will they pay for it, and how much? You can’t stop at “Do you think this is a good idea?” and the story of this marketing-measurement tool demonstrates why, because every last person said it was a good idea, and in fact it was not. How did this manifest?

﻿I didn’t have any screenshots, and you don’t need them at this point, because if the idea isn’t exciting after brief explanation then it’s too complicated. If you feel the need to say, “I can’t explain it, you just have to see it,” you’ve lost. Maybe they have to see it to fall in love, or eventually hand over the credit card, but you can’t do market research or vet ideas if you “can’t explain it.”

Also, how are they supposed to spread the word to their friends? How will they explain it when even you can’t explain it?

So I’d kick off these meetings by saying the bit about how I could measure the unmeasurable Dr. Dobb’s ads, and how I could measure 30 simultaneous campaigns. At this point everyone would exclaim, “Oh yeah, I need that, and I know a bunch of people who need that.” Validation, right?

Then I’d get into feature details, and this was exciting too. For example, with Google Analytics you have to set up your goals and campaigns ahead of time; if you get data and only later realize that a different goal is important, or that you really needed to measure a different page, or that the goal values are different than you thought, or that you got traffic from a campaign you didn’t expect, it’s too late. The best you can do is reset the configuration and wait for (pay for?) more traffic.

Not so with my tool! I retroactively apply your inbound campaign definitions and your goal results, so you never worry about getting things set up right the first time, and you can easily experiment with anything you like, any time. 30 out of 30 people agreed this was a killer feature.

I didn’t figure out that this wasn’t a proper business idea until we got to the question of price, because that’s where the agreement ended. Some said that although this would save them $1000/mo in marketing spend, they still couldn’t afford even $50/mo; it would have to be free. I know, that ROI argument makes no sense, but that’s exactly why it was a bad sign. Others said $50/mo would be easy, and I should target the small business market, however it became clear that training and marketing costs with that audience would be large. Still others insisted that I go after the “big guys” in the market, which means charging at least $1000/mo, offering professional services, or partnering with other consultants who were willing to ditch the name brands for something better — though I was never able to locate a consultant willing to entertain that idea.

Price isn’t just a number — it dictates target market, which dictates competition, feature set, and the route to those customers. If you don’t talk about price, you don’t get into these questions deeply enough.

Of course your idea is pretty good, and of course you can convince people your features are pretty interesting. But without getting down to brass tacks of pricing and business model, you haven’t proven anything about your business.

In the end, I gave up on the idea of the marketing-measurement tool. It took about two months, and I really didn’t want it to die. But I’ve seen — and had — enough ideas to know that it’s possible to have a decent idea and no business model, and that’s what I had.

How does this relate to WPEngine? Because if I hadn’t been this meticulous and self-critical about the marketing-measurement idea, I would still be floundering around with it. But I killed it, and that gave me the breathing room to have another epiphany.

WPEngine has exactly the same story with a different customer development outcome, and that’s why what I just described is a repeatable process and not dumb luck.

I needed something like WPEngine for my own blog. My server would crash when I got on HackerNews, because that’s what WordPress does unless you’re an expert. My page-load time was pretty bad. And having gone through the experience of being hacked, I knew how painful that can be, and WordPress is known for being hacked constantly (more often because of poor IT than WordPress itself, but either way you’re screwed).

I searched for a WordPress hosting service that would help with speed, scalability, and security and found nothing. I asked around and everyone said “It doesn’t exist, but man that’s a great idea, I need that too!” Of course I’d just gone through this little song and dance, so I knew those statements alone weren’t sufficient.

So I hit the meetings again — email, phone, in-person, whatever. I wanted to talk to anyone with a blog. Would they pay $49/mo if I fixed those three problems (security, speed, scalability)? What if I added a feature I had hand-built for myself (again out of need) — a push-button staging area so you can test changes before they go live?

The reaction this time was completely different. Some people said they’d pay $49/mo just for the staging feature alone. When the answer is “You had me at ‘staging feature'” — i.e. when just a part of your pitch is already enough — you know you’re over-delivering on expectations, and when you’re comparing those expectations to an actual price, you know you have market fit.

I didn’t start WPEngine until I had 30 people saying they will give me $49/mo. Not would, but will. In the end, 20 actually did. Our message resonates with hundreds of thousands of people, and new people sign up every day though we’ve still spent $0 on general marketing.

Will this process work for everyone, for every idea? No. For example if you’re hell-bent on building something “disruptive,” then you probably honestly can’t explain it, and other people may not be able to understand its utility. Surely Twitter was that way. (Still is?)

But if you’re building something that’s supposed to be straightforward, that’s supposed to solve an obvious problem, supposed to be easy to enjoy, supposed to be easy to spread the word about, and supposed to make money, then you have little excuse not to talk to 30 people before you invest hours in writing code and building websites.

Even with awesome ideas, you don’t know whether it’s a business until you talk turkey.

What’s your experience or ideas about early customer development, especially before the product exists? Can you use this technique or is it unworkable? Let’s continue the ideas in the comments.

Great article. But I hope you can shed some light on the following scenario:

We’re working on a product which we believe brings websites a whole host of benefits (keep people on-site longer, etc). It will take something around 3-4 months to build an MVP that we can actually present to a site.

In terms of customer development, we already have a few sites willing to try it out (and pay us accordingly, assuming they get the benefits we promise).

But without actually launching our product and collecting real-life statistics, we have no proof that our product actually does give the benefits described. We believe it does based on our hunch (and the hunch of the customers who have already “signed up” as our first users), but this isn’t really validated in any way.

So, given that scenario, what would you consider the best course of action? Are we just working on something that’s too far into the “this must be disruptive” field?

(I hope I’ve explained the situation properly, even though I didn’t get into specifics.)

Of course you can’t PROVE the benefit until it’s live. So what? All you’re validating now is what benefit people WANT and how much they’re willing to pay if you succeed.

If you fail, you’ve wasted your time and a little of theirs, but it was worthwhile all around just in case.

If you succeed, you’ll earn their money and probably many others besides.

Maybe I’m missing something, but nothing sounds wrong with this to me!

http://concretemulticore.wordpress.com/ Doug

First, do the math, it is pretty easy.
You are a web developer right. First guess is you could make $90K per year working for the “man”.
4 months at $90K per year = $30K in lost opportunity cost.
Is the upside worth more than $30K?

4 years at $90K per year = $360K is way harder to justify.

http://www.geoquestour.com Sylvain Gauchet

Once again, that’s a great article !

I definitely agree with the fact that you need people telling you they will (vs. would) give you money. You can also be talking to people that really see the value of your product but are not the ones actually making the final decision of buying it…Wich can make the launching harder than you thought it would be.

http://www.friendlydingo.com Ryan

I like the overall idea of the article…but all I really took out of it were 2 things:

1) Have meetings about your idea to see if people will pay for it
2) If you can’t explain your idea you should forget about it

Two things that aren’t exactly earth shattering news to anyone in the industry of startups. ;)

Also, Twitter flies directly in the face of #2.

http://blog.asmartbear.com Jason

Actually, it is pretty shattering to some. There are wonderful pundits who I very much respect who argue the opposite. For example, Sean Ellis says you must not talk about price until you have product-market fit, whereas I believe it’s essential to lead with it.

I love Sean Ellis. I think this is a point that intelligent people can agree to disagree on.

Which is exactly why it’s non-obvious!

Anyway, it was also just an excuse to tell two stories in enough detail to make it feel real instead of Twitter-isms like “talk to customers.”

http://hashmapd.com miles

I appreciate you taking the time to expand the one liner into a full real story. The real, practical story gives the simple takeaways a lot of weight for me – and makes it easier for me to forward this to my other partners who may not have heard of jargon words like ‘customer development’ before.

http://aoteastudios.com Alex

I’m wondering if getting such an enthusiastic response as in the case of WPEngine should make you reconsider the price. Perhaps it could be higher if people would be happy to pay just for one feature? Or do you determine the price later, once you at least have an MVP out?

http://blog.asmartbear.com Jason

It’s a good question. Remember that some people pay $99, others $250, and some many thousands a month depending on size, so it’s not, in fact, just $50/mo, that’s just the floor and the typical amount I expect from the average sized blogger.

You never know about price. It could be higher with one feature, and in fact we do that today where WordPress MU support costs $99 (but then you get unlimited sub-sites.)

The goal here is not to get pricing perfect. In fact if you end up with a completely different pricing structure that might be OK too!

The point is that without bringing it up there’s no validation at all. You’re trying to prove a concept, not necessarily solidify a business plan.

http://blog.gugl.org/ Roman

I still cannot believe that people weren’t ready to pay for it. It really sounds like a good idea, and it’s also sound in terms of ROI. I also somehow didn’t get the price disagreement – did you set a goal price (say $200/mo), which wasn’t met, as you mentioned that many people were ready to pay $50/mo.
I’m really intrigued to read about “How do you measure the efficacy of ﻿30 simultaneous marketing campaigns, particularly when offline”. It sounds like magic :)

http://blog.asmartbear.com Jason

People wanted everything from free to $9/mo to freemium to $50/mo to $500/mo to $5000/mo, but also wanted various other changes in features or direction to match, none of which seemed to add up.

Is there a good idea in there somewhere? Heck yes, I’m sure there is. But I guess I wanted something a little more straightforward instead of another slog.

I believe if someone put their foot down and decided “The product is X, it costs Y, and here I go,” they could probably carve out a niche. But in a field as competitive and massive as “online marketing” I thought it needed more oomph than that.

On the 30 things — yes I agree it was insane…. I’ll have to get typing…

http://aoteastudios.com Alex

I’d be keen to hear about tracking offline campaigns too. The common wisdom seems to be that online is advertising is so much better because with magazines, TV and the like it’s very hard to gauge the ROI.

http://prettygraph.com Hrishi Mittal

Nice to see a detailed example of Customer Development out on the field.

Jason
One thing I would like to point out about the sub-heading, “Price isn’t just a number — it dictates target market, which dictates competition, feature set, and the route to those customers.”

It is actually the other way (which is also what you state in your ensuing explanation). Price we can charge depends on customer segments, what job they are hiring the product for, alternatives available to them, their reference price, channels and other values they expect out of the buying experience.
Price we can charge depends on the segment and its profile.

That said if a marketer indeed decides on a minimum price below which they do not want to go to market (due to their costs and IRR etc) then that price decides which segments to target and hence the channels and routes to market.

-rags
@pricingright

http://blog.asmartbear.com Jason

It’s easy to argue that one is a cause and the other is effect, but probably the better conclusion is just that they need to match.

Greg

This technique works for exisiting products as well, during sales consultations, we’d ask prospective clients how important feature “X” was (a feature we didn’t have).

The reply was often “very” or “extremely” and we’d followup with, “Great! How much more would you pay for feature X ?” And we’d hear nothing but crickets chirping.

Finally, another post from the man! I was beginning to give up on you here, Jason. I love that you de-mystify the ideation stage. Somehow, even though you’ve given me your pitch of finding 10 people, I imagine that your idea was some sort of magic. Good for my mind to know it’s not.

http://blog.asmartbear.com Jason

A fair criticism! It’s true — skipped a few weeks and a few guest posts. I’ve got a bunch lined up for next year though; back in the saddle.

It’s almost like starting a new company takes all my time away from other things…. :-)

http://austintechgeeks.com Ricardo

Thank you for sharing the story about the birth of WPEngine. It is somewhat relieving (really!) to know that even experienced entrepreneurs as yourself, still go through the same uncertainty about startup ideas. So, no magic wand after all… ;)

http://www.startupinception.com Henry Martz

I completely agree. A lot of time and money would be saved if people had an efficient way of testing their ideas on a small scale.

One method, just asking people in your network is good, but obviously not ideal. A lot of the time, you can creatively devise ways of testing parts of your business in a low cost, low time manner that will give you a better idea of the demand.

Thanks for the interesting post. I enjoyed it and started thinking about a bunch of different ideas.

http://Techstbooks.Wordpress.Com Colindowling

Nice article. Far too many startups are started and, unfortunately, funded without any real idea of the potential market. I agree that the best way to find a good product is to build something that you need yourself. Too many people are Caught up in trying to be big stars that they forget about how hard it is to launch and prosperity with a simple niche business. If you have a product that you love them the chance is good someone else will love it. Start there.

I want to start by saying that this is a positive comment. Do not look for anything negative – it is not there. With my love for abbreviations I will use Idea#1 – Campaign Analytics and Idea#2 WE (your current venture WP-Engine).

Idea#1 – “30 simultaneous marketing campaigns, particularly when offline (e.g. print ads or trade shows)?” There are lots of tools to measure online campaigns. Why do you think start-ups now are interested in print ads and trade shows analytics? It looks like a medium to big businesses business. And you say: ”Still others insisted that I go after the “big guys” in the market, which means charging at least $1000/mo, offering professional services, or partnering with other consultants who were willing to ditch the name brands for something better — though I was never able to locate a consultant willing to entertain that idea.” Could it be a problem here that you did not manage to talk to target audience and could it be that your target audience, assuming being reached, will not talk ‘how much’ talk because they buy EXISTING products not ideas?

Audience for idea #1 – “But what if you don’t live in a place where such things happen? Then you should be working on your online presence.” Why didn’t you use this blog to reach to people potentially needing this product? I think this audience is much bigger. Yes – you do not know many readers personally but does it really matter if they need idea #1 and want to pay or discuss or beta test etc.?

Audience for idea #2 – same question – why not via blog? Too wide audience to share early idea with a risk to be stolen?
Idea validation. I do not know if it is a good example and please tell why if it is but I want to talk about iPod – product which was launched at a time when market was full of MP3 players from different vendors. Can you imagine Apple presenting an idea of a new MP3 player without having a prototype and asking if people will be buying it? Idea of presenting an idea and asking ‘how much you are willing to pay’ as a validation looks promising as is promoted by many famous people but if you read ‘Black Swan’ you know that if this is not validated by any statistical research it always should be considered as ‘as is’ advice. You either follow it or do not but it is not even close to ‘Exercise leads to a longer life’. So in fact, and again from this book, everything revolutionary can never be predicted. We will never have space trips, electricity or nuclear power if we will always present an idea and ask ‘how much’ question from audience.

Idea validation – part II. Idea of validation by presenting an idea to your target audience can only be used, from my point of view, for web applications (in a web space). We tried to classify them here(http://www.deepshiftlabs.com/dev_blog/?p=42&lang=en-us) and you will see that for social, content and commerce sites this approach will simply not work.

Idea validation – part III. There are actually many other ways to build a business from idea:
1. Build for yourself and sell later to others who need it too
2. Build MVP and pivot
3. Build for client and turn to product
4. Validate idea (with/without prototype asking potential users ‘how much’ question), use (2) afterwards
I am not sure why 4 is a better way. Many successful businesses started with 1, 2 or 3.

Safer bet.
Taking open source product and turning it into service is a safer bet for sure. Examples – WP-Engine (WP hosting), Slicehost (XEN hosting), many companies hosting SVN, Git etc. Good thing – you are not locking clients. If not happy they can take their content and move elsewhere. You provide extra features to stop them even think about it. Another example would be to take locking web services and build a downloadable product with extra features – take uservoice.com and make a downloadable product from it, take 37signals Answers portal and make downloadable product from it. I am sure there is a market for such products where people do not want to lock their Ideas and Answers into somebody else’s external web services. Surely you will add new features and will compete with services giving your clients a freedom of owning content.

You. Having an idea which is potentially targets medium/big business and “never able to locate a consultant willing to entertain that idea” and Idea#2 with WP hosting on steroids looks like a choice is obvious if to select between these two even without presenting it to an audience. Am I right?

http://blog.asmartbear.com Jason

Many thanks for the thoughtful response! Here’s mine:

“Why do you think startups are interested in print and tradeshows?” Because almost all the ones I talk to either are trying or want to try them. Most don’t do them exactly because they’re hard to measure, and marketing dollars are dear. If that can be fixed, suddenly you have more options, and more options is better.

“Could it be you didn’t manage to talk to big guys?” No, I did talk to big guys, and several consultants who recommend and install such things for big guys. They were interested in general but couldn’t commit to the dollar amount I would need to make it viable.

“[big companies] buy existing products, not ideas.” At the time I had a prototype. Big companies do take bets on startups — ask any startup selling to enterprise clients and they’ll all tell you about 1-3 early, key, large customers who took the bet with them.

“Why didn’t you use this blog to find people?” Because I didn’t have trouble getting in-person meetings as-is. If I did have trouble with that, I would absolutely have used this blog (and Twitter) to do so! In general doing things online or over the phone isn’t nearly as useful as in-person, so I always prefer in-person. In that aside I was addressing next-best alternatives.

“[Worried] the idea of WPEngine would be stolen?” Not really. What’s the difference if it’s stolen in March or in June? Again it was just because I didn’t need to.

“Can you imagine Apple showing a prototype? Actually they do exactly that, just not on a massive public stage.

“Everything revolutionary cannot be predicted.” I totally agree, which is one reason I don’t try to do revolutionary things. Don’t confuse “Starting a company” with “doing something revolutionary.” Twitter is revolutionary, and I wouldn’t have bet on it, and the 100 other similar services which died along the way is good evidence of that. An incrementally better marketing tool or an incrementally better WordPress hosting service is not revolutionary, and absolutely can be market-tested before you start.

“Market-testing commerce sites doesn’t work.” Well I personally know 3 different startups who did market-test commerce sites and “iterated” before writing code, and all three are currently growing and seem to be successful, so I disagree. In fact I disagree generally with blanket statements about what can and cannot be done.

“Build it for yourself and then sell it to others.” Yup, many successful companies were like that. And many more died because building for yourself, even with iteration, doesn’t ask “is the market big enough to sustain a company” or “is tech support a nightmare” or “how hard is this to productize” or “will they pay enough money for it” or “will it be cheap enough to find customers.”

It’s a great way to start, but if there’s great answers to all those questions that’s luck and not intent.

The point of this article (and many of mine) is to try to reduce the risk from the startup by addressing the key risk factors earlier. Of course it could be done otherwise — as I said in this article I didn’t do with with Smart Bear and that turned out fantastically. But it’s not the least risky way to go.

“I’m not sure why 4 [validating idea first] is a better way.” Because if you have reasonable first-stab answers to those questions above, the startup is less risky.

“Safe: Turn open-source into a service.” No, it’s even easier than that — just do any service whatsoever. Consulting companies are quite easy to start and run, particularly if it’s a one-person shop. Sure, open-source consulting is fine, but so is Salesforce.com consulting or marketing or design or mobile development.

Nothing wrong with consulting companies! But with relative safety comes reduction in possible reward — consulting companies aren’t sold for lots of money because the business model always scales linearly with costs. It’s just a different risk/reward profile.

“A product which only big companies wants is not as good of an idea as one that’s web-based as masses. You don’t need validation.” I completely disagree. First, you are the one concluding that only big companies want it, but actually the biggest interest I got was from small companies. Your concept of what it’s useful for and who actually thinks it’s useful is exactly opposite.

Which isn’t a fault of yours! It just demonstrates the value of customer validation, because that corrects your false assumptions.

Yes, once I had all this input about what is and isn’t interesting, and to who, and at what price, then the choice is obvious. But if you don’t go out there and ask, you’re guessing, and all of us — you, me, everyone — often guesses wrong about fundamental things.

http://www.bluefishgroup.com Michael Trafton

Great article. I love how you show that the same process eliminated one business idea and launched another. I really enjoyed reading it.

Quick question: when you say you wanted 30 people who committed to paying, was it only then that you started building the product? Or did you start sooner and the magic # 30 was when you were all-in? For example. maybe at 10 (out of 20 meetings total) you thought “this looks promising, let’s spend the next week fleshing out prototypes.”

http://blog.asmartbear.com Jason

In this case I didn’t write a single line of code until this was complete. However that’s unnecessary.

If the first 10 interviews are all sounding identical, I don’t see why you shouldn’t start development while continuing to interview people.

If the first 10 are all over the place, like with Lead Spy, then indeed it might make sense to delay development because you still don’t know what you’re going to build.