I am a New York City-based journalist covering business, entrepreneurs & technology. I'm interested in the long and winding road from concept to launch to success (or failure), and all of the milestones in between. I have a degree in business journalism from Columbia University, I've worked in daily newspapers in the U.S. and briefly in India, and I've covered M&A and business strategy. In my time at Forbes I've focused on startups, tech and growing enterprises.
Send me sensitive documents and tips (NOT everyday pitches) at https://safesource.forbes.com/

An International Army Of Tech: Applause

Last winter, when a publicly traded media-streaming company with millions of global users needed to stress-test its app on 100-plus different devices, it called a little outfit named Applause. “Anybody would know who it is,” laughs Doron Reuveni, cofounder and CEO of Applause. He’s not telling because he’s sworn to secrecy. He quickly mobilized 500 testers from 78 countries on five continents–including IT geeks in London, Madrid, Munich, São Paulo, Philadelphia and Bangalore. They kicked the app’s tires on various desktops, phones, gaming consoles and smart TVs to analyze functionality and user experience. Backup teams? Of course.

“We needed to have everything covered and completed by the morning after,” says Reuveni, 47. Applause of Framingham, Mass. relies on 140,000-plus crowdsourced testers and bug-seekers to root out imperfection. Last year it had $24 million in revenue–profitable for only brief stretches–working for the likes of GoogleGoogle, MicrosoftMicrosoft and News UK. Applause expects $35 million or so by year-end 2014. “Our focus is building a big company for a generation,” says Reuveni. “Not profitability in the next 12 months.”

To expand into Europe, Applause spent an estimated $5 million to buy Testhub in Berlin. It has also launched new analytics, letting customers test for glitches and monitor operations and user feedback. But it’s still barely a blip in what Reuveni says is a $20-billion-a-year business for testing and analytics, overshadowed by Hewlett-PackardHewlett-Packard, IBMIBM and CA Technologies, which specialize in Web and PC apps. It’s in mobile that Applause sees its future, as the app culture mushrooms in Asia and western Europe.

Here’s the challenge for the seven-year-old Applause: how to keep growing and still maintain control over its ever-expanding mass of testers. “Recruiting the supply is easy; vetting and engaging is harder,” says Matt Johnston, Applause’s 39-year-old chief strategist.

Potential testers fill out profiles of their location, languages spoken and technical experience. They’re compensated after training, completing at least one unpaid job and impressing established testers. “Only by creating a great deal of value for customers do you rise up the ranks; if you’re not creating value for customers, you’re not invited to future projects,” says Johnston.

Applause’s matching algorithm taps only 20% of testers to do paid work. Payouts are calculated according to how many snafus and failures they discover and how critical those are to a customer. About 2% of testers are full-timers for Applause and can make up to $10,000 a month; less active worker bees make under $1,000. The majority–some 80%–are simply content to be part of the network. Last year the company paid out roughly $15 million to testers.

If anyone can control a mob, it’s Reuveni and Roy Solomon, his 35-year-old cofounder and product manager. Both served in intel units in the Israel Defense Forces. Reuveni focused on signal interception and decryption. “Usually you don’t get that level of responsibility right out of school,” says the lean, avid runner. “In the military you do.” He was recruited by management-software firm Enigma in 1997, working up to senior vice president. He left in 2006 with thoughts of starting his own company.

Reuveni met Solomon at a party while visiting Israel in 2007. After leaving the IDF in 2000, Solomon worked at startups IncrediMail, Blue Security and genealogy social site MyHeritage. He zeroed in on a critical pain point: Companies build apps to reach millions of people in thousands of locations on hundreds of devices, and it all has to work perfectly. The pain, he realized, was global, “and it’s only going to get worse.”

At the time testing was done internally or by Indian outsourcers like Wipro, Infosys and Tata Consultancy. Solomon envisioned a vast pool of crowd-testers who could kick around an app–much like the way open source software works. Between his list of prospective clients and Reuveni’s tech contacts, they had enough takers for a beta test. They assembled 100 testers in three weeks for a startup they called uTest (the name was changed to Applause earlier this year).

Until then “testing of software was very inefficient,” says Joel Mesznik, president of the private equity firm Mesco and uTest’s first investor, who sank $1.75 million in 2007. He’s taken part in all five rounds, betting a total $5 million.

By 2009 the company was adding 600 testers each month–too fast for comfort. “We simply couldn’t keep throwing hours at this type of challenge,” says Johnston. So they asked vets to screen, train and mentor newbies through rigorous course work. The vets, in turn, are overseen by a seven-person team. “From the beginning we knew we were going to live or die by the integrity of our community,” says Solomon.

Security is key. Testers sign nondisclosure agreements with Applause, as well as with clients, and are banned from talking or blogging about current projects. Any breach triggers a lifetime ban from the network. To date no one has been fired for squawking, and fewer than 30 have left of their own volition.

Not everyone is a fan of crowdsourced testing. It’s far cheaper (and faster) to automate the process. “Manual testing is inefficient; everyone’s looking for a solution,” says Jay Srinivasan, CEO of Appurify, which, unsurprisingly, is in the automation racket. It charges Yahoo, Dropbox, Google and others an average $3,000 to $6,000 a month and $10,000 for 60 or so devices. Applause’s lowest-tier price is $30,000 a year to test two apps; customers with grander needs pay six- and sometimes seven-figure annual sums.

Reuveni says he is beefing up automated testing. But he’s also adding 5,000 new testers a month. Says Solomon: “We will invest in an Internet of Things community: experts who can test software that runs on cars or athletic accessories or microwaves–any connected device.”

They have the dough. In January Applause raised $43 million in a Series E round, led by Goldman Sachs, giving it an estimated value of $500 million. Testhub gave Applause 200 new European customers. Now it’s looking at Asia.

As for profits? “We’re a patient investor,” shrugs David Campbell, a Goldman managing director and Applause’s newest board member. “We’re not in any particular hurry.” That’s just the kind of backer every entrepreneur dreams of.