This Company Believes You Should Never Hack Alone

As you walk down the stairs, overlooking the office beneath you, you notice that everyone works in pairs.

Each desk includes two keyboards and two mice and, in some cases, two monitors, but they connect to only one computer. Some people are standing, and some are sitting. Some are talking, and some are listening. But no one works alone. Each pair handles a single task, both mice and both keyboards helping to build and shape the same piece of software.

These are software engineers inside the San Francisco offices of a rather unusual company called Pivotal, and they're practicing what's known as pair programming. This is a well-known means of building computer software, but it's also rare, and Pivotal takes the technique to extremes. As each pair works, neither engineer is permitted to send email or text chat – much less surf the web. If one of them needs a moment for email, he walks to a dedicated email machine against the wall.

Yes, the idea is that two brains are always better than one, and the company puts strict rules in place to ensure that each pair stays focused on the task at hand. "It's more efficient, and it produces better code," says Rob Mee, the man largely responsible for those rules. "The engineers don't go onto their laptops unless it's absolutely necessary. They don't use their phone unless it's absolutely necessary."

That may sound terribly draconian. But for many who practice pair programming, it's also a wonderfully effective way of creating new software, and at Pivotal, there's something poetic about it. You see, two of the most important engineers at Google – Jeff Dean and Sanjay Ghemawat – sat at the same machine as they helped design many of the groundbreaking software platforms that underpin the company's entire online empire. And what Pivotal is building inside this San Francisco office is software that seeks to mimic the Google way. The company aims to fashion a sweeping set of tools that lets anyone run an online operation that's as nimble and as powerful as the one operated by the Mountain View search giant.

It's a rather ambitious undertaking and, nowadays, a common one. Over the past decade, as its web operation grew to unprecedented sizes, Google was forced to create new software that can juggle and analyze unprecedented amounts of data – software that runs across thousands of computer servers. And, now, so many companies and open source projects are trying to recreate these tools and help the rest of the business world scale similar heights. But Pivotal's efforts are particularly ambitious. And it just may have the pedigree to pull it off.

Pivotal is less than a year old, but its roots go back much further. Tech giant EMC and its big-name subsidiary, VMware, created the new company, each spinning off pieces of themselves that could help the operation fashion what has been called "Google in a box." Basically, EMC and VMware each contributed a few existing software projects, but EMC also threw in an operation called Pivotal Labs, a kind of consulting firm that helps people build software. This is where the new company gets its name – and its devotion to pair programming.

Under founder Rob Mee, Pivotal Labs spent the last decade teaching its clients pair programming and other Agile software development techniques – techniques that seek to facilitate collaborative software engineering – and along the way, it has helped shape some of the biggest names in Silicon Valley, including Twitter and, yes, Google. Now, Mee and his team are lending their expertise to this unabashed effort to mimic Google. Together with others across the new Pivotal, they're working to meld several disparate software tools into a single platform that any large business can use to rapidly build and deploy online applications and almost instantaneously analyze massive amounts of data.

That's a daunting task, to say the least. But today, the new company released the first version of its sweeping collection of software tools – something known as Pivotal One – and its efforts are already backed by a third investor, the venerable multinational conglomerate General Electric, which also aims to make use of the new platform.

Some question how successful this jigsaw-puzzle-of-a-company can be. Gordon Ritter – co-founder and general parter of Emergence Capital, a Silicon Valley venture capital firm that focuses on business software and services – says that, with so many companies trying to reinvent online infrastructure, the odds are that few will come out ahead. But GE believes the company has not only the right idea – but the right people to bring this idea to fruition. "I always invest in people," says GE vice president and global technology director Bill Ruh, a nod towards Paul Maritz, the man who oversees the new Pivotal.

Straight Outta Google
———————

Paul Maritz.

Photo: Ariel Zambelich/WIRED

In Microsoft's 1990s heyday, Paul Maritz was one of the company's top three execs, ranking below only Steve Ballmer and Bill Gates, and in the aughts, as CEO of VMware, he helped remake the way the world runs data center software. He'll tell you that Pivotal first started to come together in late 2011 and early 2012, but to understand this rather complex operation, you have to go back another three years.

In 2009, at VMware, Maritz hired Mark Lucovsky, an engineer who worked for him at Microsoft, helping to build the company's Windows NT operating system, a seminal piece of software still provides the core code for today's Windows OSes. VMware had been hugely successful selling virtualization software – a way businesses could get the most out of the computer servers that ran their software applications – but as virtualization began to saturate the market, Maritz was looking for other ways the company could grow. He turned to Lucovsky for help.

After leaving Microsoft for Google – a move that famously resulted in Steve Ballmer throwing a chair across his office – Lucovsky helped fashion an particularly popular part of the search giant's online infrastructure, and when he then jumped to VMware, he brought two other Googlers with him. Together, they created an entirely new software platform that would help businesses bootstrap massive online applications. It was called Cloud Foundry, and basically, it gave businesses a way to build applications, deploy them to the net, and expend them to an ever larger number of users as time went on – all without having to worry about the nuts and bolts of online infrastructure.

Maritz saw it as something that could bring the Google ethos to the rest of the world, touting it as the next big thing in the evolution of VMware. But one of the project's chief engineers soon left the project, saying he didn't agree with the path the project was taking, and in late 2012, rumors began to swirl that the project would be spun-off into separate venture. In the spring of this year, that's what happened. It became the centerpiece of a new Pivotal.

Pivotal does fun and games too.

Photo: Ariel Zambelich/WIRED

The Startup That Isn't
———————-

Cloud Foundry never quite seemed to fit at VMware. VMware was a company that made its money selling proprietary server software, and Cloud Foundry was an open source project designed to run on any server software, not just VMware tools. But Maritz says Pivotal wasn't an effort to offload the project – or any other part of VMware or EMC.

As the new decade took shape, Maritz says, he and others at VMware and EMC realized they needed to respond, in a bigger way, to recent changes in the world of business software and hardware. "Something important was happening in the enterprise IT computing space," Maritz says, referring to the new ways that the giants of the web, including not only Google, but Amazon and Facebook and Yahoo, were remaking the the way information technology was built. "We're weren't going to get from here to there unless we did something different."

>'Something important was happening in the enterprise IT computing space. We're weren't going to get from here to there unless we did something different'

Paul Maritz

As outsiders will tell you, the two companies were very much under threat from the new way of doing things. It's not just that the likes of Google and Amazon had created software – and even hardware – that provided a far more powerful and efficient means of running massive online operations. Through what's called cloud computing, these web giants were offering their new breed of infrastructure to the rest of the world, letting any business run big applications without help from vendors such as VMWare and EMC. Sometimes, as Gordon Ritter says, you have to "disrupt yourself."

After much discussion within the two companies – and with outsiders, including big-name business advisor McKinsey and Company – Maritz and the rest of the EMC brain-trust resolved to create a new outfit that would tackle the new way of doing things.

In some ways, Maritz says, Pivotal is a repeat of what EMC had done after it acquired VMware in 2003. VMware then offered a very new kind of technology, and though it became an EMC subsidiary, the parent company set it up as a largely independent operation. Eventually, it would even go public on its own. This is still a rather unusual arrangement, and now, Pivotal uses a similar governance model.

"You've seen big monolithic companies create internal divisions, internal groups, to go after things and still keep them within the corporate umbrella. Or you get startups, at the other extreme," Maritz says. "This asks if there is a third way." Pivotal can operate without weighing down EMC, and its employees are motivated by a startup-like stock arrangement, the thinking goes, but it can still benefit from the influence of its big-name parents.

The difference is that EMC didn't acquire the new company. Maritz and team seeded this independent operation with several software projects that were already a part of EMC and VMware, including Cloud Foundry, myriad other tools for building online applications, and software for analyzing "big data."

The wildcard was Rob Mee's Pivotal Labs, which had been bought by EMC several years earlier. At the time, it seemed like an odd buy for the storage hardware giant. But it now has a definitive purpose.

It's Not For Everyone. But It Works
———————————–

Scott Yara.

Photo: Ariel Zambelich/WIRED

EMC bought Pivotal Labs at the behest of Scott Yara.

Yara is the founder of a data analytics outfit called Greenplum, that was itself acquired by EMC in 2010. After joining EMC, Greenplum hired Pivotal Labs to help it build a collaborative data analysis tool called Chorus, and things went so well, Yara pushed EMC to snap up Rob Mee and the company.

"We had a project that was no going very well, and we turned to Rob for help," remembers Yara, whose father just happened to work with Paul Maritz at chip behemoth Intel in the early '80s. "After that, the project really began to accelerate, and we really bought in to Pivotal's process and culture."

Mee's outfit didn't just teach agile development techniques. It brought newbies like Greenplum into its own offices and paired them up with its own engineers, demonstrating agile techniques – including pair programming – on the fly. To be sure, these techniques aren't for everyone. Many developers chafe at the rigidity of Pivotal way. But for Yara and others who have worked with Pivotal Labs engineers, including Twitter founder Jack Dorsey, the results are well worth the clash of cultures.

According to Roger Neel – the chief technology officer at Mavenlink, a company that has worked closely with Pivotal Labs for over three years – the process not only improves the quality of software code. It helps you introduce new hires to your code, and makes it that much easier to share tasks among engineers. Ultimately, this means you're not as dependent on any one engineer or group of engineers. "It has an enduring nature to it," says Neel. "It lowers your risk – as well as improve your overall result."

The key, says Mee, is to find engineers suited to this way of working. He hand-picks each engineer, giving candidates a collaborative coding test that has become known as the RPI, short for Rob Pairing Interview. "We hire by programming with people," says Mee, a kind of software coding philosopher who's been known to keep stacks of math textbooks and unopened whisky bottles on his office desk, "and we do it in measurable and computable way. I have 100-point scale, and we only consider people in the top 10 percent."

When the new Pivotal was created, the old Pivotal Labs was charged with steering the development of Cloud Foundry. As the project was reshaped, according to Maritz, it lost two-thirds of its existing engineers, and that includes Mark Lukovsky. But, says Maritz, it needed to evolve.

A few of the day's programming pairs.

Photo: Ariel Zambelich/WIRED

At first blush, the new Pivotal seems like a rather eclectic collection of software tools. It includes everything from the Spring Java framework – a means of building applications with the Java programming language – to data analysis tools built by Greenplum. But the aim is to provide a wide ran of tools for building and running a modern online operation – a Google-like operation – and all the Pivotal projects fit the bill.

What's needed is something that can bring all these tools together, make it easier to juggle this eclectic collection. And that's where Cloud Foundry comes in. Under Rob Mee and Scott Yara, Pivotal is working to ensure that businesses can use Cloud Foundry to easily deploy and scale not only Greenplum's software, but also any application they might build with the Spring Source tools and other development tools offered by Pivotal.

That's easier said then done. But that's why the new company has put its faith in Mee and pair programming. "They happen to have an asset they will let them just get it done," Roger Neel says. "Aside from their methodologies, Pivotal Labs is just a strike force of high quality engineers."