Crisis Management — From the Man Who Helped Save eBay

Maynard Webb looked out the window to the eBay parking lot. CNN’s van was still there, camped out, waiting for another site-wide outage.

By the time Webb stepped in as president of the company in 1999, the world’s largest auction site was going down so often it was a national news story. And as more and more dot-coms flamed out around it, it seemed doomed to fail. Webb’s job: turn things around.

His first day may have turned into all-nighter (one of many to come), but he eventually led the company to stability and major growth. This experience, combined with the time he spent on Salesforce’s board, and at Yahoo, among others, has earned him the nickname Mr. Fix-it. And for good reason.

When it comes to crises, there’s no one better to learn from than the man who has kept his cool through some of the most dramatic mishaps and turnarounds the tech world has ever seen. In this exclusive First Round Review interview, he offers advice at startup scale.

Triage

Webb was groomed for his role as the ultimate crisis manager during his time as CIO of Bay Networks and Gateway Computers. “When you’re the CIO, something usually goes wrong every day,” Webb says. “By the time many problems got to me, there wasn’t much we could do but deal with the consequences.”

Your goal as a crisis manager is to catch issues before they hit that point, and your number one weapon is listening. “You have to keep your ear to the ground at all times — but you also have to find the signal in the noise,” he says.

Working at a startup can be very noisy too. It won’t always be that your whole site is out. Bugs pop up. Features break as you grow. Old code becomes a ticking time bomb.

In a real crisis, there's usually blood everywhere. But there's a difference between bleeding from arteries and bleeding from veins.

“A lot of people chase the wrong problems. They run around and do things, but it’s completely meaningless,” says Webb. “Before you can fix a problem, you have to ask why does it deserve my attention?”

While at eBay, Webb and then CEO Meg Whitman created a crisis Richter Scale to rank issues from one to 10 — where a 1 was “who cares” and 10 was “the world’s on fire.”

“People would run in all the time saying their problem was a 10 and we’d tell them it was actually a 2 given all the other things going on,” says Webb. “So many things can go south at any time, you have to have good judgment about what’s above a 5 or below a 5. That’s two very different responses.

If your whole site crashes — that’s a 9. Maybe a 10. “Look at what’s going on with the government health care sites right now. That’s probably an 8 or a 9. If your customer can’t do what you as a business are required to do, that should be your only focus.”

A 3 looks very different. “You have a 3 on your hands if part of your site is degrading, or a little-used feature is broken, or someone wrote a bad article about your customer service. You have to address it, but it shouldn’t derail you.”

In other words, you can’t let a 3 take you away from things that are more important to your business — including your next feature or bringing in new customers.

“At eBay, it got so bad that when customers were talking about how slow search functionality was, one of our engineers said, ‘They’re lucky the site is even working right now.’ When this is the case, prioritizing your problems is everything.”

It’s critical to have a rating system so you can triage crises appropriately every step of the way. During his time at eBay, Webb was woken up regularly in the middle of the night by engineers reporting that the site was down. If together they couldn’t resolve the issue in 15 minutes, he’d wake up Whitman and the other leadership to get the best brains working on the problem.

Build Your Fire-Fighting Team

Once you know how bad a problem is, you have to make sure you have all the resources you need to fix it fast. Your most valuable resource will always be good people.

From the beginning, you need to focus on assembling a crisis-ready squad of talented engineers, product experts and community managers. But talent shouldn’t be their only key quality — they have to be wholly dedicated to the company, even under the worst circumstances.

Webb's three top tactics:

Identify these people during the hiring process by asking them the right questions. Are they actually willing to stay up all night? One question you should always ask: If something breaks at 2 a.m. but miraculously resolves itself before anyone understands it, is it okay to unplug and go to sleep? The answer should be no. Problems you don’t understand have a tendency to strike again. “At eBay, we built a team of adrenaline junkies,” says Webb. “We needed people who would stay up all night waiting for things to go wrong.”

Manage these people closely so they don’t burn out. “We made a rule where if these types of people were working, it had to be on something invaluable. If they weren’t putting out acute fires, they got to take a break.” You’ll find yourself constantly asking this team to go above and beyond, from debugging code to facing a firestorm of angry customers to sitting through a detailed post-mortem after it’s all blown over to make sure it never happens again. You have to reward these efforts, and the best way to do it is with your most treasured resource: time.

Make sure leadership is demonstrating the same behavior. There’s nothing more motivating than knowing that if you have to be awake, your CEO and president are right there with you. “You have to be an example, and that example is if there’s a problem, we’re going to be all over it until it’s fixed.” In this sense, Whitman was a model CEO, Webb says. “She always wanted to get the late-night call, even if we might have been able to handle something ourselves. She always wanted to know what was going on. It’s something special when you have a top leader who still wants to get woken up when you have a billion-dollar market cap.”

This crisis squad of engineers got to be so committed to the work that when things finally stabilized, “They were actually disappointed that they didn’t have as many chances to be heroes anymore,” Webb says. “All I could tell them was, ‘Hey guys, that’s success.’” When this is your biggest problem, you know you’ve done something right.

Hold onto the Best and Leave the Rest

Rule of thumb: When you find yourself backed into a corner, there’s nothing more important than the people on your team. And when things get tough, you have to hang on to the very best ones with everything you’ve got.

For high performers who aren’t addicted to high-stakes problem solving, you need to be able to paint a convincing picture of what things will look like when they get better, and why it’s worth it to stick around.

“They can’t feel like they’re on a sinking ship,” says Webb. “You have to make them feel like you have two tickets on the 50-yard line of the Super Bowl and you chose them to come with you. You have to inspire that level of confidence. They have to know that you’re here to give them the ride of their life and grow something big — even when the numbers are saying the opposite.”

Remember, you can be inspirational and honest at the same time. Being honest with your employees is non-negotiable, especially when things are bad. This means making sure everyone knows what’s going on when your product is breaking every day. It means looping everyone in when you’re running out of funding and cuts are imminent.

Webb takes the latter as an example. “Here’s what you say: ‘Look, here’s the cash we have left. It’s enough for X months. The fundraising environment has been a little bit tougher than predicted, so here’s what’s going to happen: I’m totally committed to what we’re doing, but we need to make some changes.’ Maybe you say the company can’t hire anyone new, or you need to re-scope the product. But make it clear you’re certain you can win. You’re thinking very carefully about it. It’s just going to take a little longer.”

In the case of eBay’s site problems, employees were constantly exposed to bad press forecasting inevitable failure. “Tell everyone all the time not to pay attention to public opinion if you can — it is almost always wrong. Things are never as good as you think when you’re flying high, and never as bad as you think when you’re bottoming out.”

When it comes to issues of this magnitude, he recommends direct intervention. Speak first to top direct reports in a one-on-one setting. “They are your lieutenants in delivering the message in a way people will accept.”

If you’re at a very small startup — between five and 10 people — do two things: Talk individually to everyone about what's going on. Then get everyone together in a group so people can ask questions, see how you handle their questions, and see what their peers are concerned about. Everyone can learn from that experience.

The other thing this open forum does is give you a sense, as a leader, of who’s thinking deeply about the situation, who’s with you for the long haul, and who you actually want by your side. In a crisis — particularly if you have limited resources — you want to make sure you’re only working with the very best. “Is everyone doing what they should be doing? Are you happy to be at the Super Bowl with every single one of them?” As Webb puts it:

Settling for mediocre is bad enough at a big company. At a small one, it's catastrophic.

Make Customer Service Your North Star

When you’re a founder or a CEO, your employees are valued stakeholders — but you probably have thousands if not millions of customers to contend with too. Incidentally, most of the crises that pop up will probably be surfaced by your customer community.

To take full advantage of this potential, set up mechanisms to listen to your users as early as possible. “At eBay, we had an email distribution list for our most vocal users, and we’d identify a ton of issues we needed to fix just by monitoring their complaints,” says Webb. “Then it became all about keeping lines of communication open.”

This doesn’t require a ton of action. In fact, it’s vital that you don’t overreact.

“All you need to say is we’re aware of the situation, our engineers are on it, we’re diagnosing the trouble, and we’ll update you within the next half hour.” When you keep things simple, you’re more likely to be understood, and less likely to elevate alarm.

“The priority is to reset expectations,” Webb says. “You want to focus their attention on when things will be fixed. Even if you estimate that it will take two hours or longer to fix, say that. Then they can go do something else. If you leave it open ended, people will just sit there and wait and get more upset.”

This is where incremental updates on a separate blog or via Twitter come in handy so people know you’re working hard. And when everything is back up, it’s mandatory that you apologize.

“Apologizing directly to customers is absolutely part of the job,” says Webb. “You want them to know if they are using your product or service and you let them down, you will say sorry.”

Before Webb joined eBay, the company experienced an extended bout of downtime that could have been fatal had it not been for quick-thinking customer service. “Meg did this beautiful thing — she had everyone in the company stop what they were doing and they all called a list of community members, just to apologize.”

A grand gesture like that builds an amazing amount of goodwill. “By calling, the company acknowledged that some of our community members were depending on us for their livelihoods. It reminded them that real people were looking out for them. It reminded them that they also wanted us to win.”

At the same time, on a smaller scale, the company developed guidelines for compensating customers for poor user experiences. This included offering rebates or extending auctions for more days if downtime persisted longer than anticipated. All of this was designed to send one message: You can trust us to be on your side.

When salesforce.com faced similar issues, Marc Benioff built a trust task force to restore customers’ faith in the company, Webb says. Now anyone can go to trust.salesforce.com and see a full rundown of system status and security alerts from anywhere in the world.

Don’t Give Up on Growth

Besides generating customer trust, having a real-time dashboard around key metrics like site speed, accessibility and capacity forces you to think about growth — and you need to keep this squarely in mind if you want to survive.

When Webb first started at eBay, he and Whitman both weren’t content to simply keep the site up, they wanted to grow aggressively too. This was a tough challenge given that the swelling size of the company’s user-base was responsible for most of its server-side issues.

To wrap his mind around it, Webb asked the engineering team one question: How much oxygen do we have left?

“When they asked what I meant, I said, well, the site is down and we’re still growing super fast. How much time do we literally have left with the resources we currently have? They had never looked at it like that — but then they came back a couple days later and said, ‘We have three weeks to live.’ That really clarified what we needed to make happen and drove a lot of hard work.”

For a year and a half, the company had only four goals: keep the site up, scale it, deliver twice as much, and quadruple product development. “If it didn’t fit into one of those four buckets, we didn’t do it,” Webb says. “Most people would have just fixed the site and frozen the checks and not made any other changes, but that wasn’t right for eBay. We could have had a very safe, stable site if we weren’t trying to grow so much simultaneously.”

Most people think what we did at eBay was crazy, but it became a huge strategic advantage for us.

This fierce commitment to growth ensured that Webb and his team were always asking forward-looking questions: When can we upgrade our technology? How much capacity do we have? When will we hit the next wall?

Only when you stay on top of all of these factors can you slowly and surely pull your company out of crisis mode, Webb says. Then it’s time to dive into the next thing.

Maynard Webb is a 30-year veteran of the technology industry. An active leaderin the technology and business community, he serves as a board member, investor, philanthropist and mentor to young entrepreneurs.

First Round ReviewThe best advice in techSubscribe for Exclusive Access