Mike Benson wasn’t looking forward to negotiating his new telecom contract. The CIO of DirecTV had invited his existing provider, AT&T, along with rivals Sprint and Verizon, to bid on DirecTV’s new contracts for 2006. Benson wasn’t just negotiating for the satellite TV company’s local and long-distance communication needs but for all of its voice, data and networking services.

Not only would he have to untangle the telecom carriers’ incredibly complicated pricing on current services, but he would have to figure out which could offer the best deal on new networking technologies such as VoIP telephony and multiprotocol label switching, or MPLS. And he knew that if he switched from AT&T to a different company, it could take up to two years to complete the transition.

"[The carriers] will assure you the migration will be fine," Benson says. "But in reality something will always go wrong."

Making the right decision is a big load on Benson’s mind. And he is not alone. Now that telecom and IT have converged of late into networked IT services, the responsibility for negotiating and managing telecom contracts in an increasing number of companies has fallen to the CIO. And many are not prepared for the challenge. According to a survey of IT execs enrolled in The Ohio State University’s CIO Solutions Gallery program, telecom contracts are the source of most CIOs’ greatest long-term strategic confusion and biggest all-around tactical, day-to-day administrative frustration. And they openly acknowledge it is their sector of greatest ignorance.

To make matters worse, the telecom arena has never been so chaotic. Deregulation has created a thicket of carriers offering long-distance, local, wireless and networking services at unpredictable rates. These carriers use dozens of different billing formats, and CIOs regularly complain about errors and overcharges.

While the past year has seen unprecedented megamergers, most notably the marriage of SBC and AT&T, these M&As have done little to clear up the confusion. The costs to corporate America couldn’t be higher, in large part because the networking services offered under the telecom umbrella are more sophisticated—and more crucial to enterprises’ day-to-day operations—than ever before. "Many people think of telecom as a cost, and it is, but it provides a function we can’t live without," says Lisa Pierce, vice president of telecom and networks at Forrester Research.

According to Aberdeen Group, the average Fortune 500 company spends $116 million each year on telecom services (for mid-market enterprises, it’s $26 million). According to several telecom sources, telecom costs have jumped into the top three line items for most companies. In addition, up to 12 percent of telecom service expenses are erroneous. Such errors result in an estimated $8 million a year in lost profits per company, according to Aberdeen Group.

"It’s not hyperbole to state that networks and telecom are the worst managed function in IT," says Eric Goodness, a research VP for managed and professional network services at Gartner. "There’s anarchy and a total lack of governance."

But a few CIOs have found a path through the telecom jungle. Some have turned to third-party telecom expense management vendors, or TEMs, that know the lay of the land and can help CIOs through contract negotiations and billing problems. Others are saving on long-distance telecom costs by rolling out small-scale VoIP deployments. CIOs and analysts interviewed for this article offer valuable insights and examples of how they’re contending with the spiraling costs of today’s telecom.

If CIOs don’t grab control over their telecom spend now, "they will be behind the eight ball," says John Nallin, the vice president at UPS in charge of worldwide telecommunications. "The best defense is a good offense."

The New Telecom Landscape

For nearly 100 years, there wasn’t much to managing telecom. AT&T’s Bell System had a monopoly on everything, and its prices were, for the most part, nonnegotiable. The breakup of AT&T in 1984 and deregulation of telecommunications in 1996 ushered in a new era. Copper changed to fiber. Network capabilities expanded. And IP became the de facto networking standard in the Internet age.

Along with new choices came new complexities: dozens of telecom suppliers offering local, long distance, wireless and networking services at various prices in a bewildering array of billing formats. For the most part, though, the brand-new competition led to consistent reductions in telecom spending every year.

Within the past 10 years, however, the telecom landscape shifted once more, and no event was more jolting than when the IT department and the telecom folks entered into a sort of arranged marriage. Because IT ran data networks, and telecom carriers were increasingly providing network-based services (for WANs and LANs), and voice could now run over networks (VoIP), all of telecom was rolled under IT’s umbrella. Networking became even more critical—computers that can’t talk to each other are essentially useless—and CIOs set out to upgrade their network infrastructures to keep up.

For some enterprises, the process of upgrading their networks has become a Sisyphean task. "It’s like painting the George Washington Bridge," says UPS’s Nallin. "When we get three-quarters of the way down the bridge, and we look back, we know we’re going to have to start painting it again when we get done."

Carriers themselves are still operating with legacy databases and networks that were designed to carry only basic telecom services. Today, telcos are still trying to upgrade their systems to efficiently transport today’s data, voice and video offerings to keep pace with new competitors. (Verizon, for one, is spending $20 billion on its fiber-optic overhaul.)

Threats to carriers’ once-protected revenue streams are everywhere—from VoIP companies such as Skype and Vonage to cable providers such as Comcast and Time Warner. And then there are the IBMs, CSCs and EDSs of the world that offer an outsourced pain reliever for all CIOs’ telecom headaches.

"There’s tremendous confusion in the marketplace with all of this consolidation," says James M. Smith, a telecommunications attorney at Davis Wright Tremaine and a former telecom executive. "For CIOs, the question becomes, What does the future bring, and which horse to ride? Go with the blue chips or go with the Vonages?"

Even today, CIOs still fight the entrenched executive view that telecom costs should always decrease. What CEOs and CFOs don’t necessarily understand is that the enormous productivity and efficiency gains they’ve seen in their enterprises in recent years have been an outgrowth of the new networked telecom infrastructure. And their networks’ bandwidth needs will only continue to grow, putting added pressure on CIOs to explain why telecom is so vital to the company’s future.

"The networks are the veins of the company, and we’re bandwidth junkies," says the global network manager at a worldwide manufacturer of retail goods. "We’re a company that lives and dies on top of [the carriers’] services."

The Devil Is in the (Pricing) Details

The vexing challenge for CIOs in this new era is figuring out just how much telecom rates should be, and then negotiating fair deals with the plethora of providers that offer various networked services. Telecom contracts, which can contain hundreds of pages of obscure terminology and restrictions, can confuse even the most legal-minded CIO. "Sorting out the legal gobbledygook to get a contract on anything [related to telecom] takes as long as it does to put the project in," UPS’s Nallin says.

Prices can vary widely between service providers, and CIOs have no way of knowing whether the prices they are being quoted by various carriers are competitive or fair. "What becomes difficult is how do you stay current with the new rate structures and contracting approaches," says Tom Lesica, senior vice president for global information technology and business operations at Avaya. Lesica just went through an RFP with the carriers that included 12 telecom services. "It’s difficult for me or for my team to constantly go through the day-to-day, week-to-week fluctuations [in prices]," he says.

One carrier tactic that muddies the waters is called margin balancing. The telecom carrier will give the CIO a low rate on an 800-number, but not point out that the rate being quoted on something else (such as data services) is actually 40 percent above the going rate. "It’s difficult to know the price points that CIOs should be aiming for," says Charlotte Yates, CEO of Telwares, a company that specializes in telecom contract negotiations and maintains historical data on carrier rates. "You might as well have a dartboard."

For guidance, companies used to be able to look to Deal Watch, a compendium of carrier rates published by the Center for Communications Management Information (CCMI). Publication ceased last year because carriers stopped providing specifics—only wide ranges on services, such as voice and frame relay, even though the Federal Communications Commission requires rate disclosure. According to Bill Goddard, product manager at CCMI, the range of rates on frame-relay service, for example, can run anywhere from $32 to $40,180 per month. (To see a comparison of AT&T’s publicly disclosed rate contracts, go to www.cio.com/ 031506). "They’re posting data publicly, but it’s absolutely no use to anyone," he says. The carriers are able to get around the FCC requirement that they must "publicly disclose" their rates because that term is so unclear. CCMI filed a formal complaint with the FCC, and it’s still waiting for an answer. "The FCC has stuck its head in the sand and wrote a nebulous order and is not particularly stringent about the enforcement," Goddard says. "As an end user it becomes extremely difficult to determine what the market rates are."

Not only can executives no longer get comparative pricing information but in some cases carriers have actually inserted legalese into the contracts to block CIOs from comparing their telecom rates with other CIOs and then bringing those rates to the carriers’ attention during negotiations, according to Yates. CIOs should be alert to such clauses because talking to your colleagues is one of the best means of obtaining comparative pricing data.

CIOs may believe that the more services you purchase from one vendor, the lower the rate it will quote you. But experts say that is not so. Telwares’ data shows that small companies receive just as many lower-priced deals as the big boys do. Therefore, going with one carrier isn’t always the best solution for large enterprises.

Carriers say that they’re not out to deceive their customers. "It’s possible that I could trick a customer [during contract negotiations]," says John Irwin, vice president and general manager of BellSouth Business, but he says he knows he’ll lose such customers when they figure it out. There’s very little loyalty in telecom to begin with. As for the negotiations, Irwin says his customers rarely pay the "sticker price." CIOs might not always hit their ideal, Irwin says, "but they’re going to be in spittin’ distance."

However, Irwin does acknowledge the difficulty of the CIO’s negotiation position. "Probably there’s no foolproof way to make sure that [CIOs] are getting the best deal possible," he says.

How to Avoid the Billing Quagmire

Because they deal with so many carriers for networked IT services, companies big and small are awash in telecom bills. According to Aberdeen Group, the average mid-market enterprise processes more than 3,000 telecom-related bills per year; for the average Fortune 500 company, it’s 15,000. The cost of just processing these bills (with no auditing for errors) is $9 to $15 per invoice. Conservatively, this equates to an average of $135,000 in processing costs for big companies and $27,000 at midsize companies—and that’s just to keep on top of the volume.

Now think about this: Gartner estimates that up to 14 percent of telecom charges are in error. Finding those errors (think needle in a haystack) and getting the carriers to reimburse your account is arduous. Meta Group says 20 percent of the problem is finding the error; 80 percent is getting the carrier to pay up. Carriers are even imposing a statute of limitations on their customers seeking compensation for billing errors. Basically, if you can’t find the billing error (say a telecom service was ordered but wasn’t installed correctly) within a certain time frame—say, three months—you can’t get that money back from some carriers.

In order to process an invoice, telecom analysts need to look over each bill (some on paper, some on CD-ROM) and audit it against terms of the contracts as well as against tariff and service guidelines, tax charges and physical telecom inventories such as the number of circuits. Every month. A VP of telecom procurement at a global financial services company says her division alone receives 7,000 to 8,000 bills per month.