The Voice over IP Insurrection

Daniel Berninger, an old friend, a seriously smart guy and VoIP guru of sorts, and more recently senior analyst, for Tier1 Research, has been a great man to bounce ideas off. He and I have chatted about many things, and each time I come away learning something new. So last week he argued, “in the battle between Bellheads and Netheads, we’re all Netheads now.” Could not agree more. Here is his long missive on the VoIP insurrection, the best and most definitive essay you will ever read on this technology, where it is headed and why it is important. This is the second of my guest columns series where I bring the experts who know a thing or two about their respective areas of expertise.What just happened?

The $3 billion dollar budget at Bell Laboratories did not include a single project addressing the use of data networks to transport voice when VocalTec Communications released InternetPhone in February 1995. As of 2004, every project at the post-divestiture AT&T Labs and Lucent Technologies Bell Labs reflects the reality of voice over Internet Protocol. Every major incumbent carrier, and the largest cable television providers, in the United States has announced a VoIP program. And even as some upstart carriers have used VoIP to lower telephony prices dramatically, even more radical innovators threaten to lower the cost of a phone call to zero—to make it free.

Advertisement

The VoIP insurrection over the last decade marks a milestone in communication history no less dramatic than the arrival of the telephone in 1876. We know data networks and packetized voice will displace the long standing pre-1995 world rooted in Alexander Graham Bell’s invention. It remains uncertain whether telecom’s incumbent carriers and equipment makers will continue to dominate or even survive as the information technology industry absorbs voice as a simple application of the Internet.

The roots of the VoIP insurrection trace back to four synchronistic events in 1968. The Federal Communications Commission (FCC) ruled MCI could compete with AT&T using microwave transport on the Chicago to St. Louis route. The same year, the FCC’s Carterfone decision forced AT&T to allow customers to attach non-Western Electric equipment, such as new telephones, and modems, to the telephone network. The Department of Defense’s Advanced Research Project Agency issued a contract to Bolt Beranek and Newman for a precursor to the Internet. And in July 1968, Andrew Grove and Gordon Moore founded Intel. Innovation in the communication sector remained the proprietary right of AT&T for most the 20th century, but events in 1968 breached the barriers that kept the telecom and information technology industries apart. For the first two-thirds of the 20th century, AT&T had manned Berlin Wall separating telecommunications and computing, but eventually, these two enormous technology tracks would be unified.

Two entrepreneurs barely out of their teens, Lior Haramaty and Alon Cohen, founded VocalTec Communications in 1993 based on the promise of packet voice technology they observed as members of the Israel Defense Force. Most military command and control used the highly survivable TCP/IP distributed data networks since the 1980’s. The challenge of transporting voice over the networks arose as an imperative to support certain very sensitive voice commands like “drop the bomb”, but the idea of commercializing packet voice did not occur to anyone until the arrival of Lior and Alon. How could slicing voice into 50 millisecond packets improve the telephone business? The tradition bound telephone industry types or “bellheads” spent their time before 1995 improving the Public Switch Telephone Network (PSTN) not replacing it.

Advances in communication from writing and paper to the printing press, telegraph, and telephone shape human progress. Some might have viewed VoIP as an interesting toy in 1995, but no one presently doubts it will dominate the communication future. The economies of scale associated with growing customer awareness and competition will produce a Moores Law like virtuous cycle of communication innovation.

The coming communication renaissance will make the 20th century telecom industry seem as quaint as the Pony Express by comparison, but what happens to the existing trillion dollar global telecom industry? The near term stakes effect millions of telecom jobs and billions of investment dollars not to mention the fact communication serves as a basic input for the economy much like oil. Can companies like Verizon and Lucent reinvent themselves sufficiently to survive the VoIP insurrection? Keep in mind the telecom incumbents remain gatekeepers for access to the Internet and regulatory uncertainly continues to weigh against investment in VoIP. Verizon presently serves as many telephone lines, collects as much in revenues, and enjoys twice the profits as the pre-breakup AT&T colossus. Let the games begin!

IP on everything
The forces putting the Internet in a position to displace the PSTN have nothing to do with voice as an application. The cycle of innovation that serves as an engine for the information technology industry makes all of the elements needed to enable VoIP faster and cheaper. The forces reducing the price of processing power exist independent of VoIP, but as a computation intensive real-time application VoIP benefits from the persistence of Moore’s Law. The number of people with access to the Internet and, in particular, broadband access continues grow with or without VoIP, but all communication applications benefit from network effects. The Internet backbone latency declines absent specific requirements from VoIP. For example, a connection between New York and Hong Kong that might have suffered 500ms round trip delay in 1996 experiences less than 150ms delay today. The emergence of Wi-Fi and other unlicensed wireless developments associated with IEEE 802 working groups moving VoIP into the mobile realm represent yet another example of this dynamic.

No one originally expected the Internet to make the PSTN obsolete, but they did seek to avoid the vertical integration making the PSTN inflexible. The people shaping the Internet viewed themselves as users not service providers, where as the PSTN existed to serve the interests of the telephone company not users. The PSTN works as one integrated system synchronized everywhere to the same clock. All of elements that make up the Internet work independent of each other. The coordination happens through an exchange of messages. This difference sets the Internet on a pace of innovation that allowed the rudimentary capacity and performance available in 1970 to improve at a rapid rate unleashing the present Darwinian survival of the fittest process. On going performance improvements continue to give data networks the capacity to efficiently address an ever broader range of applications.

The arrival of VoIP in 1995 corresponded with the arrival of a PC (i.e. Intel 486 processor) capable of managing the encode and decode processing in real-time. VocalTec offered packet voice solutions starting in 1994, but the need for special purpose hardware to support real-time two-way communication slowed deployment. Dialup Internet penetration reached nearly 20% in the United States in 1995 and interest in VoIP accelerated in 2003 as broadband penetration rates approached 20%. Interconnection with the PSTN remains the largest cost for VoIP service providers, but the need for PSTN interconnection falls as broadband penetration increases. It only took ten years after the arrival of the commercial Internet for most people to get an email address. The same outcome seems likely with regard to VoIP telephone numbers. In any case, the challenge of producing a low cost VoIP version of traditional telephone call seems largely solved. The most interesting developments remain to come as VoIP enables capabilities that go beyond the plain old telephone call. With the price of the traditional telephone call going to zero, there will be enormous incentives to offer more value.

VoIP turns telecom into a simple extension of consumer electronics business, because Internet applications exist without metering for time and location. Users of VoIP need not worry about the destination or duration of their calls any more than someone sending an email or browsing the web. People do not pay each time they play a CD, and communications seems headed in the same direction. Microsoft X-Box Player already offers VoIP for participants in multi-player games. Metering and billing calls can easily cost more than delivering the service itself, and the flat rate access billing model eliminates the need for solving inter-carrier compensation.

The decoupling that produces rapid improvements in connectivity and processing platforms also facilitates software development. People working on VoIP applications don’t need to change the nature of the Internet with each new application, and everyone with a computer becomes a potential member of the Internet development team. Applications of the Internet from email to the web to instant messaging and VoIP without exception have come from the tinkering of entrepreneurs rather than an industrial research center backed by market research.

The competition between VoIP and the PSTN shapes up much like highways versus railroads. The operator of the PSTN and railroad own their transport network. VoIP companies and car companies do not. Railroads and the PSTN support a single type of usage. Highways and the Internet allow all user types to commingle. The emergence of highways empowered people to control many more aspects of their transportation needs rather than depend on the schedules and railroad routes available. The Internet accomplishes the same thing for communications. Automobiles and highways gave rise to an entirely separate industry and provided the basis for new types of commerce. The Internet offers the same promise, and corporate chieftains with traditional telecom assets find themselves in the same position as the railroad barons when Henry Ford got rolling.

III. Disconnect
Alexander Graham Bell lasted only three years in the telephone business. His financial backers took over American Bell not long after 1876 in a manner and for reasons not unknown to entrepreneurs today. Lawyers and financiers shaped telecommunications far more than engineers for most of the next century as various forms of monopoly dominated. The accomplishments of Bell Laboratories does not change the reality of industry stagnation. No area of technology progressed slower than the telephone business in the 20th century as comparisons with automobiles, aviation, or healthcare reveals. Innovations all revolved around obtaining efficiencies from the perspective of the telephone company. Telephone service changed very little from the perspective of end users.

The collection of data networks that make up the Internet exist entirely separately from the PSTN. The public switched telephone network refers to the network addressed dialing by telephone numbers. Home users may access the Internet by dialing telephone numbers, but none of the Internet hosts with permanent IP addresses depend on telephone numbers for connectivity. The links may share conduits. The equipment may get housed in the same buildings or share maintenance staff, but the PSTN infrastructure remains entirely separate from Internet infrastructure. One could turn off all of the equipment supporting the PSTN without effecting the Internet. The PSTN and Internet come together only through special gateways, because they transport messages by entirely incompatible means.

The amazing durability of plain old telephone service (POTS) does not arise from the fact it represents some kind of wonder technology. POTS persisted for business reasons associated with monopolization of telecom and not technology or sound quality. Humans can perceive sound from 50 Hz to 20,000 Hz. POTS captures sound between 300 and 3300 Hz. The copper analog loop associated with the Bell System in the US and monopolies in other countries add various other impairments to this already limited representation of sound. POTS remained dominant until the arrival of VoIP, because of the high barriers to entry facing insurgents.

The local telephone loop still uses analog technology introduced in the 1940’s. Touch Tone phones arrived in 1963. Engineers at Bell Labs understood growing demand required offering new services, but the nature of the PSTN made it largely impossible. Imagine the challenge of software makers in a computer industry stuck with something like the Intel 286 processor for 40 years. The PSTN as a communication platform does not improve in performance, whereas efforts to create demand for computers gets facilitated by Moores Law. The PSTN supports only three kilohertz sound, so everything including data needs to get converted to voice (e.g. the modem squeal). The complexity of the software controlling the large network based circuit switches utilized by Verizon, AT&T, and others make them unsuitable for implementing new services. Each new service requires re-inventing the telephone network, but the revenue prospects for the new services don’t justify the expense. The PSTN’s resistance to innovation meant demand grew no faster than the larger economy or 3-4% per year during the last 50 years.

One attempt by AT&T to improve voice quality in the early 90’s illustrates the PSTN’s handicap. Marketing studies indicated customers might prefer a low end (i.e. bass) audio boost. Sony implemented this with a “MegaBass” switch on their Walkman product line. AT&T wanted to do the same thing in hopes of competing with MCI and Sprint on voice quality rather than price. The consumer Vice-President at AT&T, Joseph Nacchio, pushed through an $800 million project to get the job done. AT&T could not simply install a switch on telephones analogous to the Sony’s solution. AT&T had to alter the signal processing incorporated in echo cancellers throughout the network. These network wide modifications produced irate customers not more customers. The higher sound levels caused operators distress and amplified existing network quality problems. A long period with lots of effort followed to unwind the TrueVoice implementation. Joseph Nacchio famously left to launch Qwest.

The arrival of VoIP in 1995 generated not a rush to hire engineers but rather a rush to hire lawyers. The Americas Competitive Telecom Association (ACTA) submitted a petition encouraging the FCC to regulate VoIP software makers like telephone companies. The move prompted Jeff Pulver to launch the Voice on the Net (VON) Coalition with Daniel Berninger’s help in 1996. More recently, the FCC issued a notice of proposed rulemaking addressing VoIP in March 2004 with the goal of bringing the Internet and PSTN together under a single rational regulatory regime. No one expects the regulatory issues to get resolved soon as the FCC faces a challenge similar to a railroad regulator addressing the emergence of highways.

The telephone incumbents also control a key input needed by the VoIP insurgents – connectivity. It does not represent an accident that there remains a significant disconnect between the capacity of connectivity within the premise associated with ethernet local area networks and the capacity in the middle of the Internet associated with fiber deployments and improvements in the capacity of fiber. The most capacity and greatest performance improvements exist where there exists the most competition. The fact that local access remains a bottleneck or completely absent follows from the weakness or absence of competition for so called first mile connectivity. The phenomena gives rise to the broadband deployment debate, but the disconnect depends less on technology than the absence of market forces compelling providers to improve performance and reduce their prices. The growing list of access technologies offers some hope (e.g. copper, coax, wireless, fiber, power lines, satellite), but it remains unknown whether there will exist sufficient competition between these media to reduce the access bottleneck.

The Pony Express service failed prematurely in 1861 when railroad interests convinced the United Sates government to withhold a promised contract leading its entrepreneurial backers into bankruptcy. Consensus on the technology merit of VoIP does not assure its future. The telecom incumbents like the prospects for VoIP to make their operations more efficient, but they will fight mightily the potential for VoIP to alter the telecom industry balance of power. Campaigns toward national and local regulatory interventions can certainly alter the pace of change. The losses suffered on investments motivated by the Telecom Act of 1996 leave decision makers in the capital markets twice cautious. The combination of regulatory and technology openings associated with VoIP seem a powerful force for change as in the case of MCI leveraging microwave technology and regulatory openings to set up the break up of AT&T in the 70’s.

Bell Labs had several smart people who were very well aware of the promise of packet-switched technology. In the early 1980s, Dave Sincoskie, then of Bell Labs, built some of the first known self-contained voice over Ethernet telephones. It wasn’t voice over IP because IP wasn’t finalized until 1983. But it *was* voice over packet, and it worked. I had one of his second-generation VoIP phones on my desk at Bellcore when I worked for Dave from 1984-91. We used them every day to call each other and the outside world through a packet-to-PSTN gateway — the very same model that commercial VoIP uses today.

The problem then, as now, was telco marketing and upper management who were totally oblivious or even hostile to these developments. But that doesn’t mean pioneers like Dave shouldn’t get the credit, instead of those who came along well over a decade later when it had finally become fashionable.

AT&T had been usiing packet voice for longhaul transport back in the mid to late 80s. Their ATM switch the BNS-200 was the first broadband voice transport coupled with ISDN handset they had all the capabilities that VoIP triumphs.

Cudos. A very informative read indead. My question posed is how will the insurgens of VOIP into the business market (IE VIOP based PBXs and phone systems for offices and call centers) effect the general use of the technology in the public? If someone has the option to have the phone that rings on their desk ring at their home over their broadband based internet, will this help push VOIP more into the mainstream? The options are there and they sky is the limit.

Randy…. thank you for saying that and putting everything in perspective. thank you a million times … i think we have such short term memories that we forget the good work of people before us. we all stand on someone else’s shoulders and think it is the ground beneath us.

points to an accurate timeline. The first two-way packet voice conversation occurred over the ARPANET in August 1974. I was on one end of that conversation.

In the late 1970s the DoD was looking to modernize its phone system, and one widely-published study by Frank, Occhiogrosso and Gitman showed that PACKET-SWITCHED VOICE WOULD BE CHEAPER than circuit-switched voice. AT&T was bidding on the system and certainly read that study.

The VoIP companies that emerged in the 90s deserve credit for commercializing packet voice, but many of their founders were in diapers when it was invented.

Prediction: The FCC and the existing telcom network will do everything they can to squash VoIP… The telcoms because they want to keep the status quo, obviously. But the FCC? It’s all about CALEA — the federal government’s continuing efforts to erode the constitution.

If each new subdivision built in the US had individual residence wiring that the customer owned all the way out to an aggregation point and there were a method of buying out that wiring on existing subdivisions, the problem of connectivity would quickly subside. It doesn’t pay to get a T-1 to your house. If you have a 50 house subdivision, it might be perfectly manageable to get 2 or 3 and VOIP to a local network from then on. Right now those aggregation points are entirely owned by the incumbent carriers. There’s no reason why builders couldn’t put the inside/outside wiring interface at the aggregation point and allow commercial grade connectivity alternatives to be sold competitively to the entire subdivision.

Frame relay and ATM (Asynchronous Transfer Mode) are both packet style transmission technologies from the slightly converted bellhead geeks.
They both handle voice.
They are priced, however by the bellhead marketing types whose plan over many decades is (was) to decrease prices by only a few percent per year.
They planed to reserve most of the savings for themselves.
It is not clear that IP is better than ATM, it is just under a different style of administration and priced drastically differently. That plan was disrupted by internet. Also the original plans to offer quick ATM call setup did not materialize. ATM circuits are now established manually in the style of leased lines. It is sort of like the days when POTS required human operators.

Great piece! Saw it referenced on Slashdot. I would love to see more, especially as you turn this around and project forward. As an example, I believe that Skype will kill the current telephony business in just a few years. Skype is easy to use, frequently free, and has better voice quality than POTS. It already has 23 million downloads, over a billion minutes served, and just launched the Skype to POTS link.

USA the Corporate Welfare State. The Telco sector is not the only part. In 1997 the FCC was auctioning spectrum for G2-Cell and WLL for home, and most of US stayed adicted to wire. Recently 2004 more spectrum was handed out by the FCC to telcos for G3-Cell and WLL, but …. I live about 40mi south of NYC, NY, USA, in a builtup heavy populated area, and still cannot get DSL.

Oil, agri-biz, or alternative fuels … Drugs and Stemcell research one discovers ways to manage illness, one may create permanent cures which does Corporate Welfare America want.

No – Voice Over Frame Relay does not count. The operative issue is benefits to end users ala Internet not internal PSTN efficiency. We could wait another 100 years and VoIP would not get “invented” at Bell Labs. I was on the first team at to react to VoIP and there existed useful elements around the labs that came together as a VoIP solution relatively quickly (during 1995), but Bell Labs was tasked to improve the PSTN not make it obsolete.

AT&T did not publish a price tag on TrueVoice. I am publishing here first from direct sources that shall remain anonymous. Nothing about the feature depended on things outside of AT&T’s control unless you mean at customer
premise. AT&T assumed customer premise equipment performed uniformly as per
specifications. Wrong. AT&T could not sufficiently test the feature. All of these challenges and problems were due to the nature of the PSTN.

Daniel and Om, thank you for this very thorough and insanely informative article. It sheds a great deal of light on regulatory-related frustrations I recently posted about on my blog, from the poorly-informed standpoint of an end-user. You’ve earned yet another faithful reader :)

Mobile operators will meet their “demise” with Wi-Fi/Wimax when people have alternatives to calling and downloading data other than the meter-them-to-death cellular networks. The cellular business model relies heavily on billing by the minute or with data, getting people to purchase expensive data bundles that charge you by the number of MB downloaded. It’s the old metering system and their organizations are built around that. The cash cow of the moment should have been 3G. But it’s not working as planned. Why? Who wants to pay six euros or more per MB? And how many MBs have you already consumed this month? You’ll never know till you get that horrible bill. Very customer unfriendly. They did not see the rise of Wi-Fi which is becoming more and more available. And now with Skype and other voice apps available on portable devices, we can see it coming.

As I often do when I hear yet another person detail the demise of the telcoms, would Mr. Berninger bring us all up to speed on how the mobile operators will meet there demise as well? Please keep the hand waving to a minimum.

Aswath, I think this is something I have heard about – consumers completely rejected it and it was one of those bone headed moves by Joe Nacchio, which i discovered only after I had writeen broadbandits. Such a shame though. Wish I knew about this earlier. I

Not to distract from the main points of this article, I have a coiple of questions: As early as 1989, AT&T Network Systems had developed a product that packetized voice and carried on Frame Relay network. It did voice activity detection and also had a mechanism for intermediate nodes drop “tails” of the packets in response to congested network. Granted it is not VoIP and it was designed as a transmission product. Does this qualify as a “project addressing the use of data networks to transport voice”?

I didn’t know that the real bill for True Voice was $800M. As I recall AT&T took out ads asking the public to try it. Surely the careful engineers at AT&T would have tested it before rolling the feature out. In my undersstanding the problem was that the feature depended on capabilites in segments of the connection that is outside AT&T control. An analogous situation arises in IP network as well. AT&T CallVantage claims that it carries voice in QoS enabled network, still it does not control one or both the tails of the connection.

WOW–sometimes it DOES PAY to read these blogs………as a large investor in Level3, I am not sure to cheer or run for the hills and come back in 10 years and ‘”SEE”” what works out in this RailRoad Driven PSTN market…………..

WHO will win this “”VOIP WEAPONS OF MASS DIALING””” WAR?????? My company is named “”VoipNuke”’ and its NO accident it got its name when “”Voip WMD’s”” start really flying , my mantra is “”””When you SEE the FLASH(VZ), It will be too late””