Can You Hear Me Now?

All networks have problems. Whether you are dealing with the newest, best designed network leveraging all the latest technology, up-to-date software or an old, poorly designed network running antique hardware with out-of-date software, you are going to experience problems. These problems will appear in different ways: network outages, performance issues, intermittent problems, or any one of a dozen other options. It is when these problems occur and how we deal with them that is important.

When problems occur, the amount of time it takes to resolve the issue or Mean Time to Repair (MTTR) is one of the most important metrics. A problem that is resolved correctly, in the shortest amount of time, is the problem that causes the least amount of damage. As problems go, issues with DSCP tagging of network packets can range from minor annoyances to major, highly costly outages. A mis-tagged VoIP packet can result in choppy, hard to understand calls that, when they happen once may be irrelevant or if they happen all the time may be of major concern. Companies that are dependent on VoIP for phone traffic, whether for internal communications, incoming sales, support or other customer calls, need to ensure that the voice quality starts and remains high throughout the entire call. A number of things can impact voice call quality. One of them is incorrect DSCP tagging.

VoIP is UDP based, an unreliable protocol that gets the job done quickly, efficiently, and without any sort of delivery guarantee. Unfortunately, because UDP is connectionless, if a few packets are misplaced or waylaid on the way from speaker to speaker, the call quality will suffer. Using the wrong DSCP tag on VoIP UDP packets can, in a worst-case scenario, result in extremely poor call quality. Perhaps the queue the packets are being put into is already full and there is no room. Maybe the queue runs over a low quality link. No matter the cause having incorrectly DSCP tagged traffic can be a major issue and the results can be catastrophic.

Because network performance management (NPM) solutions have a deep visibility into the traffic on the network, they are uniquely placed to identify and alert on issues with DSCP tagging. Being able to see how traffic is tagged at all points on a network results in not only being able to identify when traffic is mis-tagged, but also identify the exact location (or locations) where the issue is occurring.

A solution such as Riverbed SteelCentral allows the identification of DSCP tagging issues in two ways. Leveraging the advanced reporting capabilities of SteelCentral NetProfiler and deep packet inspection capabilities of SteelCentral NetShark you can quickly and easily create a traffic report that identifies all the interfaces on a network that have traffic (VoIP or other Layer 7 applications) tagged with an incorrect DSCP value. These reports can be used to quickly create lists of incorrectly configured interfaces or devices and the situation dealt with appropriately.

Figure 1: Wait, why is VoIP traffic running under so many DSCP classes?

NetProfiler’s User Defined Policies allow you to take the identification of misconfigured interfaces or devices to the next level by proactively alerting on misconfigurations as soon as traffic starts traversing the network. A policy can easily be created to alert any time traffic (VoIP or other applications) is seen that does not conform to the proper DSCP assignments.

Figure 2: A User Defined Policy allows you to generate an alert whenever non-conforming trafficis identified. In this case the policy is looking for mis-tagged VoIP traffic.

Take the following scenario: A router fails. When the network administrator replaces the failed unit and goes to reload the configuration, he discovers the most recent backup is corrupt. He goes back to an earlier configuration without knowing that changes to DSCP were made between the two backups. The newly restored router is now up and running but with incorrectly configured DSCP tagging.

Alternatively, an administrator could be making an authorized configuration change to the DSCP settings on a router but accidentally enter the wrong value. The incorrect tags would now be on the router or switch and traffic would be running slowly. Once a problem is identified, the obvious response is to look at what changed most recently (change control, failed device replacement, etc.). However, the actual issue may not come to light until later in the day, week, or month. Mis-tagged VoIP traffic may only start to become an issue after lunch on Tuesday because other activity is starting to eat up network bandwidth. If the change was made Friday evening, you now have three days of changes to look at. With the NetProfiler User Defined Policy, you would have been alerted as soon as the first mis-tagged packet traversed the network.

Leveraging the technology built in to your Riverbed SteelCentral NPM solution you can not only gain deep visibility into what is happening on your network but also pre-emptively identify and fix problems before users notices them. As anyone responsible for customer satisfaction will tell you, the best problem is the one you solve before your customers even know there is an issue. =