Microsoft is hosting the first ever Lync conference Feb 19-21st in San Diego, CA. If you haven’t signed up yet, there is still time. Perficient is the evening sponsor for the conference, so please find us if you’re planning on attending!

This is part four of a twelve post series, to see an index of all twelve posts click here.

On the fourth day of Lync’mas my UC Team gave to me: Lync Enterprise Voice and Overhead Paging Best Practices.

Before we deep dive into the technical details, you might be wondering how does overhead paging relate to the fourth day of Lync’mas? When planning Enterprise Voice migrations, we typically work with our customers to integrate Lync certified gateways with their analog devices, including overhead paging systems. During migrations, it’s important to understand the paging system requirements and how to make the functionality work seamlessly with the best UC solution on the market. Therefore, the fourth day Lync’mas focuses on FXS/FXO interfaces and loop/ground start signaling…

In my original XMPP post, I discussed how to troubleshoot TLS errors relating to Lync and the Microsoft OCS 2007 R2 XMPP gateway. If you believe you’re having TLS related issues, please refer to my previous post. This blog will focus on Lync troubleshooting techniques and Google XMPP responses when establishing federation between the two environments.

Background

A Lync 2010 architecture was deployed with an XMPP server for instant messaging federation to Google (gmail.com and other Google App domains). Once the XMPP service was deployed, my customer started receiving the infamous “presence unknown” for Google contacts. For the purposes of this blog, let’s say the Lync SIP domain (chi.contosouniversity.edu) was used by faculty and the Google Apps domain (gmail.com &contosouniversity.edu) was used by the student population. Side note… Let’s be real, everyone should be using Lync at this point… Anyways, moving on.

The environment was built to spec, but when Lync users tried to add Google users (gmail.com or contosouniversity.edu) they would simply receive “presence unknown.” If you didn’t catch it previously, the Lync SIP domain (chi.contosouniversity.edu) is a sub-domain of contosouniversity.edu. The root domain (contosouniversity.edu) was enabled for Google Apps Talk Service.

Long story short, the presence issue was caused by a misconfiguration in the Google Apps portal prior to the Lync deployment. The sub-domain (chi.contosouniversity.edu) was mistakenly enabled for the Google Apps Talk Service. Unfortunately at the time, I did not have access to the Google Apps Portal. If you fall into the same boat, below are the troubleshooting techniques that can be used to pinpoint the misconfiguration.

The architecture looks similar to:

Troubleshooting Overview

To begin troubleshooting, we’ll first start tracing on the XMPP gateway.

Sifting through the trace, one can quickly tell that the captured trace is not extremely helpful in telling us what exactly caused the communication to fail. While we see a 481-Call Leg/Transaction Does Not ExistandConnection failed to contosouniversity.edu (or gmail.com, etc.) on port 5269 on Connection timer elapsed so removing Outgoing Queue and Outgoing Connectionerror, one might assume there is a communication issue between the Lync Edge server(s) and XMPP gateway or between the XMPP gateway and Gmail… Well, not so fast.

To ensure firewall rules are open, I first recommend establishing an external telnet session to the external IP address assigned to the XMPP gateway on port 5269. This connection should be successful. Next, I recommend establishing a telnet session from the XMPP gateway to the following Google XMPP servers on port 5269. Again, the majority of these connections should be successful.

xmpp-server.l.google.com

alt1.xmpp-server.l.google.com

alt2.xmpp-server.l.google.com

alt3.xmpp-server.l.google.com

alt4.xmpp-server.l.google.com

If all telnet sessions are successful and you’re still questioning why IMs are failing, switch gears and open Network Monitor on the XMPP gateway. If you were like me previously, you probably do not have the NtSimpleSearch Expert installed. You can quickly tell by clicking the Experts tab near the top menu. If all you see is Download Experts, you have your answer.

To make this process easier, navigate to the MSDN NMSimpleSearch website to download the expert. Once the NMSimpleSearch Expert has been installed, it should appear under the Experts window.

Forget the NMSimpleSearch Expert for a moment and start a new network capture. Send another IM from a Lync user to a Gmail user. Once the trace is stopped, the cause to our 481-Call Leg/Transaction Does Not Existerror will actually be returned from Google’s XMPP servers. To find the answer, you can 1) manually read through each packet inspecting the Hex details for the following XML information…

<stream:error><undefined-condition xmlns=”urn:ietf:params:xml:ns:xmpp:-streams”/><str:text xmlns:str=”urn:ietf:params:xml:ns:xmpp-streams”>chi.contosouniversity.com is a Google Apps Domain with Talk Service enabled.</str:text></stream:error></stream:stream>

… or 2) use the NtSimpleSearch Expert to locate error.

Assuming you want to save yourself the headache, I’ll proceed with option 2.

To use NtSimpleSearch, first save the network trace.

When saving the file, ensure All captured frames is selected

Once the trace has been saved, close and reopen Network Monitor

Open the previously saved network capture

Select Experts | NtSimpleSearch

Once the NmSimpleSearch dialog box appears, enter Google Apps in the search field

If we’re troubleshooting the same issue, at least one instance should be found.

Lo and behold, Google is telling the Microsoft XMPP gateway that the domain (chi.contosouniversity.edu) is a Google Apps Domain with Talk Service enabled.

There you have it! Simply disable the Talk Service for the appropriate domain(s) in the Google Apps portal and instant messages should start working.

With the recent release of the Lync Mobile client for Windows Phone 7, Android, and iPhone, I’ve seen numerous companies thrilled about the opportunity to extend Lync further within the enterprise! For companies interested in deploying Lync Mobility, the Lync Server Cumulative Update 4 is required. For deployment steps, I highly recommend reading the Microsoft Lync Server 2010 Mobility Guide and Jeff Schertz’s blog.

After working with various customers to deploy the new Mobility service, occasionally I get asked “Why do I have to enter my username to sign into my Lync mobile client?”

I’ve read articles that mentioned if the SIP address username differs from the AD username, than the user would have to manually populate the User Name field to authenticate successfully. For example:

SIP Sign-in Address: jdoe@contoso.com

AD Username: john.doe

The mismatch between SIP sign-in address and AD username made sense to me, but for my customers their usernames matched. For example, the SIP address username matched their AD username:

SIP Sign-in Address: jdoe@contoso.com

AD Username: jdoe

Fast forwarding, Microsoft has published a mobility troubleshooting guide (KB2636318) which outlines various troubleshooting techniques. The article makes reference to the SIP sign-in address vs. UPN, but does not go into great detail. Below are two scenarios that highlight the difference between SIP sign-in address vs. User Principle Name (UPN) and the expected Lync mobile client behavior.

Scenario 1: Matching SIP Sign-in Address & User Principle Name (UPN)

To validate the UPN of a user, first launch ADSI Edit

Connect to the Default Naming Context

Expand the domain structure and locate the user account object

Right-click the user account object and click Properties

Scroll down to the field named userPrincipalName

In the screenshot below, the UPN (kcrockett@pbtest.com) is provisioned in the same format as my SIP domain (kcrockett@pbtest.com)

Switch over to the Lync Mobile client and populate the Sign-In Address and Password field

I recently had the pleasure of working with one of our customers to implement Lync Server 2010 Enterprise Voice with Cisco Unified Communications Manager (CUCM) 7.1.3. The integration between the two systems went pretty smoothly. During our testing phase though, we uncovered an issue with ringback. When a PSTN user or a Cisco IP Phone user would dial a Lync endpoint, the PSTN user and/or Cisco user would hear silence (no ringback) until the call was answered by the Lync user or diverted to Exchange Unified Messaging. Even stranger, we had a lab environment with the same CUCM version and Lync patch version which was working fine. After capturing a handful of call traces between the two platforms, we noticed:

CUCM was inserting a PRACK in the call setup signaling for every call to Lync

There were multiple 183 session progress messages between CUCM & Lync

Cisco has a documented ringback bug (CSCtf87428) that was discovered in CUCM 7.1.3.

According to the Cisco Bug Toolkit CSCtf87428, “OCS connected to CUCM via SIP and call coming in via H323 GW we get multiple “183 Session progress” from the OCS without any SDP information. At this point CUCM is sending “Progress” to the gateway with PI = 8, so the GW stops playing the ringback. As there is no SDP information received from OCS side the calling party gets silence.”

According to the Cisco Bug Toolkit CSCtf87428, CSCtf87428 is resolved in 7.1(5.11006.2). Unfortunately, we were not in a position to upgrade the Cisco cluster and ringback wasworking in the lab. We therefore opted to not upgrade CUCM and to investigate the issue further.

As outlined in the SIP RFC 3261, I expected to see something similar to the following:

Invite

100 Trying

183 Session Progress

180 Ringing

200 OK

RTP (Audio)

After working with the client to compare settings between the lab and production CUCM environments, we uncovered an interesting setting. Under the CUCM Service Parameters for the Cisco CallManager service, there is a field called SIP Rel1XX Enabled. Below is a screenshot of the configuration that had been set. SIP Rel1XX Enabled had been set to Send PRACK for all 1xx Messages.

As indicated in the right-hand column of the screenshot, the default setting for SIP Rel1XX Enabled is disabled. We scratched our heads… This setting had obviously been manually changed, but by whom and for what reason. After circling back with other team members, we had uncovered that this had been enabled as part of a previous initiative. Long story short, the setting was deemed not critical and we received approval to change the setting back to the default value of disabled. Below is a screenshot of the updated configuration.

After SIP Rel1XX Enabled had been set back to the default value, we captured a few more call traces between the two platforms. As shown in the screenshot, the call setup signaling looked better and more importantly ringback worked!

Lync Server 2010 is extremely granular when it comes to routing voice calls. Lync administrators can configure a voice call to load balance between multiple PSTN gateways within a single site or failover to a PSTN gateway in a second site in the event the primary PSTN gateway fails. After numerous discussions with colleagues and clients, I thought I’d share my thoughts on the difference between the two configurations.

Table 1 below provides a voice routing overview. There are two sites (HQ and Branch). The HQ site is in Chicago with a 312 area code and the Branch site is in Indiana with a 260 area code.

Table 1 – Voice Routing Overview

Figure 1 below provides an example topology between the HQ and Branch sites.

Figure 2 below displays a correctly configured Voice Route with a single gateway. As outlined in scenario 1 and 4 above, Lync will not load balance voice traffic between voice gateways if only a single gateway is defined in the voice route.

Figure 2 – Single PSTN Gateway

Figure 3 below displays an incorrectly configured Voice Route with two gateways spread across the two different sites. As outlined in scenario 3 and scenario 6 above, Lync will load balance voice traffic between the PSTN gateways defined in the voice route. In this scenario however, voice traffic will round robin between the Branch and HQ site which would cause unexpected results (caller-id issues, bandwidth consumption, call quality issues, etc.).

Figure 3 – Incorrectly Configured PSTN Gateway Load Balancing

Figure 4 below displays a correctly configured Voice Route with two gateways in the same site. As outlined in scenario 2 and scenario 5 above, Lync will load balance voice traffic between the PSTN gateways defined in the voice route. In this scenario though, voice traffic will correctly round robin between the PSTN gateways in the HQ site.

Figure 6 below displays a basic voice policy (HQ-Unrestricted) that does not have a failover or least cost usage configured. In this example, UserA has been assigned the HQ-Unrestricted voice policy. All outbound calls made by UserA will route out the local HQ PSTN gateway (HQ-SBA.kcdemo.local).

Figure 6 – HQ Voice Policy with no Failover or Least Cost Route

Figure 7 below displays an advanced voice policy (HQ-Unrestricted) that has multiple failover and least cost usages configured. In this example, UserB has been assigned the HQ-Unrestricted voice policy. In this scenario, UserB will experience the following:

All local calls (312 area code) will route out UserB’s local PSTN gateway (HQ-SBA.kcdemo.local)

Indiana calls (260 area code) will route out the branch gateway (Branch-SBA.kcdemo.local)

All long distance calls will route out UserB’s local PSTN gateway (HQ-SBA.kcdemo.local)

If the local PSTN gateway (HQ-SBA.kcdemo.local) were to fail, all outbound long distance calls would failover to the branch gateway (Branch-SBA.kcdemo.local)

All international calls will route out UserB’s local PSTN gateway (HQ-SBA.kcdemo.local)

If the local PSTN gateway (HQ-SBA.kcdemo.local) were to fail, all outbound international calls would failover to the branch gateway (Branch-SBA.kcdemo.local)

Figure 9 below displays a basic voice policy (Branch-Unrestricted) that does not have a failover or least cost usage configured. In this example, UserC has been assigned the Branch-Unrestricted voice policy. All outbound calls made by UserC will route out the local Branch PSTN gateway (Branch-SBA.kcdemo.local).

Figure 9 – Branch Voice Policy with no Failover

Figure 10 below displays an advanced voice policy (Branch-Unrestricted) that has multiple failover and least cost usages configured. In this example, UserD has been assigned the Branch-Unrestricted voice policy. In this scenario, UserD will experience the following:

All local calls (260 area code) will route out UserD’s local PSTN gateway (Branch-SBA.kcdemo.local)

Chicago calls (312 area code) will route out the HQ gateway (HQ-SBA.kcdemo.local)

All long distance calls will route out UserD’s local PSTN gateway (Branch-SBA.kcdemo.local)

If the local PSTN gateway (Branch-SBA.kcdemo.local) were to fail, all outbound long distance calls would failover to the HQ gateway (HQ-SBA.kcdemo.local)

All international calls will route out UserD’s local PSTN gateway (Branch-SBA.kcdemo.local)

If the local PSTN gateway (Branch-SBA.kcdemo.local) were to fail, all outbound international calls would failover to the HQ gateway (HQ-SBA.kcdemo.local)

One of the items I love about Microsoft Lync is the fact that I can plug my Lync certified headset (Plantronics Blackwire C420) into my laptop and instantly begin using it to make and receive telephone calls. This morning, I plugged my headset in to join a Lync Online Meeting and the audio started playing through my laptop speakers. As much as my colleagues love listening to me ramble, I thought I should try to resolve the issue…

If you by chance run into a similar experience, below are a few steps that should help resolve the headset speaker issue:

Open the Lync client

Near the lower left-hand corner of the Lync client, ensure the headset icon appears

If the headset icon isn’t selected, the headset may simply need to be set as the primary device. In my case, the Plantronics headset was already the primary device.

Click the headset icon and select Audio Device Settings

If a Lync certified headset is used, the device name should automatically appear as the primary audio device. Again, in my case the Plantronics headset was already the primary device.

Everything looks correct in the Lync client… Time to look at the main OS Sound settings

Navigate to Start | Control Panel | Sound

You may have to select small icons near the upper right-hand corner

Under the Playback tab, locate the Lync certified headset speaker

In my case, the headset lost it’s association to be the default speaker for audio

To correct the issue, simply click the Set Default button

Click OK

To test the setting, unplug the Lync certified headset and ensure the setting switches back to the Speakers/Headphones

Now, plug the Lync certified headset back in and ensure the setting switches to the Lync certified headset

Before making an Enterprise Voice call, let’s use the new Audio Test Service introduced in Lync Server 2010 to A) make sure we can hear the audio with the headset and B) check the quality of our voice

Open the Lync client

Near the lower left-hand corner of the Lync client, ensure the Lync certified headset appears

Click the headset icon and select Check Call Quality

Alternatively, click the telephone icon and then clickthe Check button

The Audio Test Service should start. Simply listen to the nice lady and you should be good to go!

Have you ever tried enabling a user for Enterprise Voice or Exchange Unified Messaging and received an error stating the action could not be completed due to a duplicate record? I’ve surprisingly ran into this issue more than once recently. Whether a telephone number was used for testing purposes and simply forgotten about or caused by inaccurate documentation, below are a few tips to quickly identify a duplicate telephone number in Lync Server 2010 and Exchange 2010 Unified Messaging.

Recently, I had the pleasure of deploying a new Lync Survivable Branch Appliance (SBA) at a customer site. If you’re unfamiliar with SBA functionality, a SBA’s primary function is to provide voice resiliency in office locations where direct internet connectivity to the main data center/central site is lost. I won’t go on and on about the SBA feature set, so additional information can be found here.

Each SBA manufacturer (AudioCodes, Dialogic, HP, and NET) has their own flavor of initial installation steps to get the SBA at an operational state. The one configuration point that this blog will call out is the Configure Domain step. Typically, the manufacturer guides call for manually creating a new computer object in Active Directory and then later joining the SBA to the domain via the SBA configuration webpage (http://x.x.x.x/sbastartup). A screenshot of this option is below:

Taking a closer look, the actual SBA code runs on top of a Windows 2008 R2 instance. A few administrators out there might decide to simply join the SBA to the domain through normal computer membership as seen below:

If the SBA is joined to the domain through the computer membership option, the green checkbox next to Configure Domain in theSBA configuration webpage will not appear (as seen above). The lack of the checkbox does not necessarily represent an issue and the SBA configuration can continue as normal. After the Configure Domain task, the administrator must install the Lync software and synchronize the Lync configuration files. The Install Lync Software option should complete as normal as shown below:

As soon as the Synchronize Lync Configuration Files option is selected, the task will most likely fail with the following error in the Event Log:

As described above, if the SBA was joined to the domain through the computer membership option then the login error above is directly related to the RTCUniversalSBATechnicians security group missing from the Local Administrators group on the SBA. To resolve the issue, follow the steps below:

Click the Windows Start button

Right-click My Computer and select Manage

Expand Local Users and Groups and select Groups

Once the security groups appear, double-click Administrators

Click the Add button

Enter the RTCUniversalSBATechnicians and click Check Names

Ensure that the Lync SBA technician account has been added to the RTCUniversalSBATechnicians security group in Active Directory

Click OK twice

Log out of the SBA configuration interface (http://x.x.x.x/sbastartup)

Log back into the SBA configuration interface (http://x.x.x.x/sbastartup) using the same SBA technician account

After logging back into the SBA configuration interface, the Synchronize Lync Configuration Files step should complete successfully!

There is plenty of documentation out there on how to install the XMPP gateway with OCS or Lync (references provided at the bottom of this post). This blog will not focus on the installation of the XMPP Gateway, but rather what to do if you receive TLS errors on the Lync Edge server when communicating to the XMPP gateway.

If TLS issues pop up on the Lync Edge server, odd behavior could be experienced with Gmail such as complete instant messaging failure, one-way instant messages, and/or unknown presence.

If you open the Event Viewer on the Lync Edge server, you may notice connection failures similar to the error below.

A significant number of connection failures have occurred with remote server lyncxmpp.internaldomain.com IP 172.X.X.X. There have been 94 failures in the last 383 minutes. There have been a total of 1750 failures.

The specific failure types and their counts are identified below.

Instance count – Failure Type

14 0x8007274D(WSAECONNREFUSED)

1735 0x80072746(WSAECONNRESET)

1 0x8007274C(WSAETIMEDOUT)

This can be due to credential issues, DNS, firewalls or proxies. The specific failure types above should identify the problem.

If you start a logging trace on the Lync Edge server, you may notice a series of failures similar to the errors below.

Text: Routing error occurred; check Result-Code field for more information

Result-Code: 0xc3e93c7f SIPPROXY_E_ROUTING_MSG_SEND_CLOSED

SIP-Start-Line: NOTIFY sip:LYNCXMPP.internaldomain.com:5061 SIP/2.0

SIP-Call-ID: 059f6d06c4e84676ac28bfce083f779b

SIP-CSeq: 6 NOTIFY

Peer: lyncxmpp.internaldomain.com:5061

$$end_record

If similar TLS errors appear on your Edge server, ask yourself “Is my XMPP gateway installed on a Windows 2008 or Windows 2008 R2 server.” If XMPP is installed on Windows 2008 R2, various compatibility patches will need to be applied. The XMPP application is an OCS 2007 R2 server role and all OCS 2007 R2 services need various Microsoft patches in order to function correctly on Windows 2008 R2.

The following is the list of updates that should resolve the TLS errors between the XMPP and Lync Edge server: