Getting the best connectivity and performance in Office 365

‎11-06-201712:02 PM

Traditional enterprise networks are designed primarily to provide users access to applications and data hosted in company operated datacenters. A secondary use has been as a gateway for access to the Internet for communications and web browsing. In this model, there is minimal or no network security between users and the company operated datacenters, and a substantial security perimeter between users and the Internet with many network devices such as firewalls, anti-virus scanners, data loss prevention, and intrusion detection devices.

Because of the large network security stack, Internet connectivity for branch offices is commonly centralized and backhauled over the customer’s wide area network (WAN). This model worked well for secure access from users within the office to corporate on-premises apps such as email and document sharing where bandwidth could be assured, and minimal network security within the WAN meant network traffic was not impeded.

As enterprise adoption and reliance on SaaS apps like Office 365 continues to grow, and as employees work from more varied locations, the old methods of backhauling traffic to a central location for inspection creates latency and leads to a poor end user experience. The shift from accessing enterprise applications in a customer operated central datacenter to Office 365, and the differences in traffic patterns, performance requirements, and endpoint security needs to be acknowledged and planned differently when compared with simple Internet communications and web browsing research connectivity.

The Microsoft global network and Office 365

The Microsoft global network is one of the largest network backbones in the world consisting of high bandwidth links that have minimal network congestion, with thousands of miles of privately owned dark fiber, multi-terabit network connections between datacentres, and application front doors servers spread around the world. Over 100 public Internet peering interconnection locations on this network makes it easy for all users, regardless of location, to connect into the network using the Internet and access services such as Office 365, Azure, Xbox, Bing, Skype, Hotmail and more.

Microsoft continues to invest in the network, the geographical locations of the application front doors, public peering partnerships with ISP’s, and traffic backhauling capabilities. This allows user network traffic to enter the Microsoft global network very close to the user, and then the traffic is backhauled at Microsoft’s cost over high bandwidth lines within the network to the location where the user’s data is stored.

Microsoft global network with each of the blue dots representing Office 365 front end servers around the world

Office 365 connectivity principles

Microsoft recommends using the Internet and a simple network design for optimal connectivity and performance in Office 365. A key goal in the network design should be reducing the round-trip time (RTT) from your network into the Microsoft global network and ensure that the network traffic is not hair pinned or centralized to specific locations. Use the Office 365 connectivity principles to manage your traffic and get the best performance when connecting to Office 365.

Office 365 URLs and IP addresses aka.ms/O365IPAs a SaaS application Office 365 has a large number URL’s and IP Addresses representing Office 365 service front end servers. We refer to these URL’s and IP addresses as endpoints and customers can use them to identify specific network traffic that is destined for Office 365.

Identifying Office 365 network traffic is the first step in being able to differentiate that traffic from generic Internet-bound network traffic. Microsoft publishes the Office 365 endpoints and guidance on how best to use this data. An Office 365 administrator can use a script to fetch the endpoint details and apply it to a perimeter firewall and other network devices. This will ensure that traffic bound for Office 365 is identified, treated appropriately and managed differently to network traffic bound for generic and often unknown Internet web sites that employees may browse. See the Office 365 endpoint categories and Office 365 IP Address and URL web service to automate endpoint management.

2. Egress Office 365 data connections as close to the user as practical with matching DNS resolutionLocal Internet egress into Microsoft's networkMany enterprise WANs are designed to backhaul network traffic to a central company head office for processing before network egress to the Internet. Because Office 365 runs on Microsoft’s large global network that includes many front end servers around the world, there will often be a network connection and front end server close to the user’s location.

When compared to backhauling data across the corporate WAN, the user is most likely going to get better performance by egressing Office 365 network traffic to the Internet close to their location where it can be connected to Microsoft’s global network. Additionally, many Office 365 applications use DNS requests to determine the user’s geographic location. If the users DNS lookups are not done at the same point as the network egress the user may be directed to a distant Office 365 front end server.

By providing users with local Internet egress and local DNS resolution their network traffic destined for Office 365 can connect to Microsoft’s global network and Office 365 front end servers as close as possible to the user. Shortening the network path to Microsoft’s global network and to Office 365 front end servers in this way should be expected to improve connectivity performance and the end user experience in Office 365.

3. Avoid network hairpins and optimize connectivity directly into the nearest entry point into Microsoft's global networkEnterprise network hairpinning Office 365 bound Internet trafficMicrosoft is continuously working on reducing the distance between users and Office 365 endpoints, driving down latency and improving end user experience. There are two types of network route hairpin that may occur in connecting users to Office 365. These network hairpins greatly lengthen the network path between a user and Microsoft’s global network, and this increases network latency and reduces performance of Office 365.

As discussed, the second type can result from a cloud based network security infrastructure device. If the network device vendor has limited hosting locations and directs a user to a specific one that is distant from them they may create a hairpin route where network traffic goes from the user to the distant network device and back to an Office 365 front end server that is near the user. This can be avoided by asking cloud based network security vendors about the specific locations of their hosting and being critical of the network paths that this creates that may be different to the direct route to Office 365 endpoints on Microsoft’s global network.

The first type results from misaligned network egress and DNS lookups for a user. This can result in the user being directed to an Office 365 front end server that is close to them, but via a distant corporate egress location at a head office. This can be avoided by local egress and local DNS as outlined in the principle above.

Generic Internet web browsing traffic to unknown Internet sites can have substantial security risk and most enterprises implement network security, monitoring, and traffic evaluation technology at their Internet egress locations. Network security technology includes proxy servers, inline SSL break and inspect of network traffic, network layer based data loss prevention, and more. Network security devices is a strongly growing industry. Unfortunately, whilst all this equipment reduces the enterprise risk of Internet connectivity, it also increases the cost and resources required for Internet connectivity, and it reduces the performance for network connections.

Office 365 servers are all hosted in Microsoft datacenters and Microsoft is very transparent about datacenter security, operational security and risk reduction around those servers and the network endpoints that they represent. These security details can be found in the Microsoft Trust Center. Office 365 also has many other methods available for reducing that network security risk including the built-in security features in Office 365 such as, Data Loss Prevention, Anti-Virus, Multi-Factor Authentication, Customer Lock Box, Advanced Threat Protection, Office 365 Threat Intelligence, Office 365 Secure Score, Exchange Online Protection, Network DDOS Security, and other many other security features.

Enterprise customers should review these risk reduction methods specifically for Office 365 bound traffic and use the Office 365 in-built security features to reduce the reliance on intrusive, performance impacting, and expensive network layer security technologies for network traffic that is identified as Office 365.

The Office 365 networking product group would like to learn about your networking challenges when connecting to Office 365. Please comment on this blog to start a conversation.

There are many Cloud connectivity Service providers or Internet access Security providers claiming to have optimized connectivity with Office 365, having direct network peering with Microsoft datacenters or present in the same datacenters as Microsoft, having local DNS resolutions of Office 365 services etc.... Is Microsoft doing such kind of arrangements with such providers or it's only with ISPs?

Also in case of geographical spread organization with Office tenant in one region, the traffic in any case has to travel to central tenant location. What is the best way to connect:

- Connect directly through Internet in different geographical locations. ISPs connect you to local MS datacenters and then reach your central tenant.

Or

- Route your office 365 traffic on WAN to the location nearest to Tenant and then reach your tenant from there.

Hi @rajesh arora - Microsoft has peering agreements with over 2500 ISPs in 150+ peering locations around the world where they can exchange traffic into the Microsoft 8075 network https://aka.ms/8075. The recommended way for optimal Office 365 connectivity is to egress locally using the Internet, and perform DNS name resolution locally, which then connects the user into the nearest Office 365 application front door, after which the connection travels on the Microsoft global network to the datacenter location where the user's data is stored.

What you mentioned is about the "Client Connectivity", but what about Server to Server connectivity. For Exchange & Skype for Business Hybrid integrations, the on premise servers need to connect with Online servers and vice versa. What is the recommended way to connect and why? Direct Internet or Peering/ExpressRoute? This is especially important from Security point of view as well since exposing your servers direct on internet (through reverse proxy) is not secure and also brings reliability/robustness issues into picture (e.g DOS attack on servers). A dedicated connectivity can prevent these type of issues. Please share your thoughts around that....

Our recommendation for most customers is to use the internet path for Office 365 traffic as ExpressRoute is inherently complex to implement and invariably the internet path provides the optimal method in terms of cost and complexity. You also may want to think about connectivity which is required over the internet, for example if you move Exchange or Skype Hybrid endpoints to be accessible over ExpressRoute, they are then not reachable to your users or other entities you want to expose them to publicly. As an example, if you move Autodiscover to be only exposed over ER then your users would have to VPN back into the corporate network to get their mail in some scenarios.

Even when a customer uses ExpressRoute, we generally recommend inbound (MS to customer) connections are kept on the internet path to keep the service availability the same and also simplify the routing model.

From a security standpoint, we publish the IPs used for the service so access can be restricted to Microsoft endpoints where appropriate.

"An Office 365 administrator can use a script to fetch the endpoint details and apply it to a perimeter firewall and other network devices."

Has this been actually observed in the wild? I tried this, and engaged my (not cheap) VAR for support, and they said it was too complex. The manufacturer said it was too complex. I'm managing the 1,400+ IPs and URLs by hand.

Hi @david williams - We are making it simpler for customers to identify, differentiate and treat Office 365 endpoints much more smoothly. Please see the new announcement at aka.ms/ipurlBlog. + @Paul Andrew for awareness.

Thanks Paul for this article.
With you permission i will translate it in French for my customers.
This article is useful and actually we have to cope with in a 40 K users 0365 international onboarding. Geo location helps but not always. In some places especially in south Asia we have poor connectivity to the first Microsoft entry point. For security reasons we will also deploy Zscaller services in an large scale but we know that it will not fix all our bandwidth and latency issues. Bandwidth and latency is still an issue less for Outlook than Skype for Business (Sorry Teams ;-) and choosing 0365 will implies additional challenges and will also challenge the old network configurations. Lost of customers do not envision that before launching their 0365 unfortunately.
Laurent TERUIN [MVP]
lteruin@hotmail.com
Global architect in a (NonDisclosure) company.;-)

Awesome to see MS providing such detail about something we ran into several year ago.

I wrote an article about this specifically for Outlook experience to O365 and we wrote an endpoint testing tool to help organizations understand if their physical users/campuses are getting served "nearest endpoint" values from DNS.

One thing that i think was left out was the importance of choosing an external DNS resolver that is location specific. We first hit this issue because a customer had their internal DNS servers forwarding to Googles 8.8.8.8 hosts and for which Microsoft cannot really determine geo-location from.