It can sometimes be tricky to get Exchange push notifications working. The development of a listener is only part of the solution. The other part is getting the notifications from the Exchange server to the listener, especially when the listener expects the notifications on a secure channel. Exchange is of no particular help here. The only error message that is generated when Exchange can’t send a notification message to a listener is something like this:

This message is not helpful at all. Ok – Exchange could not send a notification a specific subscription. But why?

That’s why I wrote a small utility that should be able to diagnose the three most common problems:

The Exchange server cannot reach the notification listener because a firewall is blocking the access.

The host specified in the callback address can not be resolved to an IP address

If the callback address of the listener is secured using TLS/SSL, Exchange will not send notifications if the server certificate used by the listener is not trusted.

Instructions:

Copy the program on the Exchange server. Open a command prompt and start the program. It expects the address of the listener as the first parameter. The execution of the program can take up to one minute.

A requirement I often hear is something like this: “We have a mailbox that receives emails of some kind that need to be processed somehow”. What options are there to fulfill the requirement? As always, it depends. When the items need to be processed as soon as possible and the client processing the items can be directly reached by the Exchange server, push notifications are certainly a good choice. If, on the other hand, item processing is not time critical a suitable method of processing those items is polling. It’s much simpler to use than push notifications. A simple example might look like this:

This is a very naïve implementation as it always returns every item from the inbox folder of the mailbox. But at least it uses paging. This breaks the processing down from one very large request into many small requests (in this case 100 items are returned per request).

This method is suitable if you delete the processed items from a mailbox after they are processed. Of course, the items should not be removed until the every item has been processed. Otherwise items may be skipped since the offsets of the individual items change when an item is removed from the folder.

If the items are not removed from the store, the calling application must distinguish new items from items already processed. An obvious way to do this is to mark each item once it has been processed by marking it as read. This changes the requirement for the ProcessItems method: It should only process unread items. This modifications has been incorporated into the following example:

The only change in this example is in line 11: A search filter has been added to filter for unread items. What’s missing here is the modification of the Unread status of each message. And that is a considerable drawback of the whole solution. To touch each processed item, the UpdateItems method must be called. It is sufficient to call the UpdateItems method and pass all item ids from the last resultset to the method. But this adds a significant burden on the Exchange server and slows overall processing down. Furthermore, if someone accesses the mailbox (either with Outlook or Outlook Web Access) and accidently marks one or more items as read, those items will not be processed by the client application.

Next idea: Find all items that were received after the last time the mailbox was checked. A ProcessItems method that implements this behavior might look like this:

As with the second example, the difference to the first one is line 11. A search filter has been added that restricts the FindItems call to those items received after a specific time. This method removes the requirement to mark each processed item on the server. But it adds another caveat: A mail that is received during a ProcessItems call will be missed if no additional checks are performed. This can be a little tricky.

Luckily, the Exchange WebServices offer a more suitable solution for the whole requirement: The SyncFolderItems method. This method not only solves the problems mentioned above but also returns deleted items, if such a processing is necessary. A ProcessItems method that uses this API now looks like this:

This method only processes newly created items and ignores all other item changes or deletions. The application calling this method needs to store the synchronization state between each call. If an empty sycnState is provided, Exchange will return every item from the mailbox as “Created” item. This makes it possible to process all existing items once and then, with the same logic, every changed item. The only drawback with this method is the fact that the synchronization state can become quite big (in a folder with ~4500 items, the syncstate has a size of approx. 60kb).

Under certain circumstances a program needs to determine the name of the computer it’s running on. The first approach to get this name is to use the System.Environment.MachineName property. However, this name only reflects the NETBIOS name of the current machine. But in larger environments a full-qualified name including the DNS domain the computer belongs to. This can be something like computername.contoso.local. One example where this full qualified name might be needed are Exchange Push notification. I’ve published a component to CodePlex makes it really easy to incorporate them in an application. However, for the notifications to reach the client the component needs to tell the Exchange server a correct callback address. In a very simple network environment, it is sufficient to specify the NETBIOS hostname. But in more complex environments, Exchange might not be able to send a notification because it cannot correctly resolve the unqualified hostname to an IP address.

I have just published a new version of the my notification component on CodePlex. The new version has some breaking changes to the previous version but it should be simpler to use. Additionally, I have added a small sample application that shows how to use the component. It’s a small WPF application that allows a user to subscribe to calendars of multiple users. The application uses the new Managed API which made the whole thing a whole lot easier to write. Here is a class diagram of the component:

If you have worked with the component before you’ll notice that the Subscription class has lost some of its members. I have removed those methods to avoid confusion on how the component is utilized. The only way to create a new subscription now is to use the SubscriptionCollection class. Here is an example on how to use the component:

This is the fourth article about the .NET component I published at CodePlex recently. To all related articles, click here.

If you subscribe to a folder on an Exchange mailbox or public folder, the server will try to send notifications to the registered endpoint of yours upon every modification. If your application is shutdown, Exchange will try to reach it for a certain amount of time. Once this time span has elapsed, it will delete the subscription. If the application is restarted and a new subscription is created, it will be notified about all subsequent events. But for certain use cases, it is imperative that no events are missed – even events which occurred between application downtime must be catched. One example for this is a synchronization applications which keeps a sql server database (like a CRM application) and Exchange folders in sync.

For these types of scenarios, all Exchange events carry a watermark with them. This watermark is opaque to the client, but for the Exchange server it contains enough information to reconstruct changes made since the watermark was received. To use this features, an application has to save each watermark it gets. The watermark can be sent to the Exchange server along with the subscription request for a folder. Exchange will then replay the events which happened since that watermark has been generated.

I’ve implemented this feature in my Push Notification component in the SubscriptionCollection class. It can be serialized to either a System.IO.Stream or System.Xml.XmlWriter. The latter one uses the System.Runtime.Serialization.DataContractSerializer to perform the serialization. And the way it is used, it requires .NET Framework 3.5 Service Pack 1. If you cannot rely on having this version installed on your clients machines, you should probably use the binary serialization instead.

Here is a small code sample on how to use the serialization feature (this is an example only, so the exception handling is far from optimal…)

83: Console.Out.WriteLine("Subscription restart is complete. {0} subscriptions could not be restarted.", args.Result.Errors.Count);

84: }

85: }

86: }

This essentially is the sample from my first blog post in this series, but enhanced to take advantages of the restart capabilities of the SubscriptionCollection class. The program checks to see if there is a saved state from a previous run available. If found, it is deserialized and restarted (lines 35 to 43). Otherwise a new instance is created. Finally, the events are wired to the individual subscriptions (line 57 to 62). And once the application stops, the current state is serialized to a file.

Since the subscriptions are restarted asynchronously in the background, the SubscriptionRestartCompleted and SubscriptionRestartProgressChanged events are used to keep track of the deserialization process. The progress changed event handler is raised for each subscription restarted. Once complete, the SubscriptionRestartCompleted handler is raise. The event args this event handler contain information about all failed restart attempts.

This is the third article about the .NET component I published at CodePlex recently. To all related articles, click here.

The PushNotificationListener I created for that component uses a WCF endpoint to receive notifications. This means that it needs to open an TCP endpoint of some sort. Thanks to WCF, all the hard stuff is done by the WCF infrastructure. There are, however, two issues which must be resolved before you can receive notifications from your Exchange Server: The Windows Firewall, which blocks all incoming traffic by default and the WCF permission system. To deal with the former, you need to setup an exception for a specific port or for your executable. If you are using Windows Installer Xml, you can use the Firewall Extension to create such an exception during the setup of your application. If you want to do this directly from your application, you will need administrative rights and perform some interop stuff (see the links at the end of the article).

To open an WCF endpoint, you need administrative permissions too, by default. Because this is a very nasty requirement, the system allows you to create so-called reservations which can be tied to specific users. You can use nethsh.exe to manipulate these permissions. You can also use the httpcfg.exe tool. Or, you save you the headache that comes with these tools and head over to Paul Wheelers blog and take a look at this blog post of him: AddressAccessDeniedException: HTTP could not register URL http://+:8080/<…>. He has published a small tool (full source included) to enumerate and manage WCF port reservations.

Here is a screenshot of the reservations on my machine:

The really only real interesting endpoint here is the second from the bottom: http://+:80/Temporary_Listen_Addresses. This is a reservation in the form of an UrlPrefix String (more about that here on MSDN) that can be used by everyone (and Dominick Baier on leastprivilege.com has a nice post about the security ramifications here). However, the reservation is there and we can use it for the PushNotificationListener. This is the reason why the default port is 80 and the RelativePath property is set to a folder below the Temporary_Listen_Addresses. If you need to use another relative path or even another port, you’ll have to create a reservation for it. You can take a look at the source code of the HttpNamespaceManager tool from Paul Wheelers Blog on how to do this. Again, this is probably best done during setup. Again, if you are using Windows Installer Xml, you can use a managed custom action (using the DTF Framework) to create the necessary reservation during the setup of your application.

Controlling the Windows Firewall with C#

Here are a few links I just found on the internet. I have tested none of them, but they might give you a hint:

As I wrote in my earlier post, I recently published a component on CodePlex that simplifies using push notifications in your applications. Since I’ve not published a full fledged sample application, I will start with a series of blog posts. I will tag all related articles on this topic with “push notifications”, and you can get a list of all articles with this link: http://www.infinitec.de/?tag=/push+notifications.

I’ll start with a very simple application: A console application that creates one subscription get notifications about new mails arriving in the mailbox of the current user.

1:using System;

2:using System.Net;

3:using InfiniTec.Exchange.Notifications;

4:

5:namespace ExchangeNotificationTestClient

6: {

7:internalclass Program

8: {

9:privatestaticvoid Main()

10: {

11:// Ignore any certificate errors

12: ServicePointManager.ServerCertificateValidationCallback +=

13: (sender, certificate, chain, sslPolicyErrors) => true;

14:

15:// Setup the adapter which will be used to call into the Exchange WebService

16: var adapter = new ExchangeWebServicesAdapter(

17:new Uri("https://casserver/ews/exchange.asmx"),

18:new NetworkCredential("administrator", "password", "contoso"));

19:

20:// Create a new subscription collection to manage all the subscriptions

21: var subscriptionCollection = new SubscriptionCollection(adapter);

22:

23:// Setup a listener that listens on port 80 on the local computer

24:using (var listener = new PushNotificationListener())

25: {

26:// Register for a NewMail notification on the inbox of the administrator

The ExchangeServiceAdapter (created in line 16) is used to actually perform the Web Services calls to the Exchange Server (CAS Role). It’s a rather simple implementation, as it does not support AutoDiscover. You have to specify the Exchange server manually. It does, however, support Exchange Impersonation. And if you want to subscribe to events on public folders, you’ll have to enable this feature by setting the ExchangeServiceAdapter.IsPublicFolderAccessEnabled to true. Your Exchange Server must have Service Pack 1 installed to use this feature.

The PushNotificationListener (created in line 24): This class does all the necessary WCF plumbing to setup a host, receives the notifications and channels them to the right subscription.

The SubscriptionCollection: (created in line 21) It’s not really necessary, but it makes is more easy to handle multiple subscriptions.

The Subscription (created in line 27). You can either create a subscription by calling SubscriptionCollection.Add(), or create a subscription directly via “new Subscription()”.

Both, the PushNotificationListener and the Subscription need to be started to do some actual work. And that’s all you need to do to get a simple notification client up and running. However, there are some security settings to consider, because WCF doesn’t let you run around and open endpoints on your users machines. Additionally, the Windows Firewall needs to be configured correctly to let the notifications through. I’ll discuss this in a separate post.

Exchange 2007 de-emphasized Exchange Event Sinks in favor of a new notification system which allows an application to receive notifications about changes made to items on a public folder or mailbox folder. There are two types of notifications: Pull and Push. With pull notifications, the calling application is responsible to get changes back from the server (it has to poll the CAS role for changes). Push notifications on the other side use Web Services to call into an external application. There are a restrictions where this does not work (for example firewalls blocking traffic), but otherwise they work really well.

Compared to the WebDAV notifications that where available in earlier versions of Exchange, they have the advantage that the external application gets specific information about the event: Which item has been modified, created, deleted?

However, it’s rather complicated to completely implement a listener for these notifications. Therefore, I have created a wrapper which uses WCF to listen for incoming notifications.