In my last post I created a custom serializer that could be injected into the WCF plumbing between the dispatcher and the service class that allowed us to debug errors that occurred during serialization. This operation behaviour could be applied to a method signature in the service contract :

However, what if I wanted to apply this behaviour as a ServiceBehavior, ContractBehavior or EndpointBehavior rather than an OperationBehavior i.e. if I wanted all the operations in my service implementation or interface to use it and I don’t want to have to go through and add the attribute to every method ? How can I use my [DebuggableSerializer] attribute to decorate a service class, interface or even better, apply the behaviour to the service or endpoint using a config file so I don’t have to change my code at all ?

So, in this post I am going to extend my [DebuggableSerializer] attribute so it can be used on a service class or interface and I am going to extend the System.ServiceModel.Configuration.BehaviorExtensionElement to allow me to control and apply that behaviour from my config file to either a service or a specific endpoint.

My attribute in the previous example looked like this :

public class DebuggableSerializer : Attribute, IOperationBehavior

but now we want to use it as a ServiceBehavior, ContractBehavior and an EndpointBehavior as well as an OperationBehavior, so we’ll implement the IServiceBehavior, IContractBehavior and IEndpointBeahvior interfaces :

We also need to provide 3 overloads of the ReplaceDataContractSerializerOperationBehavior method, one that takes ServiceDescription, one that takes ServiceEndpoint and one that takes ContractDescription instead of OperationDescription. The method that takes ServiceDescription will call the method that takes ServiceEndpoint for each Endpoint and the method that takes ServiceEndpoint will call the method that takes ContractDescription and the method that takes ContractDescription will call the method that takes OperationDescription for each OperationDescription in its contract.

We now have an attribute that can be used to replace the serializer on a service contract or an operation contract or a service operation or on a whole service class or an endpoint of a service. The final step is to be able to add this behaviour to a service or an endpoint from within our configuration rather than in code. For this we need to create a BehaviorExtensionElement that will create an instance of our DebuggableSerializer behaviour. This is very simple :

Like this:

As if starting developing with WCF wasn’t a hard enough paradigm shift what with getting used to sharing schema instead of type and being cruelly abstracted away from all our lovely plumbing, the abstraction itself it can throw some interesting curve-balls. It’s all very well taking the plumbing and abstracting it away but what happens when errors occur down there…out of our reach ? The errors may still be in our code but if they are being hit by the DataContractSerializer for instance it’s hard to handle them or even catch them, for that matter. To illustrate the problem and provide a possible solution, here’s an example.

Now, we’ll introduce an error in the service that will not be encountered until the Data Contract class is serialized, i.e. it will occur out of scope of the actual service method call. We are going to change this line in our GetThing service method :

case Flavour.Banana:
return new Thing("Bernice", Flavour.Banana);

to this :

case Flavour.Banana:
return new Thing("Bernice", (Flavour)9);

So we are sneakily casting the integer value 9 into the integer-based enum Flavour even though it’s not a valid value of Flavour. I have actually seen a live version of this error where some unexpected integer values got into a database column so the line actually looked more like this :

return new Thing("Bernice", (Flavour)reader.GetInt32(3));

When the service is run and called by the client no error is seen in the service even when debugging and the error appears on the client as a timeout. So, imagine this error is intermittent because it only occurs in a very few out of thousands of database records. Some clients are complaining of timeout errors even though the service isn’t busy and there are no errors on the server side. We can’t debug this and there doesn’t seem to be anywhere we can put a try-catch block. What we really need to be able to do is debug into the DataContractSerializer but we can’t. One option is to enable tracing in the service diagnostics. This can be done either using the WCF Configuration Editor :

If we run the service again and make the same call we can view the resulting trace log in the Service Trace Viewer :

There we can clearly see the exception that occurred. The great thing about the tracing and the Trace Viewer is that if we run tracing on all the consumers and services for a particular activity and enable activity propagation we can get the trace logs from all participants and the Trace Viewer will stitch them together for us so we can see the whole thing end to end and step through.

This approach is great in many instances but sometimes when debugging it’s a bit "after the fact" and actually what you’d like is to be be able to set a break point or have the debugger catch it in the act so you can get in and look at the stack.

To help with this we really need our own serializer but the serializers (DataContractSerializer and NetDataContractSerializer) are sealed. However, XmlObjectSerializer that they both inherit from is not and it is this type that is used so maybe be can hook in there somehow. I can create a class that inherits from XmlObjectSerializer but I don’t want to write my own serializer…way too much work. However, I can use this class as a wrapper for the actual serializer and delegate to it. This effectively injects my debuggable code into the hard-to-debug WCF plumbing. Here’s my DebuggableDataContractSerializer :

Now I need to get my serializer swapped out with the real one. I remembered seeing a blog post by Aaron Skonnard way back in April ’06 in which he showed how to substitute the NetDataContractSerializer for the DataContractSerializer so let’s try that here :

Like this:

One of the many great things about WCF which you don’t often hear people talking about is that it comes fully instrumented out of the box. That’s right, the WCF runtime has full instrumentation baked right in, ready for you to use to performance profile and monitor your services and all you have to do to start using it is flick a switch in your service config and run performance monitor (perfmon.exe).

If you’ve been a serious windows user for any length of time you will almost certainly have come across perfmon. It is one of the essential tools for quick, easy system and application profiling and many applications (like SQL Server, MSMQ, the CLR, the WCF Runtime, ASP.NET etc) come with their own specific perfmon counters which let you peek under the hood when they are running. Perfmon lets you capture and display data generated by these counters, monitor customized thresholds and take actions when thresholds are breached. It is a massively useful tool.

WCF adds four sets of counters to the system. The first three are ServiceModelService, ServiceModelEndpoint and ServiceModelOperation. As you can probably guess from the names, these counters give you different levels of granularity when profiling so you can capture data at the service level, at the endpoint level and at the individual service method level. The final one is SMSvcHost. This is a set of instrumentation counters attached to the .NET TCP Port Sharing Service that lets different services listen on the same port and I will leave this set for another time.

The first three contain the counters listed below (or a subset thereof) :

Calls

Call Duration

Calls Failed

Calls Failed Per Second

Calls Faulted

Calls Faulted Per Second

Calls Outstanding

Instances

Instances Created Per Second

Queued Messages Dropped

Queued Messages Dropped Per Second

Queued Messages Rejected

Queued Messages Rejected Per Second

Queued Poison Messages

Queued Poison Messages Per Second

Reliable Messaging Messages Dropped

Reliable Messaging Messages Dropped Per Second

Reliable Messaging Sessions Faulted

Reliable Messaging Sessions Faulted Per Second

Security Calls Not Authorized

Security Calls Not Authorized Per Second

Security Validation and Authentication Failures

Security Validation and Authentication Failures Per Second

Transacted Operatoins Aborted

Transacted Operations Aborted Per Second

Transacted Operations Committed

Transacted Operations Committed Per Second

Transacted Operations In Doubt

Transacted Operations In Doubt Per Second

Transactions Flowed

Transactions Flowed Per Second

Now, I mentioned the flick of a switch in the service config to enable this feature which is off by default, so here it is :

That’s all that is required. As you can see, it is also possible to provide access to this instrumentation via WMI by switching the value of the wmiProviderEnabled attribute to true so you can write your own monitoring tools and scripts or use other 3rd party tools that use WMI.

So, here are the facts. The service is configured as a singleton. I have inserted a Thread.Sleep(200) into both methods to simulate more complex activity. The client is multi-threaded (via the thread pool) and will create 100 proxies which will randomly call one of the two service methods.

What is this likely to mean in performance terms ? Well, firstly services are inherently single threaded meaning that they will only process one call at a time. Therefore, if the call duration on our service is 200 miliseconds (give or take) then the best performance in terms of calls processed per second we can hope for is 5.

So let’s profile our service and see if we’re right. Here’s our output :

As you can see our 100 proxies made 100 service calls and our singleton made sure that there was only ever one instance processing those calls so instances created per second briefly hit 1.

Call duration averaged at .0201 seconds thanks to our Thread.Sleep(200) and as a result calls max per second was 5.009.

So what happens if we reduce our call duration to 100 milliseconds (Thread.Sleep(100)) ? What we should see is calls taking half as long and therefore being processed twice as fast.

The graph is pretty much identical except that max calls per second is now 10 and the 100 calls have been processed in half the time. As predicted.

You can continue this testing increasing the sleep latency up to somewhere between 800 and 900 milliseconds after which the client will throw an exception when it fails to connect with the service with in the default 1 second timeout. After that you either have to increase the timeouts on the consumer, increase the ListenBacklog or MaxConnections (which both default to 10) on the service, reduce the latency of the service, reduce the number of concurrent calls, abandon the singleton in favour of PerCall instance mode or set the ConcurrencyMode of the singleton to Multiple so that it can be accessed by multiple threads.

<gotchas>

1) If you want to monitor a specific service instance then the instance must be running when you go to add the counters to the graph.

2) When the service instance stops running the graph will freeze (i.e. stop capturing data) but will restart when you restart the service.

3) Setting up and configuring perfmon graphs can be time consuming so you can save them as counter logs, however you can also save a graph as html which will produce an html page with an embedded perfmon graph complete with all your counters and configurations. You can just open this page in your browser to start monitoring. This also means you can build and keep or publish a library of useful graphs.

Warning

When this blog got transferred from LiveSpaces some of the code formatting got messed up. I'm fixing this when I can be arsed (which isn't often) but if there's something bugging you let me know and I may have mercy.