Warning: This post is about a beta product. Details might change before final release.

The beta of BizTalk Server 2009 brings a welcome new feature: Some basic support for Unit Testing BizTalk artifacts; specifically for Pipelines, Maps and Schemas.

You can read the basics about how the new feature works in the product documentation; there’s not too much yet, but it’s enough to get a good idea of how you use it.

I’d like to now share a few thoughts on this feature. Note: This is just my personal opinion from playing with the API for a bit.

The Good

The first really good part about this feature is the fact that it even exists at all. Facilities for creating and automating tests of BizTalk artifacts has been a sorely lacking area of the product.

While Visual Studio had the Test option for maps and schemas since BizTalk 2004, and it was sort of useful, it required way too much manual intervention and there was no way to decently drive it through automation, making it useless for either unit or regression testing.

The second thing I like about the unit testing feature is that, in it’s current incarnation, it’s not intrinsically tied to MSTest (i.e. the Visual Studio Team System stuff). This means you can use it with NUnit, xUnit, or your favorite testing tool.

The third interesting thing about the API is that it is a very simple API. In most cases it really consists of just a single method call. Here’s an example of testing a schema:

The Not-so-Good

Not everything is rosy in BizTalk-land. Personally, I was somewhat underwhelmed by the implementation of the Unit Testing features available in beta 1; particularly since some of it could be addressed with fairly minor adjustments.

API Issues

Let’s start with the API: I mentioned before that the new testing API was simple. However, it’s actually too simple. For example, consider the small test for a schema I presented above: You simply create an input file to test with, call ValidateInstance() and check the result. Great, no?

Well, if the test succeeds, then yes, it’s just fine. If it fails, however, because the input file does not pass validation, then, it turns out, it’s missing one extremely important functionality: There’s no way to get any details about why the schema validation failed [1].

This means that diagnosing a failed test involves, at the very least, changing your project settings in VS to use the failed input file as input to the schema in VS, use the Validate Instance option in the IDE and examine the results in the output window, where fortunately you do get more detailed error information.

This is extremely sub-optimal and requires way too much manual work. Can you imagine if a change in a schema broke tens of tests and having to do this for each and every single one of those? Ouch!

Another place where I feel the current API is not as nice as it could be is that it’s strictly file-based. That is, you need to create actual files on disk to represent inputs to the tests and store the resulting outputs. This is somewhat convenient for many scenarios, and matches the existing functionality in BizTalk pretty well. It does, however, make it very inconvenient to deal with dynamically generated inputs or inputs stored someplace else.

The classic example here, would be, resources. For example, I very much like to keep my unit tests as independent from the environment and as self-hosting as possible. One very nice way of doing this is to store necessary input files as embedded resources on my testing assembly. I do this all the time with PipelineTesting and it works great. However, since you can’t provide streams instead of file paths to the BizTalk 2009 testing features, you’d need to first save the embedded resource to a temporary file, which adds a bit of friction to the process.

Compilation Changes

My biggest gripe with the Unit Testing features, is that enabling them implies a change in the build process, that changes the generated code for the BizTalk artifacts themselves.

When you enable the Unit Testing option in Project -> Properties -> Deployment, the compilation process will change the generated CLR types so that they inherit from the TestableXXX classes defined in the Microsoft.BizTalk.TestTools.dll assembly, instead of the normal BizTalk artifact classes.

These new Testable base classes, in turn, derive from the original BizTalk classes, so the change isn’t all that big. However, it does smell wrong. Could it introduce any problems/bugs if you were to enable the testing features for all builds? Hopefully no, but no way to know until you run into them.

Of course, you could just enable Unit Testing for debug builds, but then you can’t run your unit tests on the release builds of your BizTalk assemblies. It also introduces yet another asymmetry between development and deployment builds which, to be honest, makes me somewhat nervous.

Conclusion

The new Unit Testing features in BizTalk Server 2009 are a sorely needed and welcome feature, if somewhat lacking in the first beta.

Will it improve? On one hand, I’m somewhat optimistic that some improvements might make an appearance in forthcoming builds. However, I’m also not getting my hopes up, as MS is notable for trying to avoid significant API changes during after beta 1 hits the street (BizTalk, unfortunately, doesn’t use the CTP model, where significant API change requests are more likely to be considered).

[1] TestableMap.TestMap() does throw exceptions on failure instead of a Boolean return value, but said exceptions might not contain detailed error information.

Just a quick post to say Happy New Year to all of the BizTalk community. I was planning on writing about the BizTalk 2006 R2 to 2009 upgrade experience, but I managed to forget my external drive so I can’t start the VM… instead you’ll have a settle with a beautiful picture of York […]

In this short WCF screencast, you'll learn how to configure your WCF services with Metadata Exchange (MEX) endpoints – both over HTTP and TCP – allowing consumers to automatically retrieve metadata from your services at runtime. Enjoy!

Be sure to check out our growing collection of short screencasts on the Pluralsight screencast page.

In this short WCF screencast, you'll learn how to configure your WCF services with Metadata Exchange (MEX) endpoints – both over HTTP and TCP – allowing consumers to automatically retrieve metadata from your services at runtime. Enjoy!

Background

I was recently reviewing a BizTalk application where there were around a dozen orchestrations which had been exposed as either SOAP or WSE web services. The typical pattern that had been used in these orchestrations is illustrated in the below picture. You can see that there is a Request/Response port and in the orchestration initialises based of the request message then returns a very simple acknowledgement type message (just a message of .net type System.Int32 with a value of 0) to indicate the orchestration has the message and is going to do some additional work.

This is a pattern I’ve seen implemented particularly with inexperienced teams. Often when I’ve seen this pattern the code also had poor testing and the team did not have much confidence in their implementation. They ended up with this pattern to ensure that the caller knew they had the message. The main issue the customer had was that occasionally they would experience a timeout at the client end particularly during load, it wasn’t a major issue as it only happened occasionally and the client could recover from this ok, but although the client got a timeout the orchestration still processed anyway and would make the appropriate updates to the original application to update it on the progress.

The Challenge

The challenge in this refactoring exercise was as follows:

The application was already in use so we are not able to change any of the contracts used by client applications. This means the WSDL for the published WSE and SOAP web services can not change.

We wanted to reduce the number of receive instances waiting around for a meaningless response and as result reduce the load a little

Before we get into the details of the refactoring, we will first look at the code samples from the previously generated SOAP and WSE services.

Existing WSE Web Service

In the below code snippet you can see that firstly I’ve had to censor some of the code from this article, but secondly it’s a standard web service generated with the WSE adapter. It inherits from the WSEReceiver class and invokes the receive method and indicates that it’s a 2 way call, then returns the response.

Existing SOAP Web Service

The below picture shows a sample of one of the SOAP web services generated from an orchestration. Again pretty standard generated web service using the SOAP generating wizard.

Additional Notes

In addition to what I’ve mentioned above, some other things which affect this refactoring are:

The web services are generated every time we run a build and they are also versioned

The Refactoring

The following sections will describe how we refactored the solution to implement our desired improvements.

Manually Implementing the Web Service Projects

The first thing we needed to do was to change the web services from being automatically generated to being two projects within the solution. We creates these two projects but with the request and response types being generated from schemas within the solution we wanted to maintain the ability to update these web services automatically so that any changes to schema were reflected in the web services. In the msbuild for the new web services projects we extended the build process so that C# classes were generated in the web projects representing the schemas using xsd.exe. In the below picture you can see we are generating these files into a folder called Contract within the web services.

After this we basically copied the artefacts from the generated web services to the manually build web service projects. We also needed to ensure the correct references were added.

Changing the SOAP Web Service

In each of the web services we needed to make a few small changes from what was generated. The first change (the top red circle) was to change how we assign the bodyTypeAssemblyQualifiedName variable. In the generated web services this is hardcoded by inspecting the assembly, but when we want to do this manually I added a reference to the schemas assembly and then just used the highlighted code snippet to get that name.

The second change (the bottom red circle) is the change to the call into the BizTalk assemblies where we change the oneway parameter to indicate that we are not waiting for a response. Previously the zero response message came from the orchestration meaning the messages route had been into the message box, into an orchestration, back into the messagebox and then finally back to the waiting instance to reply to the caller. By doing this as a one way call the code will basically wait until the message has been persisted to the messagebox and then continue.

You can see I then return my own integer of value 0 to indicate it has worked.

Note I’ve also implemented some custom logging to log any errors.

Changing the WSE Web Service

The WSE Web service changes are very similar to the soap ones. In the picture you can again see the change to how the schema qualified name is obtained. We have also changed the oneway parameter when we call the invoke method (although I’ve missed this off the right of the screen shot)

Changing the Orchestration

Now we are managing the web services as projects within our solution we need to change the orchestrations. The picture below shows we now have a one way receive port which is no longer late bound like previously, but is now direct bound based on the message type. By the time we have initiated the orchestration the caller already knows its request hit the messagebox and it does not need to wait around for confirmation that the orchestration has started.

Changing the bindings

We finally needed to update our binding file template to change the receive locations to be one way.

Testing

We had already implemented a number of BizUnit tests and were able to use these to confirm everything that had worked before the refactoring still worked exactly as it did now and that none of the web service contracts had changed.

Summary

In this post we discussed a common anti-pattern you seen implemented with BizTalk. We were able to take a pragmatic approach to addressing some of the issues the customer was facing with this implementation while minimising the affects to the consumers of services published from this application.

I’ve been looking around the new project system introduced in the BizTalk Server 2009 beta. As you might have heard before, it’s now based on MSBuild. This is an extremely welcome change from the obscure compilation model in previous BizTalk versions that caused so much frustration.

BizTalk Projects now look much like regular C# projects (they even have the same icon in solution explorer) but they can contain BizTalk artifacts like schemas, orchestrations, maps and pipelines.

Opening up the .btproj file reveals some of the differences and custom MSBuild tasks used by the new BizTalk project system. Two things are initially important to mark an MSBuild file as a BizTalk project.

So, in fact, a BizTalk Project is a C# project. Right now you can’t add C# artifacts to the project without editing the .btproj file by hand, but one can hope this will be supported in upcoming builds.

The second part important to making a .btproj is importing the BizTalk MSBuild tasks, which is accomplished by this line:

If we look at the BizTalkC.targets file, we can see that the BizTalk MSBuild tasks are implemented in the Microsoft.VisualStudio.BizTalkProject.BuildTasks assembly, which you’ll find in your ‘Microsoft BizTalk Server 2009\Developer Tools’ folder.

Global Project Properties

There are several properties that will get set in the project files depending on your project settings that are BizTalk-specific:

<BpelCompliance/>: This is a Boolean (true, false) property that indicates if the generated assembly should be marked as BPEL compliant (I think).

<EnableUnitTesting/>: Corresponds to the Enable Unit Testing option in the Deployment tab of the project settings dialog, and controls whether the unit testing features for pipelines, schemas and maps are enabled.

Other settings are per-user settings and stored in the .btproj.user file:

<Server/>: The name of the SQL Server instance that has the BizTalk databases for deployment.

<ConfigurationDatabase/>: The name of the BizTalkMgmt database.

<ApplicationName/>: The name of the BizTalk application you want to deploy to.

<Redeploy/>: Boolean property that indicates if you want to allow redeployments from within Visual Studio.

<RestartHostInstance/>: If true, the BizTalk hosts will be restarted after each deployment.

<Register/>: If true, the generated assembly will be registered in the Global Assembly Cache (GAC).

Per-File Project Properties

The per-user project file (.btproj.user) can also include a bunch of item-specific properties, which are visible when you select a file in Solution Explorer and open the Properties window.

In this category you’ll find properties for setting paths for map input and output files for testing/debugging as well as paths for input and output files for testing schemas.

Now that the a public beta release of BizTalk Server 2009 is out, I thought I’d download it and try building and testing PipelineTesting on the new release.

After downloading and installing the beta, I proceeded to import the existing Visual Studio 2005 project file into Visual Studio 2008 with the new BizTalk project system (a huge improvement over the old one, if I may say so [1]). The import worked just fine overall.

The only change I had to make was remove the reference to PipelineObjects.dll from the projects and add it again with the new version in BizTalk 2009. The new version seems to be slightly different than the one in R2 (it’s 3KB smaller), but I haven’t looked closely to see what the differences might be.

In either case, PipelineTesting builds perfectly with the new version, and all Unit Tests are still passing in the green!

[1] I did hit a small snag with the project system: When building the imported solution, only the SampleSchemas project gets built, though all are selected to build in the configuration manager. Had to manually build the other projects from the project context menu.