Overview

Bizmonade is a new project being developed to enable unit testing of BizTalk orchestrations,
in isolation of other components, without deploying the orchestrations to BizTalk
and without doing any configuration (bindings).

It allows running orchestrations directly from unit tests written in C#, using NUnit, MSTest
or any another .Net testing tool. The orchestrations therefore run in complete isolation from
the whole BizTalk environment, reducing the time needed to do quick tests of new orchestrations
or of modifications to existing ones. The orchestrations can be tested without waiting for
deployment scripts to run, and without worrying about technical communication details
(server adresses, authentication details, file system paths, message encoding, encryption, mappings...)

The goal of the project is to allow more agile BizTalk development process by giving a shorter feedback
cycle between modifications to orchestrations and execution of unit tests.
It allows a "Test Driven Development" model, where each small modification is tested immediately,
instead of "batching" multiple modifications together to test them all later. This makes it easier to
immediately see and fix problems introduced by a change, and is known to improve development productivity
in plain C# projects (by allowing developers to stay focused on small incremental changes and getting immediate
feedback on the impacts of these changes).

Examples

In general, tests written using Bizmonade use the following pattern:

Test sends a message to the orchestration;

Orchestration sends another message (a "request" to an external system);

Message is NOT sent to the external system, instead:

The test validates the message sent by the orchestration;

The test then builds a dummy "response" based on values from the request

Test sends the "response" back to the orchestration.

Test ensures the orchestration completed successfully.

The preview release includes a set of examples (in the Bizmonade.Examples.zip file).

Most examples use "boring" scenarios that are variations of the "Hello World" classic.
They don't represent real-world business problems, but instead focus on testing one specific
BizTalk functionality (the examples represent a large part of Bizmonade's unit tests).

The examples are split accross separate projects for Schemas, Maps, Orchestrations.
However, for some examples, the schemas are directly in the Orchestrations projects.
This was done this way to ensure that Bizmonade would correctly support both cases.
(Depending on the project, there are legitimate reasons to either separate schemas from
orchestration, or to include them in the same project)

Some of these examples are shown in this document, but other examples are available
within the preview release as well.

Simple Order / PO / Invoice example

[Test]
public void TotalForOrder413ShouldBe14_99()
{
// To test the SubmitOrder orchestration, the test must reference the C# class generated
// from that orchestration. Generated C# classes are named the same name as the original
// orchestration, with the "__Simulated" suffix.
OrchestrationSimulator.Test<SubmitOrder__Simulated>()
.When(MessageReceived.FromFile<PurchaseOrder>(
Path.Combine(exampleInstancesPath, "PurchaseOrder_413.xml")
))
// Ensure the orchestration publishes an updated "PurchaseOrder" with the correct price quote
.ExpectMessageSent<PurchaseOrder>
(
msg =>
{
// NUnit asserts, if using another test tool, substitute with that tool's syntax
Assert.AreEqual(14.99m, msg.DistinguishedFields["Total"]);
Assert.AreEqual("priceQuoteSubmitted", msg.PromotedProperties[typeof(orderStatus)]);
}
)
// Simulate that the client sent an "Approved Purchase Order" in response
// to the price quote.
.When(MessageReceived.FromFile<PurchaseOrder>(
Path.Combine(exampleInstancesPath, "PurchaseOrder_413_Approved.xml")
))
// Ensure the orchestration publishes the right Invoice in reaction to the approval
.ExpectMessageSent<Invoice>
(
// Not using NUnit asserts this time, can be used with any test tool,
// but provides less deatailed errors in case of failure (for now)
msg =>
msg.GetDistinguishedField<string>("Number") == "1") &&
msg.GetDistinguishedField<decimal>("TotalPrice") == 14.99m
)
// Ensure the orchestration completes successfully
.ExpectCompleted()
.ExecuteTest();
}

In this example:

The test first sends a PurchaseOrder message which activates the orchestration;

The orchestration, after applying some rules, decides that the price for the order should be
14.99, and submits a modified PurchaseOrder back to its caller (the OrchestrationSimulator in this case)
for review;

The test validates that the price is 14.99 as expected;

The test then submits an Approved PurchaseOrder message, simulating that the client confirmed his order;

The orchestration generates an Invoice for the order and submits it to its caller;

The test validates that the total on the Invoice is also 14.99;

The test validates that the orchestration was completed successfully, without any error or discarded messages.

Expect in any order example

In some cases, the test can't reliably anticipate the order in which messages will be sent
by the orchestration. This frequently happens with orchestrations using parallel actions,
or with tests involving multiple orchestration instances. In that case, Bizmonade allows
definining a series of validation steps that can occur in any order.

Request-Response Client Test

In this example, the orchestration "ReqRespClient" talks to a server.
This server could be an external web service, a SQL stored procedure,
another BizTalk orchestration,... However, the test doesn't need to
know that and can be ran without the server: it is the test itself
that simulates responses from the server. This allows testing
the client in isolation from the server. Similarly, it's possible
to test the server by itself without the client (not shown in this example,
but can be found in Bizmonade samples).

Delivery notification NACK example

In this example, the test verifies that the orchestration behaves correctly
in case of an error (on a port with Delivery Notification enabled).
This makes it possible to test error conditions without "physically" reproducing
the error (for example, by deleting a filesystem path or by putting down a web service).

The test simulates an error by producing a NACK (Negative Acknowledgement),
which the orchestration will see as a DeliveryFailureException.

[Test]
public void WhenNACK_ShouldStillSendSecondOutputNotExpectingNACK_but_AlsoExceptionDetails()
{
OrchestrationSimulator.Test()
.WithOrchestration<SimpleDeliveryNotification__Simulated>()
.When(MessageReceived.FromFile<DeliveryNotificationInput>(
Path.Combine(exampleInstancesPath, "SimpleDeliveryNotificationInput_HelloWorld.xml")
))
.Expect(
ValidationSteps.Build()
.ExpectMessageSent<DeliveryNotificationOutput>(
// expect a first message which is published regardless of ACK or NACK
// (the orchestration will then wait for a NACK for this message)
msg =>
msg.GetDistinguishedField<string>("Greeting.Message") ==
"Output expecting delivery notification"
)
// The NACK method sends a negative acknowledgement to the orchestration.
// This is seen by the orchestration as a DeliveryFailureException. The NACK method
// therefore takes as a parameter an expression to build that exception
// based on the message.
.NACK(
msg =>
new Exception(
"Unable to deliver message with Greeting.Message=" +
msg.GetDistinguishedField<string>("Greeting.Message"))
)
)
)
// [...remaining steps excluded...]
.ExecuteTest();
}

Implementation details

Bizmonade does not depend on BizTalk's orchestration engine. Instead, it provides its own implementation
of the XLang/s language (found in ODX files).
It generates a C# representation from XLang/s code, and the generated C# code uses Bizmonade's
"fake orchestration engine" implementation. This generation is done using an interpreter generated by SableCC.
Another interesting tool that was considered instead of SableCC was Microsoft "Oslo"'s MGramar. However,
development of Bizmonade was already started before the initial CTP release of MGrammar, so it was decided
to keep the working implementation with SableCC for now. Migration to MGrammar could provide some interesting
advantages and may be reconsidered later once, the project is mature enough.

Bizmonade's "fake orchestration engine" executes orchestrations in memory and therefore lacks
some of BizTalk's features for reliability and scalability (for example, it does not persist orchestrations).
This is not a weakness of the tool but rather a desired feature: the tool should allow testing without any
dependency on the infrastructure, and adding any persistence would necessarily add an unwanted dependency.

Note that BizTalk also uses the same strategy of generating C# from ODX files. However, these
are two completely different representations. Having a different representation allows working
at an higher abstraction level (the XLang/s language), without worrying about BizTalk's internal
implementation details (which makes the tool significantly easier to implement).
The disadvantage of this strategy is that BizTalk's behavior needs to be reproduced as closely as
possible in Bizmonade, which is a huge challenge. Therefore, tests developed using Bizmonade
are not (yet) guaranteed to follow the same behavior in the real BizTalk (it only provides a
"good enough" approximation for projects that were used while
developing the tool). Hopefully, as the tool gets used in more real-world projects,
it will be possible to have gathered enough examples to more closely follow BizTalk's behavior.

The tool still depends on some of BizTalk's assemblies. These assemblies are related to message
context properties and pipelines execution (Bizmonade uses pipelines, called from the Winterdom PipelineTesting
library, to promote/demote context properties)

Relationship to other existing BizTalk testing tools

Note that Bizmonade does not replace other BizTalk testing tools such as BizUnit, BizMock, or the
Winterdom PipelineTesting library, but it can be used in addition to them.

BizUnit is closer to integration tests than unit tests. It tests several BizTalk components integrated together, running within the BizTalk runtime:

BizMock shares a similar design goal to Bizmonade: testing in abstraction of the underlying
infrastructure (servers, file system locations,...). However, BizMock achieves that goal
by providing a test BizTalk adapter, which means orchestration must still be deployed
into BizTalk instead of being tested in isolation of the whole BizTalk environment.
Like BizUnit, this tool is probably more appropriate for integration tests than unit tests.

Recommended BizTalk testing/development strategy

Start with unit tests of individual components:

Schemas (including flat file schemas), using a tool such as the Winterdom PipelineTesting library:

Ensure a flat file is interpreted correctly by the flat file disassembler

Apply a map on a XML message and validate its output (either using XPath expressions or by comparing to an expected XML message).
The map should be applied using the same classes as BizTalk, and not directly on the generated XSLT files, in order to ensure the map
will correctly execute in the BizTalk environment.
Bizmonade includes code for this (although it is primarily designed to be used internally for
the "Transform" shape)

There will usually be some duplication between unit tests and integration tests.
However, this does not mean we should forget about unit tests and directly do integration tests.
Since unit tests can be run without deploying to BizTalk, they allow faster feedback after a modification
(we can know much faster if a modification broke something, and avoid deploying to BizTalk until all our unit
tests are successful). Also, when an integration test fails, it can be harder to pinpoint the source of the failure
since multiple components are involved.

Unit tests are also not enough because they isolate the tested component from a major dependency:
the BizTalk server runtime.

Since Bizmonade has its own equivalent of the BizTalk server runtime, it is not guaranteed to reproduce
non-obvious behavior (or bugs/omissions) of that runtime. An ideal Bizmonade would have exactly the same behavior,
but it is not complete enough yet. Therefore, it is possible to have a Bizmonade test that fails for something that
would work once deployed in BizTalk. When this happens, Bizmonade should be modified accordingly
(please report such issues, see Feedback)

How to use Bizmonade

Prerequisites

.Net 3.5

Visual Studio 2008

BizTalk Server 2009 (required for some assemblies and for working in Visual Studio)

NUnit 2.5.2 with nunit.framework.dll installed in the GAC (if using NUnit)

Installing Bizmonade to the GAC and installing the Visual Studio project template

Extract files from BizmonadePreview.zip to a temporary location

Deploy Bizmonade.Build.dll, Bizmonade.Simulator.dll and Winterdom.BizTalk.PipelineTesting.dll to the GAC
(using drag and drop in Windows Explorer to C:\Windows\Assembly, or using gacutil.exe from the command line).
These DLLs can also be installed to another location if you prefer not to use the GAC, but the project template
assumes they are installed to the GAC (so projects generated from the template will have to be modified to reference the right path)

When the project is loaded for the first time on a user's machine,
Visual Studio will show the following security warning:
To have the test project work, select the option "Load project normally".
This is necessary to have Bizmonade generate C# files from ODX files
as part of the build process
(Ideally, a future release should be fixed to avoid the security warning).

The template assumes Bizmonade and NUnit 2.5.2 are deployed into the GAC,
and that BizTalk is in c:\Program Files\Microsoft BizTalk Server 2009.
If not, edit the project generated from the template to reference
these DLLs from their correct location.

On the new project, add references to BizTalk projects containing the
orchestrations to be tested.

Write tests by following the NUnit syntax, and using Bizmonade's
OrchestrationSimulator.

Run the tests with a test runner (such as NUnit GUI or ReSharper's runner)

BUG: it's sometimes necessary to compile the unit tests project twice before it
runs the right version of an orchestration. This will be fixed in a later release.

Adding to another project type (using another unit testing tool)

For now, only a template using NUnit is provided. However, it's also possible to use
Bizmonade with other testing tools, such as MSTest.
To enable Bizmonade in an existing test project, open the project file (.csproj)
in a text editor and add, after the <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets"/>
line:

Project Status

The current release is still considered a "preview" as it still lacks some important features, even though
it is a working implementation that could be experimented with in some real-world projects.
It probably won't allow testing all cases in a real-world project, but trying it in real projects
would allow identifying the limitations that need to be resolved for a more complete version.
Therefore, we would be interested in getting any comments on ways to improve the tool, or on ommited
features/behavior that should be included (see Feedback).

Call/Start Orchestration: Message, Variable and Correlation parameters only for now;

Delivery Notification (ACK/NACK);

Request-Response

Suspend and Resume (note that resume will not work correctly for Suspend shapes inside a Parallel branch, but the orchestration can still be Suspended - support for this special case of Resume will be added later)

Terminate

Features planned for future releases

Call Rules

Transactions and Compensation

XLang/s expression operators: succeeded, exists,...

Property demotion (for now, the code in orchestrations must manually synchronize between promoted properties, distinguished fields and message contents to work properly in Bizmonade tests. This is obviously incorrect and needs to be fixed in a future release).

Execute pipelines from orchestrations

Multipart messages, .Net class messages and XLang messages (only single part XML messages are working for now, others are only partially implemented and not fully tested);

Modify message contents using the xpath() function (it only allows reading from messages for now);

Support all cases of Promoted Properties and Distinguished Fields manipulations.

They do not work at all on multipart messages;

The C# compiler is not always able to infer a property's type from the way it is being used.
Therefore, the XLang/s to C# generation process should cast each property to the right type
(as defined in the message's schemas). For now, these limitations can be worked around by
manually doing the necessary conversion/cast when the compiler is not able to do it
(for example by using System.Convert.ToXXXX or
System.String.Format)

Promote relevant context properties.
Note however that closely following ALL of BizTalk's context properties would be contrary
to the tool's goal of infrastructure-ignorance: for example, it should not promote
any property that exist only on messages coming from a FTP or FILE port.
Therefore, it will only implement a minimal set of context properties:
those that are required for some use case, and that don't depend on a specific protocol.
(However, if really needed, a unit test can still promote protocol-specific properties)

Better error messages when a test fails, to make it as easy as possible to locate
the source of an error.

Features that will not be implemented

Some features would contradict the tool's goal of infrastructure-ignorance,
and will therefore not be implemented in future releases. These still represent
things that are important to test, but an integration testing tool
(which tests the orchestrations integrated in the real BizTalk and with their
real dependencies) would be more appropriate to test them.

Orchestration Persistence

Interaction with any protocol or BizTalk adapter

Promotion of context properties coming from a specific BizTalk adapter.
(Tests can still promote these properties, but they won't be promoted automatically by Bizmonade)

Side by side versionning (test some messages with version 1.0 of an orchestration, and
other messages with version 1.1 of the same orchestration, within the same test)

BAM

Feedback

Since the product is still in development, it would be a good time to get feedback on ways
the tool could be improved. Specifically, we are looking for answers to:

Is there anything important that's missing (and not part of the features planned for a future release)?

Suggestions on ways to improve the way tests are written?

Does the tool have behavior that is different from the real BizTalk's behavior?
(if possible, please provide sample orchestrations and unit tests, it would
greatly help in implementing a fix).

Suggestions on ways to automatically compare Bizmonade and the real BizTalk's behavior?
(Currently, BizTalk's behavior is checked with manual tests, but it would be
interesting to have a way to reuse Bizmonade automated tests in a real BizTalk context
and validate that both contexts return the same results)

For now, feedback can be sent using the comments section of the original
Bizmonade announcement blog post.
A temporary email address has also been setup:
bizmonade at gmail.com.

A more structured discussion board or issue tracking system will be setup later.

This is a "preview" release. It does not support all of BizTalk's features
and may not exactly reflect BizTalk's real behavior in some cases.
Also, all tests written with this release may have to be modified in
future releases, as the testing interface get improved based on user feedback.