Part 3 of this series of articles showed how we can generate .NET classes from the FpML XSD schema using xsd.exe. It showed how we can then use standard .NET serialization syntax to populate the classes from FpML documents, and vice versa.

However, as mentioned in part 3, xsd.exe generates buggy code when used with the FpML XSDs. This article will go through some of the problems with this code and describe how to fix them.

The corrected code is based on FpML 4.2. The main aim of this article is to give you a starting point if you are trying to generate .NET code from other versions of the specification.

Please note also that this article was written using the xsd.exe supplied with Visual Studio 2005 Service Pack 1. Different versions of Visual Studio, or Visual Studio 2005 without the service pack applied, may give different results. See the discussion in the Comments for this article for more details.

Generating the Code

Part 3 of this series of articles described how to use xsd.exe to generate C# code from the FpML schemas. This gave us a file called fpml-main-4-2_xmldsig-core-schema.cs, which contains a Document class which should be the root class for our serialization. Unfortunately if we attempt to create an XmlSerializer object using this code we get exceptions.

Problem 1: Substitution Groups and Extension

The most fundamental problem with our generated code is that xsd.exe has got confused about the substitution groups and associated extension that were discussed in some detail in part 2 of this series of articles.

In particular it has not decorated the Product property of our Trade class correctly to allow it to deal with all the possible products correctly.

In part 2 we saw that the product element in the FpML is replaced with various individual product types using the substitutionGroup syntax. The product element is a sub-element of the trade element in the XML. As a result in our C# code we have a Trade class which contains a data member of type Product and a public property that gets and sets this. Excerpts from our generated code are as below:

The problem with this code is that, as we saw in part 2, we need to be able to put a Swap object (for example) in to our Product data member. We have a Swap class in our code, and this inherits Product, as we’d expect. So we can store a Swap object in our Product data member.

However, .NET doesn’t know how to serialize or deserialize a swap element in the XML into our product property without being told. In particular, if the deserializer finds an element called ‘swap’ where it’s expecting ‘product’ it doesn’t know that it should deserialize into a Swap object. As we saw in part 2, in <a href="https://www.dropbox.com/s/vrm3rebp99vruot/ird_ex01_vanilla_swap.xml?dl=0 there is a ‘swap’ element where you might expect a ‘product’ element.

This is very easily fixed in this case. We simply decorate the Product property with an XmlElementAttribute that tells the serializer what to do:

This does solve the problem for swap. However, there are multiple other product types that need the XmlElementAttribute added to Product to get the serialization to work (there are over twenty in fact). Also it turns out that this problem isn’t limited to Product. There are other elements that use the substitution group and extension property in this way and have the same problem

At first glance it appears there is no easy solution to this: we are going to have to go through all the possible places where there might be errors and correct the code by hand. However, there’s a much easier solution.

Problem 1 Solution

For simple examples of this problem xsd.exe will correctly generate the required XmlElementAttribute. It’s not immediately obvious why it fails to do so with the full FpML schemas.

It turns out that the reason xsd.exe gets this wrong is because the FpML schemas are spread across multiple files. If we create one big xsd file containing all of the FpML schema files and then run xsd.exe on this the problem goes away. I’ll leave you to draw your own conclusions about the quality of the xsd.exe code.

So to fix the problem we can cut and paste all of the FpML xsd files into one file, removing the include statements that become redundant. An example of this is available. We then use xsd.exe as described in part 3 to create our C# classes.

Problem 2: RoutingIds

If we fix problem 1 as above, we still get an error when we try to create our XmlSerializer object. The error message says ‘Connot convert type ‘RoutingId[]’ to ‘RoutingId’’.

This exception arises because xsd.exe has got an XmlArrayItemAttribute wrong in class RoutingIdsAndExplicitDetails. The generated code for the routingIds property in this class is as below:

The XmlArrayItemAttribute says that the property relates to an array of type RoutingId. However, the property (correctly) is of type RoutingId[][] which is an array of arrays of type RoutingId. So the XmlArrayItemAttribute should be changed as below:

The two fixes above appear to make the generated classes work correctly. With these changes we can deserialize our ird_ex01_vanilla_swap.xml FpML document into the classes. We can then serialize it back into XML, and we end up with the same document we started. We saw this in one of our code examples from part 3. It’s not easy to test this code is working in all cases however. One approach is to take all the sample FpML files provided with the FpML download and attempt to deserialize them and reserialize them. The code is working if the final document is the same as the original one.

It also contains a basic class for comparing the original and final FpML documents and outputting any differences. It does this simply by iterating through the lines in the two files and comparing them. This may not be the best way to do this as it is quite difficult with XML.

It is difficult because there can be valid differences between the files that are hard to deal with. For example we have a ‘difference’ where the original line is:

<hourMinuteTime>09:00:00</hourMinuteTime>

After deserialization and reserialization this becomes:

<hourMinuteTime>09:00:00.0000000+00:00</hourMinuteTime>

These are clearly the same thing but our file comparer has to be able to deal with it. It does so in a very basic way by hard-coding such differences to be ignored in method ‘CompareLines’.

Extent of Testing of Generated Code

Because of the difficulties described above the testing program has only been used to test that the generated code works with the interest rate derivatives and credit derivatives sample files.

Usefulness of Generated Classes

In the last two articles we have demonstrated that we can generate C# classes based on the FpML XSD specification, and with a little work can deserialize FpML documents into this object model, manipulate the objects, and serialize back into FpML.

However, my personal opinion is that we need to think carefully as to whether we want to use these classes. FpML has a very hierarchical structure, and as a result in the generated code we have very many classes interacting to represent even a simple trade. Our object model is not very easy to understand or use as a result.

For example, suppose we want to change the notional on the fixed leg of our interest rate swap (ird_ex01_vanilla_swap.xml) once we have it in the object model. Starting with the top-level Document object that we have deserialized, the syntax is as below:

It’s hard to argue that this code is straightforward: for instance we have a number of ‘Item’ properties referenced that can be of various types. We have to know which type we want. In addition we have the issue that we are not entirely sure that the code generated by xsd.exe is free of bugs, even after the work we have done to patch it up.

As a result in a current project I am working on we have decided not to use the classes generated by xsd.exe, but instead to deserialize into flatter structures of our own design.

Conclusion

This article has shown that it is possible to fix the code generated by xsd.exe from the FpML schemas such that we can deserialize/serialize FpML documents into/out of the object model. We have also shown that it is difficult to test that this will work correctly in all cases, and that the resulting object model is not all that easy to use.

Licensing of FpML Specifications

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.

Part two of the series looked in some detail at the structure of an FpML trade XML document, and showed how to investigate its validation against the schemas using Visual Studio.

This article will show how to generate .NET classes in C# from the XML schema documents. These classes can in theory be used to load FpML documents into objects. These objects can then be manipulated by the code and saved out as FpML. However, there are some problems with the Microsoft tools for doing this, as we shall see.

Motivation

FpML is a powerful way of representing financial products in a standard way. It can be used to pass trade data between financial institutions in an XML format that both parties can understand. It can also be used to pass trade data between computer systems within a financial institution. If all systems know about FpML then we have a standard platform-independent human-readable representation of our data that everyone can use.

Clearly there are some disadvantages to using FpML in this way (it’s very verbose, and we will probably need to modify it for our own needs anyway, in which case it stops being standard). However there’s a prevailing view that FpML is a good place to start when defining what trade data and messages we will pass around in a service-oriented architecture in a bank.

However if we want to use trade data passed to us as FpML we need to get it into a format we can program against. Obviously if we are programming in .NET languages we want to have objects, or even DataSets. Furthermore we’d like to be able to modify those objects or DataSets and then turn them back into valid FpML documents.

The XML Schema Definition Tool (xsd.exe)

Microsoft have provided an ‘XML Schema Definition Tool’ called xsd.exe that purports to allow us to do this. It claims to be able to turn XSDs into classes. These classes can then be used to automatically load associated XML instance documents into objects, and back into XML again after manipulation. xsd.exe is also capable of creating DataSets from XSD files.

Obviously loading XML instance documents into objects is XML deserialization, and turning them back into XML is serialization. Once we have the classes we can use .NET’s standard serialization mechanisms for this.

As we shall see, this doesn’t work particularly well with the FpML schemas.

Navigate to a folder where the FpML XSD files are. If you created the console application used in parts 1 and 2 of this series of articles navigate to the project folder (which has the XSDs in it).

Run the command below: xsd /c fpml-main-4-2.xsd xmldsig-core-schema.xsd
This generates a file called fpml-main-4-2_xmldsig-core-schema.cs which contains the classes we need.

Note that the /c parameter asks xsd to generate classes. There is also a /d parameter that asks xsd to generate DataSets. We will discuss this option later.

Note also that we only need reference the root schema file (fpml-main-4-2.xsd) for this to work: the other schema files are referenced from this file (with include statements) and xsd can work this out. However xsd can’t work out what to do with the xmldsig-core-schema.xsd file unless we tell it to process it. This is because only the namespace is referenced in the schema files, not the file itself.

Using the Generated C# Classes

If we look at the fpml-main-4-2_xmldsig-core-schema.cs file we see that we have nearly 37,000 lines of code, including over 650 classes. As you’d expect these classes use .NET’s XML serialization attributes throughout, so we can serialize into and deserialize from XML correctly. The root class is Document.

To use these classes we need to create an XmlSerializer object based on the root Document in code. This is standard .NET serialization code:

XmlSerializerxmlSerializer = newXmlSerializer(typeof(Document));

Then in theory we should be able to deserialize any FpML document into these classes using the XmlSerializer. The syntax we’d use for this is as below:

Once we’ve deserialized into objects based on our classes, we should be able to serialize those back into XML. Clearly the final XML should be the same as the initial XML. The syntax for the serialization is as below:

Unfortunately the classes generated by xsd.exe have a number of problems, and unless we correct these the basic harness will not work. In fact we can’t even create the XmlSerializer object successfully with the generated code.

Part four of this series of articles will examine the various problems with the code that xsd.exe has generated, and will discuss how to correct them.

Corrected Generated Classes and a Working Harness

Corrected generated classes are available. As will be discussed in part four this code has been corrected such that it appears to work in most circumstances. However we cannot be certain that it is free of all bugs.

This article has shown that in theory we can generate classes from the FpML XSDs using xsd.exe. We should then be able to deserialize FpML documents into these classes, manipulate the resulting objects, and then reserialize back into valid FpML documents. However, xsd.exe has some problems that prevent this from working correctly.

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.

This article will focus more closely on the FpML itself, by looking at the structure of an FpML trade XML document in some detail. It will explain how the FpML schemas fit together to validate such a document. In particular it will examine the use of base and extension types, and of substitution groups, in the XSDs.

As mentioned in part 1, FpML is complex and there’s little documentation. This article will attempt to redress this by explaining how it works assuming only that you know the basics of how an XSD works.

These articles assume you understand the basics of XML Schema Definition language. If you don’t there are a number of good tutorials available on the internet. For example there’s quite a good one at http://www.w3.org/TR/xmlschema-0/. These articles will reference the tutorial for the less obvious XSD structures.

The fpml-main Schema

In part one of this series of articles we showed that the file ird_ex01_vanilla_swap.xml is a valid document in accordance with the FpML schemas. ird_ex01_vanilla_swap.xml is an example file provided with the FpML download that represents a simple interest rate swap. However, at first glance it’s not entirely clear how the large number of XSD files that define the FpML schema fit together to validate this XML document.

If we look at file fpml-main-4-2.xsd and collapse it to an appropriate level we see something as below. I’ve moved the schema attributes onto multiple lines for clarity:

As we can see, this XSD defines a number of namespaces, sets our targetNamespace for the schema, and includes a number of other XSDs from the set of FpML XSDs. It defines one complex type, ValuationDocument. Remember that complex types only have to appear in the XML if they are referenced as the type of an element: ValuationDocument does not have to appear in any XML just by virtue of being in this document.

The schema finally defines the one mandatory top-level element in this document, which is an element called ‘FpML’ which is of type ‘Document’. If we expand the FpML element definition we find it just contains an annotation (comment). So in effect all this schema is saying is that our root element for an FpML document must be called ‘FpML’ and it must be a Document.

Schemas Needed to Validate an Interest Rate Swap

It’s also worth noting here that we don’t need all the XSDs that are imported here to validate the interest rate swap successfully. In fact we can comment them all out apart from fpml-ird-4-2.xsd, provided we add an import for fpml-doc-4-2.xsd (which is included in fpml-valuation-4-2.xsd). We need to comment out the ValuationDocument complex type as well if we do this.

Note you can use the usual comment out/uncomment buttons in Visual Studio to comment XML (and hence XSD) code.

So the ird_ex01_vanilla_swap.xml file will validate with fpml-main changed as below. This makes it clear that we are only using a subset of our XSD files to validate an interest swap, which is what you’d expect. Note that fpml-ird-4-2.xsd itself includes a number of other FpML schemas, so we are not JUST validating with the two included below:

Document Type

We can now navigate to the Document complex type, which is in file fpml-doc-4-2.xsd. Remember that to do this in Visual Studio all we need do is right-click “Document” and select “Go to definition” (or hit F12). We see that Document is defined as ‘abstract’. This is equivalent to a class being abstract in C#. It means we can’t use Document directly as the type of an element, but have to derive from it first. Document also contains an attributeGroup that references StandardAttributes.atts. This has one mandatory attribute ‘version’ (which applies to the FpML element), and can have value “4-0”, “4-1”, or “4-2”.

So now we need to know what complex types derive from the Document type. The easiest way to find this out using Visual Studio is to search our project for “Document” (in quotes), since any extension has to use the extension syntax which looks like:

<xsd:extensionbase="Document">

If we do this we find there are two types that extend Document: “DataDocument” and “Message”. Obviously the ‘Message’ type of document is used for messaging. Here we are only looking at trade representations in XML, which use the DataDocument type of Document.

The XML Document

If we now look at our instance XML document we see that, as expected, it starts with an FpML tag, which has a version attribute. It then uses the xsi:type attribute to specify that we are using a DataDocument.

Most FpML trade documents will have the general structure below, as we shall see. That is, the root FpML element will contain a trade element and a number of party elements. The trade element will contain a tradeHeader element, and an element that represents a specific product, in this case swap. For other product types the swap element will be substituted directly with some other product element.

DataDocument

We saw above that our FpML element is of type DataDocument. The DataDocument definition is in fpml-doc-4-2.xsd, and a summarized version is below. If we look at the full version we can see there is a comment that says that the data document is intended to ‘contain sets of data without expressing any processing intention’:

As expected this extends Document. It adds a sequence, starting with a Validation.model group. The element in this group is optional, so it doesn’t don’t appear in our XML at all.

The next item in our schema is a choice between a sequence of trade and portfolio elements, or of an event element. Our XML uses the trade and portfolio choice. Note that both of these are optional (minOccurs = 0), and our XML has no portfolio element. We examine the Trade element in more detail below.

Finally in our sequence we can have multiple (or zero) elements called ‘party’. These are intended to identify any parties to the trade (see the comments in the full version of the XSD). They are of type Party: this has an ‘id’ attribute, a required ‘partyId’ element, and optional partyName and account elements. If you look again at our XML you can see that we have two ‘party’ elements defined, with just the id attribute and mandatory partyId provided.

Trade

So in an FpML document we have a root element called ‘FpML’ that has to be of type Document. Document is abstract, and one extension is DataDocument, which can contain an element called ‘trade’ of type Trade.

The XSD definition of the Trade complex type is below:

The first thing to notice is that everything is optional (minOccurs=0) apart from the tradeHeader element, and the referenced product element. These two elements are the only ones that appear in our sample XML (although indirectly in the case of product, as we shall see below).

TradeHeader

By now you’ve probably got the idea of how this works and how to use Visual Studio to navigate through the schema and find out what needs to be where in the XML. As a result we won’t go into how the tradeHeader element is constructed, which is straightforward. The comment on the TradeHeader complex type says ‘a type defining trade related information which is not product specific’, which is self-explanatory.

It’s interesting to note that the ONLY information relating to a trade that is not ‘product specific’ here is the trade date and at least one trade ID (along with whose ID it is). If you are constructing a base trade class for a hierarchy of trade types this is really the only data you can meaningfully put in it.

Product and Swap

The product element is a little more complex. If we look again at our sample XML we see that we have a ‘swap’ element in the XML where we expect the ‘product’ element to be.

If we go to the definition of the referenced product element we find that it is of complex type Product, is abstract, and is defined in fpml-shared-4-2.xsd.

However, in spite of the product element (and its associated type Product) being abstract the schemas are not using straightforward extension here. Hence there is no ‘xsi:type’ attribute in our XML instance document. Instead we are using substitution groups.

If we locate the definition of ‘swap’ in fpml-ird-4-2.xsd we see that it is an element of type Swap with substitutionGroup of product. This means it can be substituted directly for the product element in our final XML (as can any other element tagged in this way):

The substitution group syntax on its own means that element swap and type Swap need have no relation to type Product. Swap could be a completely new type with no reference to the base Product type. We can substitute whatever we like for the product element.

However, in the FpML schemas complex type Swapalsoextends type Product, which means this element has to take account of the definition of the Product complex type.

So, to summarize, we have a ‘swap’ element in our XML document where there is a ‘product’ element in the XSD. We can put the swap element in there because of the substitution group syntax: we just substitute the swap element for the product element. However, the swap element is also of type Swap which extends type Product, so it must take account of the Product definition as well.

This is certainly difficult to understand.

Product itself is abstract, has an id attribute and contains a named group called ‘Product.model’. However, all of this is optional and does not consequently appear in our sample instance XML.

In my opinion even if we ignore the use of extension the use of substitution groups is confusing for developers. Most XSD constructs map to object oriented concepts. However, because we are using substitution groups here we are effectively saying that absolutely any type can be a member of our parent trade type at this point in the schema. It’s like including a generic object data member in a class that can be used to contain ANY other class. In OO terms we have no idea what interface this object will implement. It’s hard to represent this in UML as well: see http://www.xmlmodeling.com/documentation/specs/substitutionGroup. Finally, as we shall see in part 3, the Microsoft tool for creating classes from schemas (xsd.exe) struggles with the combination of extension and substitution groups used in the FpML schemas.

Swap

The Swap complex type is again fairly straightforward. Everything is optional except for at least one element called swapStream, of type InterestRateStream. Clearly these represent the swap legs for our interest rate swap, and consequently we have two of them in our sample instance XML. As mentioned above you’ve probably got the idea of how this all fits together by now, and how to investigate further, so we’ll leave the schemas at this point.

Conclusion

This article has attempted to show how validation using the FpML schemas works, and to give an idea of the structure of a trade XML document in FpML.

The FpML Specifications of this document are subject to the FpML Public License (the “License”); you may not use the FpML Specifications except in compliance with the License. You may obtain a copy of the License at http://www.FpML.org.
The FpML Specifications distributed under the License are distributed on an “AS IS” basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.
The Licensor of the FpML Specifications is the International Swaps and Derivatives Association, Inc. All Rights Reserved.