If you like this tool, support it by voting, if you don't like it, make your vote verbose...

Pre-Introduction

The XsdTidy tool has been entirely rebuild from scratch using CodeDom which is much easier to handle than Emit. The new version is called Refly and available at the Refly article.

Introduction

XsdTidy is a refactoring tool to overcome some silly limitations of the exceptional Xsd.exe (see [1]) tool provided with the .NET framework. More specifically, XsdTidy addresses the following problems:

Name normalization: if your XSD schema is using lower case names or more generally non ".NET" normalized names, you will end up with types that will make the FxCop (see [2]) spit out hundreds of infractions.

Fixed Array Sizes: xsd.exe handles multiple elements by creating an array. There is no problem when you are loading the data, but unfortunately this is not convenient if you want to populate a document since arrays do not support Add or Remove. XsdTidy uses ArrayList for more flexibility.

Default Constructor: Xsd.exe does not care about providing a default constructor that initializes the fields with the proper values. This work can become very silly when the object structure is getting big.

XsdTidy achieves refactoring by recreating new classes for each type exported by the Xsd.exe tool using the System.Reflection.Emit namespace. It also takes care of "transferring" the Xml.Serialization attributes to the factored classes. Hence, the factored classes are more .NET-ish and still outputs the same XML. Moreover, there is no dependency between the refactored code and the original code.

As a nice application of the tool, a full .NET wrapper of the DocBook schema (see [3]) is provided with the project. This .NET wrapper lets write or generate DocBook XML easily with the help of Intellisense.

Fixing problems

Name conversion

The .NET standards define specific naming convention for all types of data: arguments should be camel case, function names capitalized, etc... This is really helpful to keep the framework consistent. Tools like FxCop help us stay on the "normalized" side.

This problem is tackled the dumb way: given a dictionary of "common" words, the class NameConformer tries to split a name in separate words, after that it renders it to the needed convention.

There is much room for improvement on the list of words and the algorithm to split the name, any contribution welcome.

FixedArraySize

Arrays are replaced by System.Collection.ArrayList which are much more flexible. Moreover, array fields are created by default using their default constructor. This is to economize you the hassle of creating a collection before using it.

Properties

Fields are hidden in properties, which is more convenient to use. Moreover, collection fields do not have set property according to FxCop rule.

System.Reflection.Emit

The System.Reflection.Emit namespace is truly and amazingly powerful, it enables you to create new types at runtime and execute them or store them to assemblies for further use. Unfortunately, there are not much tutorials and examples on this advanced topic. In this chapter, I will try to explain my limited understanding of this tool.

What is Emit?

The Emit namespace gives you the tools to write IL (Interpreted Language) instructions and compile them to types. Hence, you can basically do anything with Emit. A typical emit code will look like this:

If you are a newcomer, it can look cryptic, but we'll try to explain the above a bit.

Where to start ?

The problem with Emit is that debugging is complicated: if you generate wrong IL code, the framework will not execute it, throwing an error without giving any clue.. Moreover, you usually don't have the time to learn the dozens of the code that are part of the OpCodes class. Therefore, it would be nice to always have some "model" IL and then try to implement it with Emit.

Hopefully, creating this model is easy! It is possible, using decompilers such as Reflector (see [4]), to read the IL code of any .NET assembly. The idea is simple: open a dummy project where you create the model class that needs to be factored, compile and use a decompiler to read the IL of your model and there you go...you have IL code!

Emit, the basics

I will cover some very basic facts about using Emit. As mentioned above, the most efficient way to learn is to work with a dummy project and Reflector on the side. We will see here how to make a basic C# statement in some instance method where value is the first argument and field is a member of the class.

Getting a ILGenerator

Usually, you start by creating an AssemblyBuilder, then a ModuleBuilder, then a TypeBuilder and finally you can add methods to the TypeBuilder using TypeBuilder.DefineMethod which returns a MethodBuilder. This instance is then used to retrieve an ILGenerator object which we use to output IL code:

MethodBuilder mb = ...;
ILGenerator il = mb.GetGenerator();

OpCodes

The OpCodes class contains all the IL operations. It has to be used in conjunction with ILGenerator.Emit as we will see in the following.

Arguments

Each time you call a method (static or non-static), the method arguments are accessible through OpCodes.Ldarg_0, OpCodes_1, ... In an instance method, OpCodes.Ldarg_0 is the "this" address.

Labels

Labels are used to make jumps in the IL code. You need to set up Labels if you want to build instructions such as if...else.... A Label is defined as follows:

Label isTrue = il.DefineLabel();

Once the Label is defined, it can be used in an instruction that makes a jump. When you reach the instruction that the Label should mark, call MarkLabel:

il.MarkLabel(isTrue);

Comparing values to null

Comparing a value to null is done using the OpCodes.Brtrue_S. This instruction makes a jump to a Label if the value is not null.

Creating objects

To create object, you must first retrieve the ContructorInfo of the type, push the constructor arguments on the stack and call the constructor using OpCodes.NewObj. If we use the default constructor of ArgumentNullException, we have:

You can clearly see the "jump across the exception" with the label isTrue.

Assigning fields

The last step is to assign the field with the value (stored in the first argument). To do so, we need to push the "this" address on the stack (OpCodes.Ldarg_0), push the first argument (OpCodes.Ldarg_1) and use OpCodes.Stdfld:

WrappedClassNamespace is the namespace from which the types are extracted

OutputNamespace is the factored namespace

Version is the version number: major.minor.build.revision

NDocBook

DocBook is an XML standard to describe a document. It is a very powerful tool since the same XML source can be rendered in almost all possible output formats: HTML, CHM, PDF, etc... This richness comes to a price: DocBook is complicated for the beginner and it tends to be XML-ish.

This was the starting of the article for me: I needed to generate DocBook XML to automatically generate code in GUnit (see [5]) but I wanted to take advantage of VS intellisense.

The first step was to generate the .NET classes mapping the DocBook schema using the Xsd.exe tool. The generated code had some problems that would make it unusable: non-nullable fields where not initialized automatically and this would lead to a lot of manual work.

Hence, the second was to write XsdTidy and apply it to DocBook. So here's an example of use:

<?xmlversion="1.0"encoding="utf-8"?><bookxmlns:xsd="http://www.w3.org/2001/XMLSchema"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><title>My first book</title><subtitle>A subtitle</subtitle><toc><title>What a Toc!</title></toc><part><title>My first page</title></part></book>

Now, with Intellisense on our side, I am much more comfortable with DocBook...

Conclusion

System.Reflection.Emit is a powerful tool that deservers more attention than it currently has. It can be used to generate optimized parsers (like Regex is doing), runtime typed DataSets, etc...

History

20/2/2004, initial try-out.

References

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

Share

About the Author

Jonathan de Halleux is Civil Engineer in Applied Mathematics. He finished his PhD in 2004 in the rainy country of Belgium. After 2 years in the Common Language Runtime (i.e. .net), he is now working at Microsoft Research on Pex (http://research.microsoft.com/pex).

It has been a couple of years now since this article has been posted. XSD.EXE still has the same drawbacks of using arrays and such. I wasn't successful in using this tool from a cmd line nor could I figure out how to use it in an application correctly. Can someone tell me how I would be able to use XSDTidy (or some other app) to pass in an XSD file and generate a class file that has strongly typed collections (and possibly instantiates any child collections so I can avoid using "New" so often).

I wrote a Visual Studio (2008) macro that takes an XSD and generates a .cs class file using XSD.exe then does a regular expression search & replace to add generic lists for all the fixed arrays created by XSD.exe. I can share if interested.

It's a great tool and I am really impressed. I have a question (might be dumb), I see that all the properties are being generated as "virtual". Is there any specific reason??

Also, It would be nice if the default path to xsd.exe is like this "C:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Bin" instead of "C:\Program Files\Microsoft Visual Studio 2003\SDK\v1.1\Bin"

I had an error in my xsd and xsdtidy launched the xsd window and closed it before I can read what is the error. Is there anyway to pause that window or show the error in a dialog??

These are very minor issues, but thought I should bring them to your notice.

I'm happy to see that you like XsdTidy. I think the most important about it, it is that... you actually contribute to it quite easily. XsdTidy is part of REfly which is hosted at http://mbunit.tigris.org.

If you have requests and so, you can either post them at refly@mbunit.tigris.org or using the issue tracking there.

The concept of extending the output of the XSD tool is a good one. Here is an alternative implementation that generates constructors, properties, strong collections, IClonable implementation, comparers for the properties, ISerializable, and more.

but i can't seem to get the solution to open correctly, no matter what i try, it says the referenced component "NDocBook" could not be found,(at compile time) (and it appears with an exlamation next to its icon under Test)

but if i doubleclick on it to open the object browser, everything works fine...

i realize the ndocbook part is just a test, but i would love to be able to play around with it...

The NDocBook example is interesting, but I don't understand how it helps the author. Writing the document as a series of method calls looks more complex to me. Can you explain how "intellisense" comes into play and how it helps? Is this something that the GUI does for you to generate that code because it knows the schema?

I was more thinking about automatic report generation, not real "literal" context.

Intellisence helps you becauses it proposes the properties of the object which are possible childs/arguments of the corresponding docbook tag. When I write b. , intellisense pops the possible properties so I know what I can add.

This sounds *really wicked* man, I gotta try it. Although I have already coded around XSD.EXE's shortcomings, you have no idea how many times I've typed (MyType[])myArrayList.ToArray(typeof(MyType)). So using this would take some pretty hefty refactoring as I'm using like eight docs genned from XSD.EXE...