Introduction

This article spawned from a post on the Danish site DotNetForum.dk. The poster asked how you would go about sorting a list of, say, person objects, in a dynamic fashion like SQL. At that point, somebody mentioned the IComparer and IComparer<> interfaces and that you probably could make some sort of generic comparer using those interfaces. And a link to an article by Peter Provost named “Generic Property Comparer Class” was also provided. The article explains how to make a generic comparer using IComparer in .NET 1.1. Peter’s solution is okay considering .NET 1.1 but it’s rather bad in a .NET 2.0 context for several reasons:

It only sorts one property at a time.

It only sorts in ascending order.

It isn’t strongly typed.

It is very slow!

Note: Point 3 and 4 are just an artifact of the fact that .NET 1.1 is lacking both generics and dynamic methods.

So this made me decide that I would try to come up with a better solution utilizing the new generics in .NET 2.0 plus some other very cool features of the framework.

A key point to keep in mind when starting a project like this is performance. I mean, nobody wants to wait 5 seconds while a list gets sorted. Another key point in this project was that it should be dynamic and should, if possible, enable the user to enter a string with a SQL “Order By” like syntax, e.g., “FirstName, Age DESC, LastName”.

The Solution

The first solution I jammed out was a comparer which only used generics and was able to sort ascending as well as descending on any number of properties. So this solution met most of the important requirements. However, even though I made this before I saw Peter's 1.1 solution, they were in fact very similar except mine used generics for specifying the type of objects whereas in Peter’s solution, you pass in the item type as a parameter. Unfortunately, this solution also had the performance problem. After a few performance tests, it was clear to me that this solution was not an option.

The problem is that you can’t really fulfill the requirements without dynamically generating some amount of specific code. At least, that was the conclusion I made after a while. Fortunately, .NET 2.0 has a new feature called “dynamic methods”.

A “dynamic method” is a method you create in-memory during runtime. I won’t dive deeper into this subject but I recommend that you check it out. Basically, what I do is I compile a strongly typed version of the “Compare” method in-memory and call it using a delegate. As you will see in the next section, this is a very powerful way of getting a strongly typed method while keeping it completely dynamic and very fast.

Performance Comparison

I’ve done a few performance tests of the most commonly used sorting methods along with Peter Provost’s solution. The results are listed below:

Fig. 1: Sorting times for 10 to 100000 items.

As you see in fig. 1, the dynamic comparer solution is a good way of sorting your custom entity lists. The dynamic comparer is sharing the first place with the hard-coded comparer. The hard-coded comparer seems to be starting out slowly but that is just a measuring error. This clearly shows that from the runtime's perspective, the dynamic method used to sort is pretty much like the hard-coded one except that we call it through a delegate.

On the second place, we have both the weakly and the strongly typed datasets. Both of them are following the same line which is a little steeper than the other methods. That means that they are getting slower at a faster rate compared to the other methods. Some might think that the strongly typed dataset should be sorting faster than the weakly typed dataset but think again… The strongly typed dataset is just an overloaded version of the weakly typed dataset, so in fact, it is using the same sorting methods as the weakly typed version. It is also worth mentioning that populating a dataset with 10000 items or more is much slower than populating the regular lists.

Finally, on the third place, we have the generic class comparer. As you can see, it is much slower than the other methods of sorting. Not because it is badly designed or coded, but because .NET 1.1 supports neither generics nor dynamic methods.

Declare a sort string or a SortProperty array. Note: This could be created and maintained by a custom editgrid.

Declare an instance of the DynamicComparer class using the generic constructor.

Call the sort method on the List<Person>.

That’s it!

So what’s going on behind the scenes? Let’s begin at line 2 of the previous example. At the time of instantiation of the DynamicComparer, the class will first parse the input string into a SortProperty array. These SortProperties will then be validated against the type we specified when instantiating the DynamicComparer, in this case, “Person”. This validation checks if the Person class is public, if it has any public properties named “FirstName” and “LastName”, if these properties are readable, and finally, if these properties are of a type that implements the IComparable interface. If any of these validations fail, the class will throw an exception. If the validation succeeds, the class will then generate a dynamic method using the specified SortPropertyies and instantiate an internal delegate pointing to the method. The “Compare” method on the DynamicComparer will then in turn be able to invoke this delegate when called. This means that the comparer is ready for use.

Note: If you want to change the sorting of an instance of the DynamicComparer, you can call the “Initialize” method on the instance and pass in a new ORDERBY string or a SortProperty array.

At the next step, we call the “Sort” method on the “persons” list passing in a reference to the comparer.Compare method.

Conclusion

The dynamic comparer seems to be a very efficient way of getting a flexible comparer while keeping it fast, and I’ve yet to see a better or even just a different solution that has the same flexibility and performance. Still, a couple of things could be improved in the comparer but mostly in the instantiation process. I leave it to you to come up with new suggestions and ideas to improve the comparer. I'm looking forward for your feedback!

Secondly, I'm not sure why you are experiencing that particular problem, but I do know that my original code had some bugs in it and that the implementation Marc Brooks did is much more robust. So I recommend you try his implementation out and see if that works better (which I'm sure it will). You can grab his implementation at the Dynamic Reflection Library's project site on Codeplex[^]. If you have any comments/bug reports/suggestions to Marc's implementation please post them on the Codeplex site, thanks.

ok. i just noticed that if i have null string intermixed in my List's it errors out. any help would be wonderful. I do love this solution that you've put out here. I hope you'll get a chance to look at this soon.

i tried looking into the "#region Generate IL for dynamic method," but i got lost too quickly to get anywhere with it. (at the bottom i have a separate question on templates)

=========================================
===========Error Message=================
=========================================
Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

is it possible to write the DynamicMethod piece in a similar way as to a generic template? (see below.) I'm assuming you can't, since it looks like it's generating assembly level code. Or is the generic implementation method just slower? I'm just curious, since i know so little about this. I am in no way trying to be critical.

I'm not planning to make any fixes or changes to the current implementation of the DynamicComparer. But don't be sad, go take a look at Marc Brook's implementation of the DynamicComparer at CodePlex[^]. I'm pretty sure his implementation fixes most of the bugs in my initial prototype/example implementation.

As to your second question, I'm not really sure I understand it, but I'll try to answer it anyways. Early on in the project I did experiment with a generic comparer but it turned out to be dreadfully slow. This is mainly because you then have to use reflection to lookup field and property types at runtime as well as casting the values before comparison. So no-go for a generic only implementation.

If you wanted to implement a simpler DynamicComparer it could probably be done in .NET 3.5 using LINQ expression trees. However, they might be a little slower than Marc's implementation since expression trees probably has to be more generalized, and thus has to do more checks at runtime (I haven't tried it though so I might be completely wrong). Also, Marc's implementation is extremely streamlined and is faster than compiled code in some (maybe even most) cases.

/// <summary>/// used to sort generic collections./// will accept n number of columns where/// those columns are in any n number/// of composite classes for each item/// </summary>/// <typeparam name="T"></typeparam>public class EntityComparer<T> : IComparer<T>{ private List<OrderItem> _orderItems = new List<OrderItem>();

Yes, the comparer that I've suggested can order any type of object, please take a look at Marc Brooks implementation for an improved version of the DynamicComparer.

The problem with the EntityComparer you've sent is the it will probably perform very poorly because it's using reflection, type casting, recursion and other nasty stuff in the CheckSort method. Actually I think this will perform worse than the "GenericClassComparer" from the chart in my article.

I didn't get the updated version yet. What I implemented is the verion attached on this article. Can anybody send me a email with the updated version, please? zhucalvin@yahoo.com
I do appreciate it if anybody can help me out.

That's due to the emitted branch instructions being 'short' or relative branches. You can change the emitted opcodes to be regular branches instead of short branches. The other option is to grab my current version from CodePlex Dynamic Reflection Librarywhich avoids this issue.

First of all, good work you did on the DynamicComparer. It's really fast indeed.

I made some changes to it to support sorting on properties of properties.
Let's say I have an Customer class which has a CurrentAddress property of the type Address.
The Address class has properties like City and Street. What I wanted to do, is sorting customers on their city and street. As far as I could see, this functionality is not supported by the comparer.

After doing some research on dynamic methods, I modified the DynamicComparer to meet my needs described above. Now I call the DynamicComparer like this:

foreach (SortProperty property in sortProperties) // For each of the properties we want to check inject the following il.
{
string[] properties = property.Name.Split('.');

ilGen.Emit(OpCodes.Ldloc_0); // Load local variable at position 0.
ilGen.Emit(OpCodes.Brtrue_S, lbl); // Is the local variable in the evaluation stack equal to 0. If not jump to the label we just defined.
ilGen.Emit(OpCodes.Ldarg_0); // Load argument at position 0.

if (childPropertyInfo.PropertyType.IsValueType) // If the type is a valuetype then we need to inject the following IL.
{
if (!localVariables.ContainsKey(childPropertyInfo.PropertyType)) // Do we have a local variable for this type? If not, add one.
localVariables.Add(childPropertyInfo.PropertyType, ilGen.DeclareLocal(childPropertyInfo.PropertyType)); // Adds a local variable of type x.

int localIndex = localVariables[childPropertyInfo.PropertyType].LocalIndex; // This local variable is for handling value types of type x.

ilGen.Emit(OpCodes.Stloc, localIndex); // Store the value in the local var at position x.
ilGen.Emit(OpCodes.Ldloca_S, localIndex); // Load the address of the value into the stack.
}

// Compare the top 2 items in the evaluation stack and push the return value into the eval stack.
ilGen.EmitCall(OpCodes.Callvirt, childPropertyInfo.PropertyType.GetMethod("CompareTo", new Type[] { childPropertyInfo.PropertyType }), null);

if (property.Descending) // If the sort should be descending we need to flip the result of the comparison.
ilGen.Emit(OpCodes.Neg); // Negates the item in the eval stack.

ilGen.Emit(OpCodes.Stloc_0); // Store the result in the local variable.
}

ilGen.MarkLabel(lbl); // This is the spot where the label we created earlier should be added.
ilGen.Emit(OpCodes.Ldloc_0); // Load the local var into the eval stack.
ilGen.Emit(OpCodes.Ret); // Return the value.
#endregion

Good work on the upgrade... As far as I can tell it should work just as well as the ordinary DynamicComparer. Unfortunately I don't have the time to test it any more than this at the moment. However, you should take a look at the implementation Marc did. I'm not sure if it has child property capabilities but his implementation is far more robust than mine is so it would be a better base for your upgrade. Also, If you're planning on switching to .NET 3.5 you should take a good look at expression trees which could probably help making the DynamicComparer much simpler.

I'm not sure if my implementation of the DynamicComparer works with enums but I'm pretty sure that the implementation that Marc Brooks made do sort enums. You can find a link to his implementation in one of the earlier post in this comments section by him. Please try his implementation out or alternatively have a look at Linq and expression trees in .net 3.5 which can provide similar functionality and that will definately work with enums.

As much of the functionality of the DynamicComparer will be overlapped by the functionality of te various Linq implementations in .NET 3.5. Therefore I won't develop/support this project any further as the Linq project will get a go-live license in june. Thanks for all the support and help which made this project a very rich learning experience.

The null check seems to be invalid when working with custom objects, the fix seems to be using OpCodes.Brfalse_S instead of OpCodes.Brtrue_S.

// CreateDynamicCompareMethod(...)
-> ilGen.Emit(OpCodes.Brfalse_S, lbl); // Is the local variable in the evaluation stack equal to 0. If not jump to the label we just defined (fixed by replacing Bftrue_S).

I've got my generic list List where EventDTO has a string member called "Location". The "Location" column in my SQL database is an optional varchar. I populate this list using a SqlDataReader, knowing that if "Location" is null in the DB, the member will also come back null. So, I try to sort on this column and I get a NullReferenceException in the "Compare" method (line 65 in the original DynamicComparer class).

Also, did you ever update this class to handle Nullables? I saw on another thread that you had talked about it, but I couldn't download your updated code.

Thanks for your bug-report... I recommend that you take a look at Marc Brook's more robust implementation of my technique into his very cool utility library called "Utilities" which you can download here[^]. If you still get the error after switching to his implementation please submit an bug-report to Marc.

The update to the library using nullables has turned out to be a dud. Early performance tests showed the code to be awfully slow. So even though the code were MUCH simpler I didn't think it should be released.

Hi - I realise this is a bit out of place - but I can't figure out where to find this info on the net.

I've been playing with ADO.NET - Datasets, etc for a couple of months now - and I can't seem to find one basic part of SQL - the unique keyword. I know how to make a column unique - but not a search query.

e.g. How would i perform the following query?

SELECT UNIQUE col1, col2 from table

where the table might include many columns of which col1, col2 are not primary keys - or are not the only primary keys.....

I'm sure this is really easy - but I've wasted hours without success. Can anyone help me out?