Monday, September 29, 2008

Web applications often have pages that show a table of items. Often these items are filtered by some search criteria. We should be able to show a page of items at a time (usually called paging) and it's nice to be able to sort the items by clicking on the column headers (usually called column ordering). The one thing you really don't want to do when faced with these requirements is get the entire set back from the database and then filter,page and sort them either on the web server or the client.

Today I want to show you a simple framework for doing this with the MVC Framework. The MvcContrib project already has an excellent grid HTML helper written by Jeremy Skinner, so I've merely used the power of LINQ and extended the Grid to provide paging and column ordering.

Here's the page I want to produce. It shows orders from the Northwind database, you can filter by customer and shipper, click on the columns to order by the column's contents and page through the results. They all work together, so the pager remembers the filter and the column order and the column orderer remembers the page and filter:

The filtering (by Shipper), paging and ordering are all executed on the database. Only twenty rows are returned by the query.

The key to making this work is a re-useable definition of the columns we want to display. Here's my one for this page:

publicclass OrderTableInfo : TableInfo<Order>

{

public OrderTableInfo()

{

AddColumn(order => order.OrderID, "Order Id");

AddColumn(order => order.Customer.CompanyName, "Customer");

AddColumn(order => order.Shipper.CompanyName, "Shipper");

AddColumn(order => order.OrderDate, "Order Date");

}

}

I've created a new TableInfo class. Simply inherit this and add the columns you want to be displayed in the constructor. Each column definition is simply a Expression<Func<T, TParam>> that returns the property of the root object that we want to display. The expressions are reused both to grab the value of the property we want to display and as input to the OrderBy clause that does the column ordering.

As you can see we create a new instance of the OrderTableInfo that describes the table we want to create. We then get the orders to display from the database. The SortColumns(orderTableInfo, Request.QueryString) extension method looks in the query string for a key called 'sortby' and matches its value to the column name described in OrderTableInfo. If it finds a matching value it uses the expression from OrderTableInfo in an OrderBy clause. ToPagedList(Request.QueryString.PageNumber(), pageSize) appends Skip() and Take() clauses to do the paging. We then pass the list of orders and the orderTableInfo to the view.

Here's the section of the view that renders the table:

Pager is an extension method on HtmlHelper that uses the IPagedList from MvcContrib to render the pager. The table of orders is rendered by the MvcContrib grid, but instead of specifying our columns here we use the column specification from OrderTableInfo. I've created a new HtmlHelper extension method 'CreateGridColumnBuilder' that creates the Action<IRootGridColumnBuilder<T>> that defines the columns for the Grid.

You'll notice that the 'Order Date' shown in the screen shot above is a short date time (in UK format). So what happens when you want to add formatting to the column? Maybe adding a link column for example. I've provided an overloaded version of TableInfo's AddColumn method that takes a full MvcContrib grid column definition that you can use like this:

publicclass OrderTableInfo : TableInfo<Order>

{

public OrderTableInfo()

{

AddColumn(order => order.OrderID, "Order Id");

AddColumn(order => order.Customer.CompanyName, "Customer");

AddColumn(order => order.Shipper.CompanyName, "Shipper");

AddColumn(order => order.OrderDate,

col => col.For(order => order.OrderDate.Value.ToShortDateString()),

"Order Date");

}

}

Note the Order Date now defines both the property we want to sort by and the full column definition. The problem here is that there's nothing to stop you from providing totally different values for these, so take care :)

Wednesday, September 24, 2008

Yesterday we went live with the first commercial customer for Suteki Shop: Jump the Gun. They’ve been selling Mod clothing from their shop in the North Laines of Brighton for 15 years. Brighton of course is the spiritual home of the Mods so the shop is a local landmark. It’s really exciting to see Suteki Shop taking real orders from the customers of such a cool business.

Of course the main technical interest with Suteki Shop is that it’s built with the MVC Framework. That’s really nice if you’re a developer since it makes building complex websites that much easier. Suteki Shop’s development mostly took place in the Spring of this year. I worked on it part time while I was also working for a contract client, but I guess total development time was in the region of six weeks. That’s pretty good for a site of Suteki Shop’s complexity. More importantly it hasn’t just been hacked, but built with a scalable component oriented architecture. I am very confident that I’ll be able to build features into the software without countering excessive scale pain.

Probably the main thing that Jump the Gun wanted from their eCommerce site was search engine discoverability. The MVC Framework gives you this by default. The site is totally dynamic but each page has it’s own URL. How about a sky blue Harrington Jacket?

It’s essential to provide these kind of URLs if you want Google to index your product catalogue.

The other way to help search engines is to provide clean HTML. The MVC Framework makes that a sinch. If you’re using Firefox (you’re not???) go to View->Page Style->No Style. Now try to use Suteki Shop. It works fine. You can view the product catalogue categories and buy stuff with no problem. Not so easy if you rely on WebForms and third party controls. It also means that Suteki Shop relies entirely on CSS for its styling. Creating a very different looking shop front should be a simple job for a good designer.

The next thing JtG wanted was a simple to use admin system so that they could have full control over defining the product catalogue and order management. Rather than having an entirely different admin site, editing is built straight into the public web site. Logging in as an administrator gives you this view:

An admin user can define categories and products, including uploading product pictures (which are automatically resized). They can see the results straight away. They can also define the list of countries to ship to and postage rates. The order management system is very simple, it’s a simple list of orders generated by a search form. You can mark orders as dispatched or rejected. The JtG guys, who aren’t in the least bit technical, have found it very easy to use.

Another thing I wanted to provide for Suteki Shop was a simple and easy to use content management system. I’ve been heavily influenced by my experience of blogging. I wanted my customers to edit the content as easily as I write this blog. To that end I’ve implemented the MetaWeblogAPI so that you can edit content with tools such as Windows Live Writer. I wrote about this in more detail here. It makes for a very easy to use CMS. Too easy maybe :)

After spending some time looking at ASP.NET Data Services I now think that this might have been a better way of providing this functionality. Since I’m using LINQ-to-SQL as my ORM it would be very simple to layer the Data Services AtomPub API over it and let blogging tools talk directly to that. In fact it would be easy to provide the entire product catalogue as a RESTfull web service this way.

When the JtG guys said they wanted flash and videos on their site I was initially not keen on the idea. My poor little server, how would it cope, streaming videos all day? Amazon S3 turned out to be a great solution for this kind of high bandwidth content. I can even get Amazon to bill JtG directly for the cost of hosting the video content. It wouldn’t be hard to upload all the static content to S3 and leave the server just hosting the asp.net generated dynamic content. But that’s for the future.

There’s still a lot of tweeking and polishing to do, but on the whole I’m very pleased with the way the software has come together. I wouldn’t dream of attempting something like this with WebForms, it’s a technology that just can’t deliver this kind of site. I’m now talking to a number of other potential customers. It will be very interesting to see how well the design copes with the demands of multi tenancy and what kinds of variation is possible without having to fork the code.

I’d really like to hear from anyone who’s thinking of using Suteki Shop as the basis for their eCommerce site. Of course you don’t have to tell me, the licence is totally permissive. You can take the code and build something entirely your own out of it without mentioning that it’s got anything to do with me.

Saturday, September 20, 2008

I was inspired by the Photosynth session at REMIX. It's a fascinating technology, not so much for what it currently delivers, which is very cool, but it's future promise. I had a very interesting discussion with a chap after the session, who's name I didn't get. He was saying that Photosynth is obviously only a stepping stone on the way to 3D model generation from photos. The practical applications for such a technology are far reaching. Just think of the implication for robots that are able to create a hi-fidelity model of their surroundings. Or for game designers who will be able to reproduce real environments several orders of magnitude faster than they can today. My favorite sci-fi scenario is the possibility of combining the 3D models with 3D printers to quickly and easily reproduce any artifact.

In the meantime, back on Earth, I've had a play with Photosynth myself. Here's my first attempt, a 'synth' of my Dining room.

You can see my current stack of computer books, my old Dell (but with spanking new Das Keyboard), some awful paintings I did at school, my old LPs, the piano (a nineteenth century Bluthner) and my Eastman John Pisano guitar.

Scott Guthrie was kind enough to show a screen shot of the first commercial Suteki Shop implementation at REMIX UK last week. For anyone who was at the keynote speech, I was the 'Mike' he mentioned. I was lucky enough to have a couple of conversations with him during the conference. He's a very down to earth and approachable guy.

Jump the Gun are a legendary Modernist fashion shop in the North Lanes of Brighton. Brighton has, of course, been the spiritual home of the mods ever since their heyday in the 60's so I'm really proud that they've chosen Suteki Shop as their e-commerce platform. It's a great example of a long tail business and they hope to become the world's leading on-line retailer of mod clothing.

We're planning to go live this week and I'll be making a big announcement when that happens, so watch this space!

Model binders based on the IModelBinder interface. You can override the default binders by setting properties on the static ModelBinders class.

New UpdateModel methods on Controller that bind a given model to the Request.Form. The model binders are also used to bind request form values to action parameters.

A ModelStateDictionary that maintains the results of a binding operation. This is maintained in the ViewData.

Updated HtmlHelper input control renderers (like TextBox) and a new HtmlHelper ValidationMessage extension method. These work together to show validation messages on the form by reading the ModelStateDictionary that's contained in the ViewData.

On the whole it's a nicely thought out framework. I especially like the ability to slot in your own implementations of IModelBinder. But there are other things I don't like so much, especially the way the UpdateModel methods on the controller work. In fact I think that the base controller class is getting far too god-like, with too many concerns over and above executing action methods. Having all these controller methods makes actions hard to test and the whole framework less 'plugable'.

In his post, Scott shows how to implement a validation rules engine that you can plug in to your domain entities. I really don't like this approach, it relies on the developer always remembering to call a particular validation method after setting properties. That or an over intimate relationship between the entity and the ORM.

I like to put validation in my domain entity's property setters. It just seems the most natural and obvious place for them.I've blogged about this before, but today I want to show how it can work very nicely with most of the preview 5 framework.

Have a look at this property of my domain entity:

publicvirtualstring Code

{

get { return code; }

set

{

code = value.Label("Code").IsRequired().Value;

}

}

IsRequired is an extension method that raises a ValidationException if value is null or empty. The ValidationException it raises has a nice message "You must enter a value for Code". Now if we simply use the UpdateModel method out-of-the-box like this:

[AcceptVerbs("POST")]

public ActionResult Edit(int id, FormCollection form)

{

var centre = centreRepository.GetById(id);

try

{

UpdateModel(centre, new[]{"Code", "NetscapeName"});

centreRepository.SubmitChanges();

return RedirectToAction("Edit", new { id });

}

catch (Exception)

{

return RenderEditView(centre);

}

}

We get this message on the page when we forget to enter a Code:

Not the nice message that I was expecting. The reason for this is that UpdateModel has a try-catch block that catches Exception and inserts its own message: "The value '<input value>' is invalid for property '<property name>'" whatever the exception may have been. That's not really very useful. Here's a section of the code from the MVC Framework source (I think I'm allowed to put this on my blog):

Having a blanket catch like that around the property setter is not a good idea. What if some unexpected exception happened? I wouldn't know about it, I'd just assume that there was some type conversion error. Also there's no way to get your own exception message into the ModelState at this stage. Scott's blog post shows a work-around where you populate the ModelState at a later stage in the controller, but I want my framework to do that for me.

OK, so I don't like the UpdateModel method on the controller, I'd much rather have my validator injected by my IoC container. To this end I've implemented my own ValidatingBinder:

There are a few things to note about this class. The first is that the actual property binding itself is delegated to a list of property binders that implement the IBindProperties interface. This means I can chain as many specialized property binders that I like and have the IoC container inject them into the ValidatingBinder. All the information needed by the binding operation; the target entity, the form values and the ModelStateDictionary are contained in a BindingContext instance that also knows how to get a particular property value from the form values. I've also implemented IModelBinder.GetValue so that it will work with action property binding as well.

The most important thing for this discussion is the try-catch block in the UpdateFrom method. Note that I look for specific exception types. If I can find a ValidationException I pass that to the ModelStateDictionary which means that the message I want will be shown on the form.

OK, so here's my action method, but now it's using the ValidatingBinder:

[AcceptVerbs("POST")]

public ActionResult Edit(int id, FormCollection form)

{

var centre = centreRepository.GetById(id);

try

{

validatingBinder.UpdateFrom(centre, form, ViewData.ModelState);

centreRepository.SubmitChanges();

return RedirectToAction("Edit", new { id });

}

catch (ValidationException)

{

return RenderEditView(centre);

}

}

When we attempt an update without a code we now get this nice validation exception:

I hope this post doesn't come across as too much of a criticism of the validation infrastructure in preview 5. Most of it I like. I have to have a specialized binder in any case for binding NHibernate entities when I'm updating an object graph, so it's very satisfying that the new control renderers can be made to work with it without too much sweat.

Tuesday, September 16, 2008

Here's a class that takes an array of ISubThing as a constructor parameter:

publicclass Thing

{

readonly List<ISubThing> subThings = new List<ISubThing>();

public Thing(params ISubThing[] subThings)

{

this.subThings.AddRange(subThings);

}

public List<ISubThing> SubThings

{

get { return subThings; }

}

}

Now can you guess what happens if we resolve this using MicroKernel? I naively thought that all my registered ISubThing services would get passed to the constructor, but in fact you get a handler exception saying that the container cannot find anything to supply the subThings property:

[Test, ExpectedException(typeof(HandlerException))]

publicvoid DoesNotResolveArraysByDefault()

{

var kernel = new DefaultKernel();

kernel.Register(

Component.For<Thing>(),

Component.For<ISubThing>().ImplementedBy<First>(),

Component.For<ISubThing>().ImplementedBy<Second>(),

Component.For<ISubThing>().ImplementedBy<Third>()

);

var thing = kernel.Resolve<Thing>();

}

Hammett explains the problem here (check the comments) and shows an ArrayResolver that can be added to the kernel to enable the behaviour I was expecting. Now this test works:

[Test]

publicvoid ShouldResolveArrayOfDependencies()

{

var kernel = new DefaultKernel();

kernel.Resolver.AddSubResolver(new ArrayResolver(kernel));

kernel.Register(

Component.For<Thing>(),

Component.For<ISubThing>().ImplementedBy<First>(),

Component.For<ISubThing>().ImplementedBy<Second>(),

Component.For<ISubThing>().ImplementedBy<Third>()

);

var thing = kernel.Resolve<Thing>();

Assert.That(thing.SubThings.Count, Is.EqualTo(3));

Assert.That(thing.SubThings[0], Is.InstanceOfType(typeof(First)));

Assert.That(thing.SubThings[1], Is.InstanceOfType(typeof(Second)));

Assert.That(thing.SubThings[2], Is.InstanceOfType(typeof(Third)));

}

As Hammett explains in his post, this is a dangerous thing to add to your container, because the implementation he gives doesn't check for circular dependencies. Say we have this class which implements ISubThing:

publicclass Circular : ISubThing

{

public Thing Thing { get; privateset; }

public Circular(Thing thing)

{

this.Thing = thing;

}

}

As you can see it takes a Thing in its constructor, so we'd expect a circular reference exception to be raised by the container. But this test shows that doesn't happen:

[Test]

publicvoid DoesNotDiscoverCircularDependencies()

{

var kernel = new DefaultKernel();

kernel.Resolver.AddSubResolver(new ArrayResolver(kernel));

// a circular reference exception should be thrown here

kernel.Register(

Component.For<Thing>(),

Component.For<ISubThing>().ImplementedBy<Circular>()

);

// this crashes the test framework!

// var thing = kernel.Resolve<Thing>();

}

Instead, if you un-comment the last line the test framework crashes.

Andrey Shchekin has a great series of posts comparing different IoC containers:

It's interesting that only StructureMap actually resolves array dependencies out of the box. It's such a nice feature with endless applications that I would have expected it to be more common. Interestingly MEF (which is trying very hard not to be seen as an IoC container) does this as one its core competencies. but then being able to discover a collection of components that supply a particular service is almost an essential requirement of its stated role as a plug-in framework.

Sunday, September 14, 2008

I had a lot of fun yesterday at the ALT.NET UK (un)conference. Ian, Alan and Ben do a great job organizing it and they managed to raise a lot more sponsorship money this time which meant that they could hire a larger venue, Conway Hall.

Friday night was spent suggesting sessions. There was some kind of dance thing going on in the main hall and it was difficult to hear what was going on at times but there was still a huge range of ideas put forward. Later we retired to the pub where I had some great conversations. I especially enjoyed hearing Sebastien Lambla talking about his open rasta RESTfull web development framework. I've heard him presenting about it before, but maybe it needed a couple of pints of Czech larger for it to make sense. I'm awaiting its release with some anticipation now.

Saturday morning kicked off with a 'park bench'. I'm not sure what the subject was since I spent a little too long enjoying the hotel breakfast and complementary FT. BTW good choice of hotel Ben! I think it was something like 'what does ALT.NET mean?' There was a suggestion that we needed a list of principles somewhat like the Agile movement, but I have to agree with Alan here; I think it would be a bad idea. I didn't get to the bench, but to me ALT.NET is really simple, it means building a community around .NET development that is not controlled by Microsoft. You don't need ALT.Java because that market is diverse and competitive enough that no single vendor is perceived as the single source of tools and guidance. But that is exactly the case with .NET development. The majority of .NET shops simply look to Microsoft for both. ALT.NET provides a convenient label for the .NET community to coalesce ideas around. This is good for .NET development in general, but also good for Microsoft itself.

The first session I attended in the morning was covering ORM use. We had a great discussion of the pros and cons of NHibernate vs Linq-to-SQL vs EF. I ranted a good deal about how any ORM in the .NET space had to have a LINQ provider. We also had a good discussion around repository patterns, where LINQ fits into the specification pattern and DTOs. We stayed in the same room to talk about Multi-tenanting. There were three or four people present who were actively involved in developing multi-tenanted applications so this was a very useful discussion. It mostly revolved around database sharding vs multiple-database patterns and security concerns.

During lunch we had a very interesting and wide ranging chat about IoC containers that then became a discussion on the horrors of sharepoint development.

Ian Cooper lead a very good session on Domain Driven Development in the afternoon. I've heard Ian talk about DDD several times now, but I always seem to learn something new. I would love to see a project that he'd worked on.

This brings me to a suggestion for future ALT.NET conferences. There are some excellent conversations but I do think that it would help if more people brought laptops and we had some projectors available where we could demonstrate real code. Talking about code is very difficult without having examples in front of you. I don't mean that we should be giving presentations, but that at least we should all come prepared to show off something of what we're doing. Of course this is problematic with the commercial code that most of us work on, but being able to fire up Visual Studio (or notepad Peter :) and show rather than tell would be a huge advantage. So Ian, Alan and Ben, if you want to know what to do with the sponsorship money next time: projectors!

Thursday, September 11, 2008

"Many thanks for your regular updates I have now spoken to XXX (who was very charming). He has confirmed our suspicion that <WTF system> in its current state is unable to handle student names that are reserved SQL keywords such as Min, Max etc"

Tuesday, September 09, 2008

I really enjoy Stephen Walther's blog. It's a fantastic mine of information on the MVC Framework. Recently he's been talking about Validation: here and here. He describes using validation attributes on his entities and then providing a validation framework that reads the entities and checks form post values against them. OK, that's my one-sentence summary, it's really quite sophisticated and you should read his posts because if you want to take that kind of approach his solution is well worked out.

I don't like it though. Call me old-fashioned, but I really like to have validation expressed in the domain entity itself. It seems a more DDD friendly way of doing things. Property setters, or constructor arguments, should throw exceptions if the values passed to them fail validation rules. Entities should enforce business rules rather than relying on convention to make them work. With Stephen's method it's easy to set an incorrect value on an entity. You have to explicitly invoke his framework in order for the attributes to be evaluated.

Here's an example from Suteki Shop. Please ignore the LINQ-to-SQL nastiness, but here's a partial class for my Category entity. In the OnNameChanging partial method (that's triggered when the Name property is set) I'm using my validation extensions that I described here. If the value is not present (null or empty) a ValidationException is raised. You can't set Name to an empty or null string from anywhere in the application without this being checked.

A common concern with this method is that the validation rules get triggered when an entity is retrieved from the database. Fortunately LINQ-to-SQL sets the backing field rather than the property setter so this doesn't happen. Any decent ORM will allow you to do the same thing.

The next issue is binding. You need a way of gathering any validation exceptions and presenting them all back to the user. I do this with a custom ValidatingBinder. The Category controller Update action looks like this:

The ValidatingBinder itself is mostly taken from the excellent MVCContrib. The main thing it does differently is collecting any ValidationExceptions raised from property setters and bundling them all into an uber ValidationException that is raised back to the controller.

I've recently updated ValidatingBinder to implement the new IModelBinder interface from Preview 5. Very simple to do, and I'll write more about this soon. Here's the ValidatingBinder code:

Code Rant

Notepad, thoughts out loud, learning in public, misunderstandings, mistakes. undiluted opinions. I'm Mike Hadlow, an itinerant developer. I live (and try to work in) Brighton on the south coast of England.

All code is published under an MIT licence. You are free to take it and use it for any purpose without attribution. There is no warranty whatsoever.