Pages

Monday, October 24, 2011

Continuing the URL shortener sample from thesse post: part I and part II, I'll go from (stupidly) storing the shortened URLs in a static variable, as was the case in the code in part I, to storing the shortened URLs in MongoDB. The reasons for choosing MongoDB for persistence in this sample (compared to the "traditional" NHibernate + SQL Server approach) are:

We will only store and retrieve one document at a time. Those are atomic operations in Mongo, so as far as this sample goes Mongo is fully consistent.

Mongo is simple to setup, and very simple to code against. For instance there is no O/R mapping layer at all.

In summary, Mongo is a nice frictionless option for this sample.

Enough talk, show me the code!

Decoupling interface from persistence
The first thing to do is to pull the persistence related code out of the Nancy module. We use the following couple of tests to drive that. They use a Moq mock to specify the interface to the persistence code; that is the UrlStore interface:

Firstly, as stated above they specifify the interface UrlStore. They do so via the mock fakeStorage, and the calls to fakeStorage.Verify(...) which tells us that UrlStore should have two methods, SaveUrl and GetUrlFor.

Secondly we again use the Nancy.Testing.Browser type to call into the ShortUrl Nancy module. Remember that this goes though the whole Nancy pipeline, just like a real request from a browser. So we're genuinely testing the module from the outside. In the first test we do a HTTP POST, and in the second one we do a HTTP GET.

Thirdly in the constructor we're using another feature of Nancy.Testing; the ConfigurableBootstrapper. To understand this bit, a little detour is required. Nancy is very modular, everything in Nancy can be swapped for an alternative implementation via configuration. Furthermore Nancy apps uses IoC/DI from the get go. Out of the box Nancy uses TinyIOC, but that can (of course) be swapped with any other container - and NuGets exists for most (if not all) popular containers. The swapping out of things in Nancy apps is done through code, in a bootstrapper, which is a class in the app that implements INancyBootstrapper, and sets up the whole of Nancy and the app. To make this easy Nancy provides the DefaultNancyBootstrapper which can be used as the base class for app specific bootstrappers. Back to tests above. The ConfigurableBootstrapper, provides a simple way for tests to create specialized configurations through a fluent interface. The contructor in the code above registers the mock UrlStore as a dependincy, which means registering it in the TinyIOC container. In other words the parts of a the app that receives a UrlStore from the container will get the mock one when in the context of these tests.

But the UrlStore interface doesn't exists, so lets write it to make the tests compile:

1namespace ShortUrl

2 {

3publicinterfaceUrlStore

4 {

5void SaveUrl(string url, string shortenedUrl);

6string GetUrlFor(string shortenedUrl);

7 }

8 }

The tests don't run yet though, since our app does not use UrlStore at all. To make it do so, we change the application code to take a UrlStore in the module contructor, and use it for storage (particularly lines 7, 14 and 22):

And that makes the above tests run. But the end-to-end tests shown in part I don't run. The reason is they don't register a UrlStore with the container. They use the DefaultNancyBootstrapper, where no UrlStore is setup. To get past this we need to do a few things: Change the end-to-end tests to use an app specific bootstrapper, create an implementation of UrlStore that uses Mongo, and create an app specific bootstrapper. We'll start by driving out the Mongo implementation of UrlStore.

Shifting gears
To implement UrlStore against Mongo we need to shift TDD gears to using integration tests. The following tests, that drive the implementation of MongoUrlStore assume a Mongo server is running on localhost at port 27010:

These two test cases simply use the UrlStore interface, but also go directly to the Mongo instance to check that things are stored there. This is all quite straight forward, the one thing to notice though is that these test only assume that there is a Mongo server running, not that there is any particular database or collection present from the start. This is one of the ways Mongo is easy to work with, the databases and collections used in the code are just created for us behind the scenes. No friction.
To satisfy these tests we simply implement the UrlStore interface as follows:

1namespace ShortUrl.DataAccess

2 {

3using System.Linq;

4using MongoDB.Bson;

5using MongoDB.Driver;

6using MongoDB.Driver.Builders;

7

8publicclassMongoUrlStore : UrlStore

9 {

10privateMongoDatabase database;

11privateMongoCollection<BsonDocument> urls;

12

13public MongoUrlStore(string connectionString)

14 {

15 database = MongoDatabase.Create(connectionString);

16 urls = database.GetCollection("urls");

17 }

18

19publicvoid SaveUrl(string url, string shortenedUrl)

20 {

21 urls.Insert(new {url, shortenedUrl});

22 }

23

24publicstring GetUrlFor(string shortenedUrl)

25 {

26var urlDocument =

27 urls

28 .Find(Query.EQ("shortenedUrl", shortenedUrl))

29 .FirstOrDefault();

30

31return

32 urlDocument == null ?

33null : urlDocument["url"].AsString;

34 }

35 }

36 }

The thing I really want to point out here is line 21. We just tell Mongo to store an anonymous object and the driver does the rest for us (i.e. serializes it to json). No O/R mapping code!
Now we have an interface for persisting things, our module uses that interface, and we have an implemtation of it against Mongo. The only thing remaining is wiring things up.

Shifting gears back the end-to-end tests
As mentioned earlier the end-to-end tests from part I don't run right now, to make them run we first change their setup to use an app specific bootstrapper:

14public BaseUrlSpec()

15 {

16ShortUrlModule artificiaReference;

17 app = newBrowser(newBootstrapper());

18 }

We've changed line 17 to use Bootstrapper instead of DefaultNancyBootstrapper. The type Bootstrapper does not exists yet, so lets create it, and have it register MongoUrlStore with the container:

We use the DefaultNancyBootstrapper as a starting point, and just use one its hooks to do a little bit of app specific setup, namely registering MongoUrlStore. Using other overrides we could configure anything in Nancy. I really like that a lot.

Sunday, October 16, 2011

In the last post I introduced a litte sample web app coded up with the Nancy web framework. The code presented in that post left a few things open for later enhancement. This post takes a stab one of those things: The part where a string.Format is used to create a bit of html, which is then just returned directly to the client, namely the POST action in this code:

Specifically the last method, 'ShortenedUrlView' is bad code: It's brittle, it doesn't produce valid HTML, and it's open for script injection. Therefore I want to separate that bit of view code from the module code: I want a view engine. Luckily Nancy supports a number of standard view engines(Razor, Spark, DotLiquid, NDjango) each of which are only a NuGet package away, but it also comes with its own appropriately named Super Simple View Engine, SSVE. For this sample SSVE will do, so let's just stick with that.

We add a file called shortened_url.sshtml to the Views folder and put this into it:

Monday, October 10, 2011

This post is about a low ceremony approach to web apps in .NET. It's about just writting the app, without the framework getting in the way. It's about the same sort of feeling that Mogens Heller Grabe talks about with his notion of frictionless persistence with MongoDB. Only here it's about the web part.

I'll take you through a short and simple URL shortener sample app done with the Nancy web framework. A couple of more posts will probably follow on the subjects of view engines, persistence and hosting for this app. Also with an eye on low ceromony and frictionsless development.

What is Nancy?
Nancy is a lightweight open source web framework in .NET. It has a declared goal of being the developers super-duper-happy-path. Nancy is lightweight in the sense that the number of concepts and the amount syntax you have to grok is minimal. That's why I like it, and that's why it provides the frictionless experience I'm after here.

First things first: The first red-green cycle
One of the first very nice things to notice about Nancy is that it supports TDD really well (much more so than ASP.NET MVC), so lets start looking at the sample by looking at the first test:

9publicclassBaseUrlSpec

10 {

11 [Fact]

12publicvoid should_respond_ok()

13 {

14var app = newBrowser(newDefaultNancyBootstrapper());

15var response = app.Get("/", with => with.HttpRequest());

16var statusCode = response.StatusCode;

17Assert.Equal(HttpStatusCode.OK, statusCode);

18 }

19 }

This is just an ice breaker test to get us going: It just tests that our app responds with an http 200 OK status code. There are nevertheless a couple of things to notice:

Nancy.Testing provides us with the Browser class, which gives our tests an easy way to interact with the app as if they were a browser: The browser object allows for doing GET, PUT, POST and DELETE, PATCH and OPTIONS against the app, including setting up the body, the headers and the protocol. In the case of the above we're doing a GET over http to the relative URL "/", no body, no special headers.

Nancy.Testing also provides a BrowserResponse type that allows for testing all sorts of things on the response from the app. The variable 'response' in the above has this type. Here we just check the status code, but in later examples we'll do a lot more.

There, that's the red part.

To make this run we only have to write this little snippet of Nancy based code:

4using Nancy;

5

6publicclassShortUrlModule : NancyModule

7 {

8public ShortUrlModule()

9 {

10 Get["/"] =_ => HttpStatusCode.OK;

11 }

12 }

Firstly this little piece of code shows the central concept of Nancy: It provides a DSL for responding to http requests. It does so by letting you specify what code should run on GET, PUT, POST, DELETE, PATCH and OPTIONS requests for different URIs. In the above we only specify one such action for one URI: GET for '/'. Secondly it shows another defining trait of Nancy: The framework really tries to make things easy for you, in this case illustrated by the fact we just return a 200 OK status code, but Nancy accepts that and turns it into a full response for us.

GETting the form
So returning a 200 OK does not make for much of an app. What we want here is a very simple URL shortener, so lets turn the front page of the app into a form with a field for the URL the user wants shortened and a submit button. Again we start with a test:

Here we see some more of Nancys test emphasis: The BrowseResponse type mentioned above provides a Body property that lets our tests search through the body of the response using CSS selectors and make assertions on the results using a set of fluent 'Should' style extensions. For instance the above asserts that there is exactly one input tag with the id 'url' and that there is one label tag which furthermore contains the text 'Url: '. This, I must sat, is just awesome, and is a very good level to test at: Just below the pixels of the UI, and just above the real http stack of the real web server.

To make this run we need only do slightly more than in the previous cycle: We alter the application code to this:

Now we see Nancy trying to make things easy again: Just indicate in the application code that you want to return a view by saying View["index.html"], and Nancy looks in your views folders, finds the view and off we go.

POSTing the form
Next step is reacting to a POST of the form we just made, by shortening the URL. Let's set up a test for that:

49 [Fact]

50publicvoid should_return_shortened_url_when_posting_url()

51 {

52//when

53var baseUrlPostResponse = app.Post("/",

54 with =>

55 {

56 with.FormValue("url", "http://http://www.longurlplease.com/");

57 with.HttpRequest();

58 });

59

60 baseUrlPostResponse.Body["a#shorturl"]

61 .ShouldExist().And

62 .ShouldContain("http://");

63 }

Here we see how the app object is used for sending a POST to the app, and along with it the appropriate form values; in this case just the url. This starts to show the flexibility and ease of use of the Browser type.

Considering how we've handled the GET request until now, handling the POST is as straightforward:

The return of a string in the last method. That bubles all the way up, and becomes the repsonse back to Nancy. Again Nancy is nice to us, sets the content type to text/html and the status code to 200 OK. This is not the nicest code on my part, so I might just return to refactor it in a future post.

The above of course is not very clever about the shortening, and does not exactly persist the short URLs in a good permanent place. I'll return to the persistence in a furture post.

Redirect
Only one part is missing for a simplistic but functioning URL shortener: The app must redirect the shortened URLs to the original longer one.
Again lets start with the test:

65 [Fact]

66publicvoid should_redirect_to_original_url_when_getting_short_url()

67 {

68//when

69var baseUrlPostResponse = app.Post("/",

70 with =>

71 {

72 with.FormValue("url", "http://www.longurlplease.com/");

73 with.HttpRequest();

74 }).GetBodyAsXml();

75

76var shortUrl = baseUrlPostResponse

77 .Element("a")

78 .Attribute("href").Value

79 .Split('/')

80 .Last();

81

82//then

83 app.Get("/" + shortUrl, with => with.HttpRequest())

84 .ShouldHaveRedirectedTo("http://www.longurlplease.com/");

85 }

We do the POST again, find the short URL with a bit of LINQ-to-xml and then do a GET to that. And then just assert that the redirect happened. Pretty straight forward except maybe for the Linq-to-xml part.

To make the last test run we add this to our application code:

14 Get["/{shorturl}"] = param =>

15 {

16string shortUrl = param.shorturl;

17return Response.AsRedirect(urlMap[shortUrl.ToString()]);

18 };

which is an action for a whole set of URIs, namely the URIs matching the URI template "/{shorturl}". In the action we access the 'shorturl' part of the URI through the param.shorturl; all parts of the templatized URI will be availbe like that through the dynamic param. Nice.