The random thoughts of Richard Fennell on technology and software development

We have had an internal timesheeting system written in Access services running without any problems for the past through months. At the end of last week, when people tried to submit their timesheets they started getting a -4002 error saying the macro (that saves the weekly sheet) could not be started.

Checking the server event logs, Sharepoint logs and Access services log tables showed nothing. So as all good IT staff do we tried the traditional IISRESET command (on both our Sharepoint web servers) and it all leapt back into life. The only change on our server in the past week has been been the ASP.NET security fix, and associated reboot, but I cannot see why this should effect Access Services, it looked as if it had basically Access services just failed to restart fully after the server reboot.

I have found a problem with repeated calls to indexed SharePoint Lists with Typemock Isolator 6.0.3. This what I am trying to do…

The Problem

I am using Typemock Isolator to allow me to develop a SharePoint Webpart outside of the SharePoint environment (there is a video about this on the Typemock site). My SharePoint Webpart uses data drawn from a pair of SharePoint lists to draw a map using Google maps API; so in my test harness web site page I have the following code in the constructor that fakes out the two SPLists and populates them with test content.

So basically only the last Isolate.WhenCalled is being used, this is not what I expect from the Typemock documentation. .This states that, worst case, the first Isolate.WhenCalled should be used for the first call and the second for all subsequent calls, and actually the index string should be used to differentiate anyway. This is obviously not working. I actually also tried using null in place of the both the index strings and got the same result.

A Workaround

I have managed to workaround this problem with a refactor of my code. In my web part I used to moved all the SPList logic into a pair of methods

I then used Isolator to intercept the calls to these methods, this can be done by using the Members.CallOriginal flag to wrapper the actual class and intercept the calls to the private methods. Note that I am using different helper methods to create the list of my own data objects as opposed to List<SPListItems>

My workaround, in my opinion, is a weaker test as I am not testing my conversion of SPListItems to my internal data types, but at least it works

I have had to go down this route due to a bug in Typemock Isolator (which has been logged and recreated by Typemock, so I am sure we can expect a fix soon). However it does show how powerful Isolator can be when you have restrictions in the changes you can make to a code base.Wrapping a class with Isolator can upon up a whole range of options.

SPDisposeCheck is a great tool for SharePoint developers to make sure that they are disposing of resources correctly. The problem is it is a bit slow to run, a problem as this will mean developers will tend not to run it as often as they should. A good solution to the problem is to run it as part of the continuous integration process. There is are posts on how to do this via unit tests and as a MSBuild task, but I wanted to use a TFS 2010 style build. Turns out this is reasonably straight forward without the need to write a custom activity.

I created a build template based on the Default one.

After the compile and before the test step I added a InvokeProcess activity

I set the InvokeProcess properties as shown below, the edited settings are

Arguments: String.Format(“””{0}””””, outputDirectory) (remember you need the enclosing “ if your path could have spaces in it)

SPDisposeCheck has a great feature that the number of errors is returned by the command line exe, so the InvokeProcess is able to store this in the SPDisposeOutput variable. We are able to use this fail the build

This is done with a simple if check. If there are any errors found I write a build error message and set the TestStatus to failed. You might choose to set the build status to fail or any other flag you wish. The potential problem with my solution is that the TestStatus value could be reset by the tests that follow in the build process, but for a basic example of using the tool this is fine.

So it is easy to added a command line tool to the build. The key reason it is so easy is that SPDisposeCheck returns a number that we can use to see if the test passed or failed. hence we did not need to parse any text or XML results file. I wish more tools did this.

Gil at Typemock has been posting about some ideas we discussed over breakfast at the Typemock Partner conference a while ago, I have been a bit slow at commenting, so I though I better add to the conversation. Though Typemock is an excellent mocking framework, for me basic mocking is not its biggest win. All the ‘classic auto mocking’ of interfaces to speed TDD style working is great, but I can do that with any of the .NET mocking frameworks. All they do is mean I don’t have to write my own test stubs and mocks, so saving me time, which is good but not the keep win I was looking for.

For me there is another way to save much more time and that is to reduce my ‘build, deploy, use’ cycle. In the land of SharePoint this is a significant time saving, or at least has been for us. It has meant that I can replace the build, create WSP, deploy WSP, let SharePoint/IIS restart and then view a web part, with a build and view in ASP.NET page that uses Typemock to fake out all to SharePoint calls. This is what Gil has termed Isolation Driven Development (IDD) Now isn’t a three letter _DD name going a bit far, I am even not sure there enough in it for me to write a book!

That said this is a solid technique which can be applied to any complex environment where developers or testers need a means to mock out significant, costly, or just slow components to ease there daily work process, often enabling some manual testing process, thus making them more productive. If you read the TPS books it mentions a lot how workers should optimise their work space to reduce wasted time the spend moving between machines or roles, this is just such a move.

When you drop a combo box on a Access 2010 web form and databind to a query or table with a value column and display column (e.g. the query select id, description from table1) you don’t get what you would expect. It shows just the value e.g. a set of integers in the combo.

This is not the case if you are using a standard Access client form. the wizard that gets run sort it all out for you, and if it does not, you just set the ‘Bound Column’ property to sort it out.

On web forms the fix is simple, but not obvious.

Databind the combo as you normal do to the query/table.

Go to the combo’s properties format tab

Set the column count to 2

Set the column widths to 0cm;2cm (basically hide the first column and show the second)

Got time today to start our proper internal TFS 2010 rollout. This involves linking up the new TFS to our SharePoint 2010 farm and merging in the contents from 2008 TFS and 2010 RC servers.

All went fairly well, our SharePoint farm caused a few problems with its rather nice redirection feature. This allows you to gradually move SharePoint content from an older server to a new one. A page is requested from the new server, if it is there it is used, if it is not the content on the old server is used. This caused a couple of problems during the install of TFS that content was not on the new SharePoint server so it redirected to the old one. However, the correct response was that the content was not there and TFS need to install or configure it. Once we realised this was going on there were no major issues with the TFS 2010 installation.

So we had a nice new TFS 2010 install. The next task was to move over content from the 2008 server. Our 2008 server was virtualised so I snaphot’d it, made sure we had a good SQL backup and did an in-place upgrade, and this is where I hit a problem. Our new TFS 2010 install and the 2008 were using the same central SQL server. Our new TFS 2010 install had created a TFS_Configuration DB, I had had the option to put in prefix such as TFS_2010_Configuration but decided I did not need it. This choice came back to bite me. When you do an in-place upgrade of 2008 to 2010 it rolls up all the old TFS DBs into the new structure, creating a new DB, with you guess it, the name TFS_Configuration. If you are doing a new install you can alter this DB name, but this is not the case for a upgrade, you have to use the standard name. So I was a bit stuck. The solution was to:

take my new TFS 2010 server temporarily off line

rename the TFS_Configuration DB

do the in-place upgrade on the new 2008 server to 2010, so it creates as new TFS_Configuration DB

detach the newly upgraded Team Project Collection on the TFS admin console

In my previous post I talked about using Isolator to mock Sharepoint to aid the speed of the development process. I find this a productive way of working, but it does not really help in the realm of automated testing. You need a way to programmatically explore a webpart, preferably outside of SharePoint to check its correctness.

You could use the methods in my previous post and some form of automated web test, but this does mean you need to spin up a web server of some descriptions (IIS, Cassini etc. and deploy to it) An alternative is look at the Typmock addin Ivonna. This a creates a fake web server to load you page and tools to explore it.

I will describe how to use this technique using the same example as my previous post.

Previously I had placed all the code to fake out SharePoint in the Page_Load event of the test harness page. As I am now trying to write an unit/integration test I think it better to move this into the test itself so I would delete code I placed in the Page_Load event other than any property settings on the actual webpart. I would actually refactor the lines creating the fake URL context and fake SPSite into some helper methods and then call them from my new test. I would then load the page in Ivonna and check it’s values.

I have tried to show this below, using a couple of techniques to show how to get to components in the page.

1: [TestMethod, Isolated]

2:publicvoid LoadWebPage_SpSimpleWebPart_3EntriesInList()

3: {

4:// Arrange

5: TestHelpers.CreateFakeSPSite();

6: TestHelpers.CreateFakeURL();

7:

8: TestSession session = new TestSession(); //Start each test with this

Now I have to say I had high hopes for this technique, but it has not been as useful as I would hope. I suspect that this is due to the rapid changing of the UI design of client’s webparts making these tests too brittle. We have found the ‘mark 1 eyeball’ more appropriate in many cases, as is so often true for UI testing.

However, I do see this as being a great option for smoke testing in long running projects with fairly stable designs.

I have found the most productive use of Typemock Isolator with SharePoint is to use it to reduce the time of the F5 cycle (build/deploy/use). If you are using a VPC of some type to do your SharePoint development, as many do, this process can easily take a couple minutes, and these minutes add up.

In my experience webparts usually make fairly simple use of the underlying SharePoint site, by this I mean that they get some data from an SPList(s) or remote data source and render in some way. Or the reverse, they gather data that they drop to an SPLits(s) or remote data source.

So why not remove the requirement for SharePoint during development from the equation? Mock it out with Isolator and an ASP.NET test site.

Consider this scenario…

You have a part that lists people name and email address in a combo box

This data is stored in an SPList

The webpart must also list the URL of the site it is hosted on.

All fairly straight forward, but how to implement the wrapper around it?

The first we create an ASP.NET Web Application and an ASP.NET web page in the same solution as the WebParts class library.

In this web application reference the webpart project

Edit the .ASPX to reference the webpart assembly (line 2) and create an instance on the page (line 11-17). It is best to do this declaratively to avoid any question of the ASP.NET personalisation system.

If you browse to this page you will see a null object exception as it loads the web part but it fails when it calls to SharePoint. This is because we have not mocked that yet. Seeing this error does rely on the fact you have a nice big global Try/Catch in the webparts Render() and CreateChildControls() methods. Note a technique I normally recommend but I think vital for this type of component else errors get swallowed by SharePoint

Add a reference to the SharePoint assemblies and Typemock Isolator (which of course you need a licensed copy of, or the 30 day demo) in the Test web application

In the Page_Load event you now need to add the code to do the faking, I think the comments below explain what is going on. (There is a good argument to refactor much of this into a TestHelper class so it is easy to reuse).

1:using System;

2:using System.Collections.Generic;

3:using System.Linq;

4:using System.Web;

5:using System.Web.UI;

6:using System.Web.UI.WebControls;

7:using TypeMock.ArrangeActAssert;

8:using Microsoft.SharePoint;

9:

10:namespace TestWebSite

11: {

12:publicpartialclass SpSimpleTestWithMockedSP : System.Web.UI.Page

13: {

14:protectedvoid Page_Load(object sender, EventArgs e)

15: {

16:

17:// set the name of the list to read data from

18: wp1.DataList = "ListName";

19:

20:// set the fake return value for the currently running context

21:// we can us null as the current parameter as this is what this web page will return

Once this code is entered you can browse to the page again and you should see a working webpart that thinks it is talking to a real SharePoint site.

So now you have a way to run you SharePoint dependant webpart outside of SharePoint.This means the F5 cycle is reduced to seconds as opposed to minutes, and debugging is loads easier.

What I find this particularly useful for is sorting CSS and JavaScript issues, where is loads of tiny edits to text files.

THIS DOES NOT MEAN YOU DON’T NEED TO TEST IN SHAREPOINT, but it does means you can avoid much of it. The SharePoint phase become far more a test/QA/UAT process as opposed a development one. This model as a developer productivity enabling one.

In my next post I will talk about Mocking Sharepoint for Test with Typemock Isolator

Interesting time at QCon yesterday, shame I was only there one day, I do like the events that are not limited to a single vendor or technology. The multi presenter session I was involved in on Microsoft interoperability seemed to go well, there is talk of repeating at other events or podcasting. It is a nice format if you can get the sub-sessions linking nicely, like themed grok talks.

Due to chatting to people (but that why you go really isn't it?), I only managed to get to one other session, but I was the one I wanted to see, Roy Osherove’s on using CThru to enable testing of monolithic frameworks such as Silverlight. It got a few things clearer in my mind over using CThu, a tool I have tried to use in the past but not had as much success as I hoped. So I think I will have another go at trying to build a SharePoint workflow testing framework, the problem has rested on the back burner too long. I think I just need to persist longer in digging to the eventing model to see why my workflows under test do not start. Roy’s comment that there is no short cut for this type of problem to avoid an archaeological excavation into the framework under test, I think is the key here.