Document and records management (DM/RM) has traditionally followed a strict file plan, with profile forms used to create and maintain records. These are important best practices; and, for regulated industries such as accounting firms, healthcare, IT, food safety etc., they are necessary to provide evidence of document controls.

However, a common complaint about document management is that its file structure is too rigid and its forms too cumbersome to effectively support users. These systems rely on the carefulness and attention of users to retain, dispose and secure records. The combination of inconvenience and user dependency make many traditional DM/RM systems error prone on the best of days.

“It’s too hard to find what I need” is a common user complaint about traditional rigid records management systems. Part of the challenge is that no single file storage structure can account for the myriad organizational structures represented by human brains. From a business perspective, frustrated users are a risk because they are prone to take shortcuts, make errors or simply avoid using critical systems.

Microsoft SharePoint gives businesses the opportunity to manage records and documents with appropriate protocols in place while also delivering a more flexible user experience. They key to unlocking these capabilities is to establish a properly structured, standards-based and controlled vocabulary for meta-data management. This enables document management and records management developers to build virtual entity structures – also called virtual file plans – that provide users with multi-faceted browsing experiences.

In short, SharePoint offers a user-driven records search and browsing experience while maintaining the necessary file structures by making the storage structure invisible to the user.

At Tango Technology Group, we use SharePoint and Silverlight, combined with DM/RM best practices and principles, to deliver records management solutions that meet both business and usability needs.

When designing a branded SharePoint site, the first task for any web designer should be to create a general SharePoint Theme. A SharePoint theme can be designed through the web browser, and then, with a bit of work, can be saved locally or deployed to a SharePoint site as a feature. The advantage to creating this theme before you do any heavy lifting with the master page is that you’ll find that a lot of the work will be done for you right from the start. What a theme will do is set the colour scheme for a lot of the main parts of a SharePoint page. By creating the theme first, you will avoid doing a lot of searching for the proper styles to override within your master page. Unfortunately, once you design your theme through the browser, there is no easy way to get your theme saved locally so that it can be deployed to other sites. However, Microsoft PowerPoint actually supports the same theme file type (thmx) that SharePoint uses, so you can create a local copy of the theme using this program. In this blog post I will explain how to design your SharePoint theme through the browser, and then I will direct you to some resources that explain how you can create a local copy of the theme and in turn deploy this theme to your SharePoint site.

Designing a Theme

The easiest way to design your theme is to create the theme through your SharePoint site. The reason for this is that you can see the effects of your changes almost immediately with the click of a button. This is essential when you’re doing the fine tuning of your theme.

To access the theme editor through the browser simply go to Site Actions –> Site Settings and then under the Look and Feel section click on Site theme. This will bring you to the page shown in the image below:

On this page you’ll find your most useful tools are the “Select a Theme”, “Customize Theme”, and “Preview Theme” sections. In the “Select a Theme” section you can choose from a wide array of built-in themes that come out-of-the-box. If you’re lucky, there may be a theme there that is already close enough to your desired theme, so you can take it as your starting point. In the “Customize Theme” section you can edit the specific colours within the chosen theme. This section will be covered in detail in the next paragraph. Finally, in the “Preview Theme” section you can see your theme applied to your site without actually fully applying it. What the Preview button does is open a popup that shows an exact copy of your site so that you can see how your site would look like if you applied the theme as it is right now. Granted, it only allows you to see the homepage of your site, however this is usually enough to yay or nay your current design.

In the “Customize Theme” section you can pick and choose all of the colours that you wish to have in your theme. After some experimentation, I found that the most prominent colours are contained in the first 5 settings of this section. The biggest problem with using this designer is that the colour categories are completely vague. It is impossible to know what rules apply to what. A good way to get around this problem is to pick 5 completely different colours for the first 5 settings in this section. For example, pick red, blue, green, and yellow for the “Text/Background” sections, and pick pink for the “Accent 1” section. After applying this theme to your site you will get something that looks like this:

Although this is not too easy on the eyes, it gives you a clear division of which settings are applied to which components. Once you have this as your starting point you can simply change the colours one by one, and you can be sure about which components you are customizing with your colour choices.

Once you have defined what your theme colours are you will have a good platform to start your SharePoint design. In the next blog posts I will start to explain how to customize the individual parts of a SharePoint page.

SharePoint gives you a lot of control over what your site looks like. On top of applying a site theme, you can also create your own CSS rules, and customize the master page that will be applied to the entire site. However, if you’re coming from a web designing background where you’re used to having complete control over the source of the web site, there are a few things that need to be considered before diving in to SharePoint design. The most important of which being: with SharePoint, you do not have complete control over the source of the site. Since SharePoint is a dynamic Content Management System, almost all of the components are generated dynamically based on user content. As such, when modifying the master page, you will not always be able to see the actual HTML source of a component… what you’ll see instead is a Content Placeholder or a SharePoint element that defines the high-level properties of the component. When this is the case, there are usually only two things you can do with that component: (1) move it, and (2) style the component using CSS. With that being said, SharePoint sites are still based on .Net, so technically you can rip everything out and create your own master page from the ground up. However, doing this will throw a wrench in to the dynamic nature of SharePoint and stop your users from having a fully-featured SharePoint site. So it is not recommended to do this if you want to keep SharePoint as a CMS solution.

On that note, this blog (and the next few blogs) will be targeted around re-designing SharePoint while maintaining all of the functionality that comes with it. The first step to accomplishing this is to know what you’re dealing with. In the image below I’ve blocked out the various components that are standard for almost every page that you’ll see in SharePoint. Each block represents a component that can be moved around and styled, but cannot be torn to pieces. Below the image I will go in to a little more detail about what can be done to style and customize the component. This should be enough to give you ideas about what is possible with SharePoint design.

1 – Main ContentThis is where all of the content is rendered. This block can contain lists, libraries, calendars, custom web parts, tasks… anything you create. As such, styling this area needs to be considered on a per-content-type basis rather than simply on a master page basis. For example, if you start changing the styles for the root container, you then need to go through and test how different SharePoint content renders within the newly styled container. Different content will behave differently in this container so it is not exactly trivial to style this section. In the blogs to come I will give you some tips and tricks to styling this area, and describe how the standard content (document libraries, lists, web part pages, etc.) will behave in this container.

2 – Quick Links SidebarThis is where all of your Quick Links will appear. SharePoint allows users to completely customize what goes in to this block, so you don’t have much control over the layout of the content from a master page perspective. What you can define are the CSS rules for how you want the headers to look, how you want the items to look, as well as how you want the selected item to look. Again, customizing this component is worthy of its own blog post, so I will certainly cover this in a future blog.

3 – Top Link NavigationThis is the block that contains the global navigation links for your SharePoint site. Similarly to the Quick Links Sidebar, SharePoint allows the user to define what these links are, so again you are limited to CSS rules when customizing this component. The two main things that you have control over is the style of the selected link, and the style of the unselected links.

4 – Page BreadcrumbThis is the block that will contain the title of the current site, as well as a dynamic breadcrumb that provides links that lead you from the current page to the site root. This is actually one component that can be ripped apart in to two parts: The root site title, and the current page title. Both these parts can be moved around and themed separately, so you can make this part look however you want. You can even style the separating arrow in between the different titles.

5 – Page DescriptionThis is the block that contains the description of the current page. The content of this block will always be text, so it is completely customizable with CSS.

6 – Site IconThis is the icon that is used across the site. It is customized by the user through SharePoint, and does not have any size restrictions. However, since the whole point of “branding” a SharePoint site is to make it have the look and feel of your company, this logo will most likely always stay relatively static. So you can keep this in mind when you decide where it should go.

7 – Search AreaThis block contains the search box that allows your users to search all of the content of the site. The search box and button are pretty static (as they are contained in a delegate control), but the container can be styled with CSS. It is also not an inconceivable task to create a custom search box and deploy it as a feature. However, this task will be out of the scope of any future SharePoint branding blog entries.

8 – Social Tags (Delegate Control)In the master page this block is actually just a Delegate Control. As such, it could technically contain anything, so in general your styling of this area is limited to CSS. By default it is populated with a few buttons that allow the user to tag a page, or add notes that everyone can see.

9 – The RibbonThis block contains the heart and soul of SharePoint 2010: The Ribbon. This block is shipped in one single piece that cannot be ripped apart. On top of this, it is by far the most dynamic component of SharePoint (besides maybe the content itself). The content of the ribbon changes depending on what type of component the user is currently focusing on. As such, it is any designer’s nightmare to style it. On top of this, the ribbon is not affected at all by Site Themes… so it is completely up to you to get it to fit with the theme of the site. Styling this part will most definitely be the topic of a future blog post.

The first step to re-designing any site is to understand what you have to work with… and SharePoint is no exception. I hope the above diagram will give you a good high level view of the different parts that go in to a SharePoint page, so that you can make informed design decisions during the initial phases of your re-branding journey.

Many people have difficulty setting up the connection to SharePoint 2010 such that users are not prompted to enter their username and password. SharePoint does support integrated authentication, but there are a few settings that can interfere.

On the SharePoint side:
Try different configuration options in SharePoint. This is done through the SharePoint Central Administration website:

Choose a Web Application and click the Authentication Providers button.

Try switching the IIS Authentication setting from Kerberos to NTLM – both are capable of working, but Kerberos may require additional IIS configuration.

On the client side:

Make SharePoint a Trusted Site

Modify Windows Vista and Windows 7 registry settings

Set Local Security Policy to allow NTLM response

Make SharePoint a Trusted Site

Internet security settings may prevent Windows from passing credentials to the SharePoint site. To properly configure this setting:

Edit: If you haven’t already tried adding the SharePoint site to the Intranet Zone, try that first it is the proper place for a local SharePoint install to appear and already has the correct permissions. If you cannot do so, try the Trusted Sites alternative below.

Go to Internet Options

Go to the Security tab and select the Trusted Sites zone, then click the Sites button

Add the URL for SharePoint to this list and click OK to get back to the Security tab

Click the Custom Level button

Scroll to the bottom of the list of settings and make sure that User Authentication->Logon is set to Automatic Logon with Current Username and Password

Click OK

Modify Windows Vista and Windows 7 registry settings

Windows Vista and Windows 7 have trouble authenticating with WebDAV, which effects SharePoint’s document library explorer view and also effects opening Sharepoint files dirrectly into MS Office applications.
Windows Vista may require a hotfix to be installed as well as following the steps below to edit the registry. See the link at the bottom of the article for the hotfix.

Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters

If there is no parameter called AuthForwardServerList, you will have to create it, otherwise you can just add SharePoint to the list of URLs already there.

On the Edit menu, point to New, and then click Multi-String Value.

Type AuthForwardServerList, and then press ENTER.

On the Edit menu, click Modify.

In the Value data box, type all of the URLS used by the SharePoint server, and then click OK.

Exit Registry Editor.

Set Local Security Policy to allow NTLM response

A default local security policy in Windows 7 prevents LM and NTLM responses. This may cause credentials to fail and retry because Windows is unable to see the response. To check this setting:

A recent project that we have taken on is to completely re-brand our internal SharePoint site. What we wanted to achieve was to have our internal site match some recent branding changes, while still maintaining all of the functionality and flexibility of SharePoint. This is still a work in progress, but I thought I would share what we’ve achieved so far.

BEFORE

AFTER

The biggest changes to look for are the look and feel of the Quick Links, the overall centered content, and the top navigation bar. Getting this overall look and feel took some trial and error, and some hard lessons. As such, over the next while, I will be posting some blogs that are focused around SharePoint branding in general, and some tips and tricks that we found throughout our re-branding journey.

One of the big selling points for SharePoint 2007 is the ability to have MS Word automatically detect the fields that are set up in Sharepoint and make those values available within the document. This feature is called property promotion when SharePoint pulls the values from the Word doc and fills in the list fields with those values. The reverse, when SharePoint pushes the values defined in the list into the word document, is called property demotion. This works quite well, until you have an event handler that automatically sets the value of a field in SharePoint. Some problems you might encounter are…

Property promotion sometimes occurs after you automatically set the field value, overwriting your value with a blank or whatever value was set in the document.

The logic is fairly simple. Just spin up a thread that waits for SharePoint to be finished modifying the file and list item, then sets the auto field value. When this tactic is used, the auto value gets set in SharePoint and in the document for all reasonable test cases. (i.e. when you copy using Word SaveAs to add a new document, when you use the Windows Explorer View, or when you use the standard SharePoint Web UI to Add or Edit the document.)

One thing to keep in mind is that you can’t determine how many times the code will be run. ItemAdded and Item Updating may each be called once, multiple times, or not at all, depending on what method was used to update the document. So when the value of the custom field is calculated, you must use a formula that gives you the same result every time.

SharePoint 2010 has now included an excellent out-of-the-box feature that allows you to automatically assign a unique document ID to all of your documents that are uploaded to your SharePoint site. This is a site-collection scoped feature, so the document IDs are guaranteed to be unique across the site collection that the feature has been activated for. In order to ensure that the document ID is unique across your entire farm, they have included a configuration setting that sets a specific prefix for all of the document IDs assigned in your site collection. By setting a different prefix for each site collection that uses this feature, you can ensure that each document ID is truly unique across your farm.

Activating the Feature

Since the document ID feature is scoped for an entire site collection you need to be the site collection administrator. If you go to any site in the site collection and go to the Site Settings, you should see a Site Collection Administration section. Under this section click on Site Collection Features. In there, you will see the Document ID Service feature.

This is the service that you need to activate to enable this feature. Once this feature is activated you can edit the view of any document library (through the Library Settings) to include the new Document ID column. When you first activate the feature, you will notice that there are no document IDs assigned to the documents that were already in your document libraries. This is because the task of assigning document IDs to existing documents is performed by a timer job on the server. I will explain how this timer job works later in this post. First, I will explain how to configure the document ID prefix settings.

Setting the Document ID Prefix

Once you’ve activated the Document ID Service, you will see a new entry under the Site Collection Administration section called Document ID Settings. This is where you can configure your document ID prefix.

By default, SharePoint will assign a non-sensical hash as the Document ID prefix, so for usability purposes I would highly suggest changing this value before applying document IDs to all of your existing documents. The settings contained in this page are pretty self-explanatory… the “Assign Document IDs” checkbox is used to turn the document ID assignment ON and OFF, the text box just below that is used to set the prefix for all of your document IDs, and the checkbox just below that is used when you wish to propagate the changes you made to the prefix to all existing documents in your site collection.

One important thing to note (as is indicated by the red text) is that any changes made to these settings will not be applied to existing documents until the timer job on the server is run. This brings me to the next step in activating the Document ID feature: Running the timer jobs.

Running the Timer Jobs

Any changes you make to the prefix, in the settings above, will be applied immediately to any new document that is uploaded to your document library. However, changing the document ID of existing documents is done by a timer job on the server. In fact, there are two timer jobs that are set to automatically apply your configuration settings for the Document ID feature: the Document ID assignment job and the Document ID enable/disable job. You can find these timer jobs on the ReviewJob Definitions settings page in Central Administration, under Monitoring –> Timer Jobs. Once here, look for the jobs that are registered to your web application.

The “Document ID assignment job” is the job that takes your configured prefix, and assigns a document ID to all of your existing documents. By default, this job is run daily between 10:00PM and 10:30PM. If you checked the checkbox in the Document ID settings named “Reset all Document IDs in this Site Collection to begin with these characters”, then this job will go to all of your existing documents and change their prefix to the new one that you have set. The “Document ID enable/disable job” is the job that essentially “saves” any newly configured settings for the Document ID feature across all sites. By default, this job is run daily between 9:30PM and 9:45PM. Sometimes, it may be necessary to see your changes immediately. In order to do this, you simply need to click the link in the Job Definitions settings page and then click the Run Now button.

* * * * *Important: If you have made changes to the Document ID prefix, you may need to run the “Document ID enable/disable job” first before you run the “Document ID assignment job” in order to see your changes.
* * * * *

Once you have run these two jobs, you should now see the Document ID column filled out with your newly configured Document ID:

Accessing Documents

Once all of your documents have IDs assigned to them, you can then reap the benefits of the Document ID feature. There are two ways that you can access documents using their Document ID: (1) Through a well-formed URL, or (2) through the “Find by Document ID” web part.

To access Documents through a well-formed URL, you simply need to type a URL in to the address box of your browser that is of the form: http://<Server_Name>/_layouts/DocIdRedir.aspx?ID=<Document_ID> . Or alternatively, you can copy a document’s URL directly from the Document Library by right-clicking on the link under the Document ID column and clicking “Copy Shortcut” (in IE).

In order to use the “Find by Document ID” web part, you simply need to go to any editable page in your site, and add the web part to the page. The web part can be added through the Insert –> Web Part ribbon menu, under the Search category. Once the web part is on a page, you can simply enter a Document ID in to the text box and click on the arrow button. This will find the document you are looking for and automatically download it for you.

By using the Document ID to access your documents, you are ensuring that your users can always access frequently used documents through a single URL, even when a document is moved to another location. This can save a lot of time and questions whenever your document libraries are undergoing some refactoring. If you are using the Foundation edition of SharePoint 2010, unfortunately you do not have access to this feature… however, if you are using the Standard or Enterprise editions of SharePoint 2010, then you can (and should) take full advantage of this new feature.

I was recently tasked to automatically set a document ID value in a SharePoint 2007 document library. (In SharePoint 2010, an excellent document ID feature is included out of the box) I figured it would be easy since SharePoint supports calculated columns. However, I needed to be able to insert the document ID value into a Word document as a field. Sharepoint does not include calculated fields as part of the document metadata so using calculated columns was not a good solution for me. An event handler seemed like the best way to go.

There are a lot of actions a user could take that should trigger my code. See my previous post SharePoint Event Handling – When do events fire? I needed my code to be triggered no matter which action the user took. The best approach I found was to use the ItemUpdating event and then force ItemUpdating to be called by triggering an update in the ItemAdded event.

I found that there were circumstances when the column value would not get set or would be cleared and I would get the default value for the column instead. After hours of debugging and testing, I discovered that it was only clearing my column when ItemUpdating was triggered and the value of my column had already been set properly and hadn’t changed. So I made the obvious change and removed the piece of my code that skips explicitly setting the column value if it has already been set.

This fixed the problem … Now I needed to figure out why. I stepped through the code with the VS debugger and found that the properties.AfterProperties[COLUMN_NAME] had the correct value for the entire method, but some time after the ItemUpdating method finished the value was cleared, unless I had explicitly set the value during that pass. I did some research, but wasn’t able to find a reasonable explanation for this behaviour. I will add to this post once I find one. In the meantime, I hope this quick fix will help some of you avoid the same type of problem.

There are a lot of different events that are exposed for us to hook our code into. Unfortunately, WHEN those events get triggered is poorly documented and inconsistent across the SharePoint framework. I wrote this article to list out the different event hooks available and to document my test results when I built some automation into a document library using event handlers. My goal was to have some code run on every document added to the library to modify the field values. I was surprised by how much work was involved in getting it to behave consistently. Hopefully the material here will save you from some of the suffering I went through.

Event Receiver Classes

There are several Event Receiver classes that you can extend from that each get called at different times:

SPEmailEventReceiver – This class exposes a single event, EmailReceived, that is fired when a list receives a new email item.

SPItemEventReceiver – Events that happen to items in a list or library (i.e. new item created, item updated, attachment added, file moved, etc.) This is the really important one for document library automation, and is the focus of this article.

SPListEventReceiver – Events that happen to a list (i.e. new list created, list deleted, field added to the list, etc.)

SPWebEventReceiver – Events that happen to the website (i.e. new website created, website deleted, website moved, etc.) and the site collection deleted events.

You can also directly add an event handler through the SPEventReceiverDefinition class, but this is usually not necessary for most common tasks.

SPItemEventReceiver Events

These are the events you need to hook into in order to perform any kind of automation related to documents being added to the library. There are about 20 events in the SPItemEventReceiver class that you could hook into, you can see the full list at http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.spitemeventreceiver_members.aspx, but the main ones we are interested in are those that may get called when a document is being added to a document library.

ItemAdding

ItemAdded

ItemUpdating

ItemUpdated

Different methods of adding a document to the library

SharePoint supports several different paths for adding a document to a document library. Each of these paths triggers events in subtly different ways. I found most of the bugs I ended up working on with the system I built were related to events being called more than once or not being called at all depending on the method used to add the document to the library.

The different ways of adding a document to a library are:

Through the Upload/Add New Document feature in the web UI.

Through Office integration by using the SaveAs menu item to change a document’s location in SharePoint.

Using non-Office programs through the SaveAs menu item to change a document’s location in SharePoint.

Through a mapped drive or explorer view, either copy and paste, or directly drag a file into the explorer window.

The Testing Setup

I built a stub event handler class that extended the SPItemEventReceiver and logged a message for each event handler that was called. I wired it to my document library and then tried adding documents to the library in various ways to see what got called.

The Results

Here are the different sets of events that were called for each test.

When using Office Word SaveAs

ItemAdding – ItemAdded
Office applications seem to have their own integration that avoids calling the ItemUpdating/ItemUpdated methods.

When using NotePad SaveAs

ItemAdding – ItemAdded – ItemUpdating – ItemUpdated – ItemDeleting – ItemDeleted – ItemAdding – ItemAdded – ItemUpdating – ItemUpdated
This was one of the more surprising results for me. I have no idea why the system deletes the first item and then re-creates it. My guess is that it has something to do with how Windows saves things to network drives.

When using MSPaint SaveAs

ItemAdding – ItemAdded – ItemUpdating – ItemUpdated – ItemDeleting – ItemDeleted – ItemAdding – ItemAdded – ItemUpdating – ItemUpdated – ItemUpdating – ItemUpdated – ItemUpdating – ItemUpdated
Yes, It actually called ItemUpdating/ItemUpdated three times when saving from MSPaint. I included this to demonstrate the point that the events called are different depending on which program you use to save the file.

When using the WebUI Upload or Add Item buttons

ItemAdding – ItemAdded
After these initial events you get the item properties screen where the user can enter values for the fields in the library. Then the next sete of events fire…
ItemUpdating – ItemUpdated

Windows Explorer Copy-Paste or Drag-N-Drop

Lessons Learned

In my particular case, if I set the field values during ItemAdding/ItemAdded, then the user had the opportunity to override those values when using the WebUI to add a document. However, if I set the values during the ItemUpdating/ItemUpdated events, the values didn’t get set at all when a user saved a file with an Office application. I came up with two solutions to this issue:

Force ItemUpdating to be called.

By adding a properties.ListItem.SystemUpdate(false) to the ItemAdded event, you can force the framework to invoke the ItemUpdating/ItemUpdated methods … even when MS Word is used to save the file. Then you just set the field values in the ItemUpdating method as usual.

Modify the Item Properties Form

You can override the Edit Properties form on document library items so that it doesn’t allow the user to set values for the fields you want. Then the user can’t modify/override the values you set in your code during the ItemAdded event.