This is actually the first of three parts to the course, with the next two parts coming in Q1 2014. Here’s the table of contents for part 1:

JavaScript Fundamentals for SharePoint Developers

SharePoint and JavaScript: Getting Started

Core JavaScript Object Model Programming

Reading/Writing SharePoint Data with the JavaScript Object Model

Programming Security in SharePoint with the JavaScript Object Model

Fiddler and an Introduction to REST

Reading SharePoint Data with REST APIs

Creating, Updating and Deleting SharePoint Data with REST APIs

Deploying JavaScript Libraries

Downloading JavaScript Libraries

Common Third-Party JavaScript Libraries

Final Thoughts for Part 1

Altogether, part 1 here is just over 6 hours long. Parts 2 and 3 add on an additional 9-10 hours for a grand total of over 15 hours of SharePoint JavaScript goodness by the time I’m done. I’ll post the outlines for the next 2 parts soon.

I’m obviously a bit biased, but I think there’s a LOT of good material in here – including some that I don’t think you can find anywhere else, or that other published content is just plain wrong or incomplete. It intentionally starts out very basic but then quickly moves into the deep end of the pool, covering both REST and JSOM. There are also modules on the best/proper ways to work with libraries (both deploying to SharePoint and downloading to the user’s browser) and finishes with a discussion on the REST vs. CSOM debate and which one you should use.

Stay tuned for more highlights on the content. If you don’t have a Pluralsight subscription, you can get one for $29/month, or wait for the companion e-Book (more on that later!)

*: I say “real” because there is a course I recorded for Critical Path Training originally that was converted to a PS course but I don’t really count that one

This is quite obvious when you think about it, but it’s easy to miss nonetheless when you’re searching for something in a hurry and could lead to some problems:

When looking up a method or property for the SharePoint 2013 JavaScript Object Model, make sure you’re looking at the correct version on MSDN!

It’s easy to tell which version you’re in, it’s right at the top:

With the server-side OM, a lot of times it didn’t matter which version you referenced because, unless you were looking for something about a new capability, the server-side OM was pretty well established and the core bits didn’t change very much.

This is not so with the JS OM. It is very much still evolving and even core bits are seeing massive changes. One case in point, the Web class had 14 methods in SharePoint 2010 but that count more than doubles in 2013 to 30 methods.

If you’re looking at the wrong page in MSDN, you’re going to miss a lot of interesting capabilities.

There are two prevailing methods of registering a callback for the executeQueryAsync function:

context.executeQueryAsync(

Function.createDelegate(this, _onSucceed),

Function.createDelegate(this, _onFail)

);

and:

context.executeQueryAsync(_onSucceed, _onFail);

As you can see, the difference is simply the inclusion or exclusion of the Function.createDelegate call. If you’ve always wondered what the difference between the two was, as I have, the answer is pretty simple: the former allows you to override what the value of the this variable will be inside the function.

In the latter case, not using Function.createDelegate, when you are inside either the _onSucceed or _onFail callbacks, this will reference the callback function itself – onSucceed or onFail respectively.

In the former case, using Function.createDelegate, when you are inside either the _onSucceed or _onFail callbacks, this will reference whatever you pass in as the first parameter to the createDelegate function. In this example, we’re passing in the current value of this, so inside the callback, this will be the same as in the function that made the call to createDelegate. It’s important to note that you can pass any object in as that first parameter:

Function.createDelegate(window, _onSucceed)

Function.createDelegate(new myWidget(), _onSucceed)

Function.createDelegate({}, _onSucceed)

Function.createDelegate(someOtherObject, _onSucceed)

will all work fine, making this equal to window, a new instance of myWidget, an empty object, and whatever object is represented by someOtherObject, respectively.

Synopsis and Key Take-Aways

Custom field types in SharePoint provide the ability to customize all aspects of SharePoint form fields– from how they are displayed, to how values are validated to how values are stored within SharePoint and a couple of other things along the way. They provide significant capability for enforcing business rules and providing a much more user friendly experience than what is available via the out-of-the-box field types. Unfortunately, this power has never been available in a cloud-hosting scenario such as Office 365. Things are changing for the better, however, with new capabilities available in SharePoint 2013 Preview. We now have the ability to provide the same functionality to our end users without running custom code on the server. With SharePoint 2013 Preview, we can modify the presentation and validation of a custom field (or technically any field) on any form in SharePoint as well as in Views simply via JavaScript. This is an incredibly powerful capability. It provides a nice, easy, standard (and supported) way of customizing the end user experience to be more efficient and friendly.

This article introduces the scenario, walks through the end user experience using a sample custom field and then dives into a code-level review of how to implement this new functionality. The commented source code listing for the required JavaScript is available for download.

Note that as of the initial publication of this article (October 2012), this is based on the SharePoint 2013 Public Preview. Once I can download RTM I’ll look to revisit this and update the article as necessary.

Introduction

Custom field types in SharePoint provide the ability to customize all aspects of SharePoint form fields– from how they are displayed, to how values are validated to how values are stored within SharePoint and a couple of other things along the way. They provide significant capability for enforcing business rules and providing a much more user friendly experience than what is available via the out-of-the-box field types (single line of text, yes/no, date/time, user, url, etc.)

Unfortunately, this power and flexibility is simply not available in a cloud scenario such as Office 365 because they require that files be deployed to the server file system which you can’t do in a cloud solution. For the 2010 release of SharePoint, there have been various attempts at simulating custom field types in a cloud scenario, but all have suffered from varying degrees of fragility or other problems.

Fast-forward to SharePoint 2013 and Microsoft has made things much easier for developers and much less fragile all around with regard to custom field types. For the first time, it is possible to truly get most of the value of custom field types in a cloud scenario such as Office 365 or an on-premises scenario (without all of the work of doing it the old way). Notice that I said most of the value. There are some things that are still not possible and two major caveats of which you must be aware. First the caveats:

Even though we are building things which look and act like custom field types, we are not technically building custom field types. We are technically just customizing the rendering and validation of the out of the box field types. More on this later; just remember it for now

All of our customizations are happening on the client side via JavaScript. There are implications to this for your data and business rules. What happens if a user has JavaScript turned off and therefore your customizations do not run? What are the implications to the security and validity of your data? We will touch briefly on custom validation later, but just keep it in mind – it’s not guaranteed to run)

Acknowledgements

Before going any further, I need to give credit where credit is due. Andrew Connell and I hacked up the first portion of this together. I took the rough POC AC and I had pulled together that almost did everything we wanted, finished it off and cleaned it up to produce the samples in this article. Keep an eye on AC’s blog as he’ll be posting a follow-up article taking some of this in new directions.

Overview of the Solution

With all of that said, let’s see what it is that we’re going to get at the end of this. Here are some screenshots and descriptions of what we’re going to build out in the rest of this article:

The New Item Form

In this example, we’ve customized the rendering of our custom field:

Site Column 1 is our custom field. Technically, it is just a simple text field. Without our customizations, it would render as a textbox. Our customizations change it’s rendering to a dropdown list.

The choices available in the dropdown (One through Five) are added dynamically. In a real world scenario, these could come from a web service call. To keep things simple and focused here, I’m simply hardcoding an array. You’ll see when we get to the code how simple it would be to make this truly dynamic.

We’ve also added some rudimentary validation. For the sake of this demo, FOUR is not a valid choice:

Notice, too, that our field is marked as required in SharePoint and regular field validation continues to function:

One of the things I’d like to play with in a later version of this is the possibilities for additional validation via CSOM and perhaps even some event receiver work to provide additional server-side validation.

Display in Views

Once we’ve entered a legal value, our item is shown in a SharePoint View. We also have the opportunity to customize the rendering here as well:

For the sake of this demo, I’m not actually showing the field value in the View. Instead, I show a JavaScript link that pops up a simple alert when clicked to show the field value:

The Display Form

Again, for the sake of this demo, I’m customizing the presentation of our field on the Display form as well. Instead of showing the actual value, I show it’s numeric equivalent:

The Edit Form

The Edit form looks largely identical to the New form (internally, it’s actually rendered via the same code). The only difference is that we have to make sure that the current value of the field shows as selected in the dropdown:

Technical Walkthrough

Now that we’ve seen what we’re building, let’s dive into how to make it all work. Once you’ve figured it all out, it’s actually pretty easy. (Figuring it all out, though, that was a royal PITA, especially during a beta cycle).

The heart of everything we’re doing here is a new property on the out-of-the-box SPField class (and all of it’s children) within the SharePoint server object model: JSLink (don’t bother clicking on the link – there is NO documentation worth anything there right now – just a placeholder). Similar to the JSLink property on SPView, SPField.JSLink gives us the opportunity to specify a custom JavaScript file that will be used to render our field. Cool.

From our SharePoint 2013 App (or, really, any SharePoint 2013 solution), we add an attribute to the Elements file that defines our custom Field to specify the JSLink value:

<Elementsxmlns="http://schemas.microsoft.com/sharepoint/">

<Field

ID="{23d7e8ae-29c3-4f05-ac4a-08bbf2bc2b1d}"

Name="SiteColumn1"

DisplayName="Site Column 1"

Type="Text"

Required="TRUE"

JSLink="~site/Scripts/SampleJSField.js"

Group="Custom Site Columns">

</Field>

</Elements>

The SampleJSField.js file will now be used to render our field wherever it is used. The full SampleField.js file is available at the end of this article for your reading pleasure..I’ve added some comments to explain what is going on, but the following sections spell out the highlights

IIFE

Because we are only specifying a JavaScript filename in our JSLink attribute, and not a particular method to be called, we need some way to actually execute our code. We do this via an IIFE (Immediately Invoked Function Expression) declared right at the top of the file:

You could do this different ways, but this is the most concise and easiest. Basically, in one shot, I’m declaring an anonymous JavaScript function and invoking it. No magic inside the IIFE itself, it simply calls a function to set up our custom field objects (initFieldTypes), creates an instance of my custom JSFieldText object (passing in the field name and a couple of function objects – more on these later) and then initializes the field.

I’m not going to go into details on the whole initFieldTypes function here. It’s in the full source listing and it’s a bunch of pretty funky JavaScript at this point. I plan on cleaning this all up but there’s no sense in doing that until I’m working with RTM bits. Eventually, I’d like to centralize all of this in a common JavaScript library that can be used by multiple fields, but that will come later. For now, it was enough just to build out a POC to show that this all works. A good chunk of the code in the “Global Code” section of the full source listing is based on some code from http://www.ruzee.com/blog/2008/12/javascript-inheritance-via-prototypes-and-closures but I’m not sure whether I’ll stick with that. It introduces some complexity and I’m not sure it gets me a lot. In the short term, however, it filled a void and let me focus on more pressing things.

Registering Our Custom Rendering

I will, however, focus on a few key areas within the initFieldTypes function. The first is where we actually override the default rendering and tell SharePoint to use our custom rendering. This is done via the initField function on the JSFieldText object:

if (hasValue(this.get_fieldName())) {

var newCtx = {};

newCtx["Templates"] = {};

newCtx.Templates["Fields"] = {};

newCtx.Templates.Fields[this.get_fieldName()] = {};

//Views are handled slightly differently because of the way they are called by SP

This code sets up an object hierarchy in a manner prescribed by SharePoint 2013 and registers the new functions to render each of our possible views of this field:

Views

Display Form

Edit Form

New Form

The last line of the snippet is what actually registers our custom handlers with SharePoint.

Views are handled slightly differently because of how they are called internally by SharePoint. For Views, we simply pass the callback that was supplied in the constructor of our field. Note that we do check for null and undefined via some helper functions. If we set the View or any of the Forms to a null or undefined value, SharePoint will use it’s default rendering.

The Forms all use a function on our JSFieldText object because we need to do a little processing before invoking the callback.

Once the callbacks are registered, they’ll be called by SharePoint instead of the default rendering mechanisms.

One of the things that you’ll notice about all of these callbacks is that we’re not dealing with HTML objects. In each case, we need to return the actual HTML text to render our desired UI, so we’re doing a lot of string manipulation. That’s just the way SharePoint expects things to be, so we just run with it.

Technical Details: Views

The View callback is a simple method that modifies the value written into the standard SharePoint View as we saw in the screenshots above. Here’s the code for that:

Notice that the name of our function (viewFunc in this case) was the value passed into our JSFieldText constructor earlier. The other callbacks operate the same way. Note also the use of the eval statement in there. I’m not thrilled with that and would like to find a way around it but haven’t come up with anything yet.

Technical Details: Edit Form

The Edit Form callback is used to render the field on an Edit Form. Here’s the code for the sample I built:

/// Function called to render the NewForm & EditForm view of the field

This one is a little more complex than the View, but it is still just building up an HTML string to be returned to SharePoint and ultimately stuffed down the pipe to the client’s browser. There are a couple of helper functions that you can review in the full source code listing. In the listing above, all I’m rendering is the select box to present the choices. The actual field used by SharePoint is handled in the getStandardInputHtml function. Remember, it renders as a hidden field and I use a simple JavaScript function (moveValue, shown below) to copy the selected value from the dropdown list to the hidden field so it is available to SharePoint.

The moveValue function also handles my custom validation:

function moveValue(targetId) {

var mySelect = document.getElementById('myNiftySelect')

var hiddenField = document.getElementById(targetId);

var selectedVal = mySelect.options[mySelect.selectedIndex].text

if (customValidationPasses(selectedVal)) {

hiddenField.value = selectedVal;

}

else {

mySelect.selectedIndex = mySelect.selectedIndex - 1;

selectedVal = mySelect.options[mySelect.selectedIndex].text

hiddenField.value = selectedVal;

}

}

function customValidationPasses(newValue) {

if (newValue === "Four") {

alert("Four is right out! Counteth thou to THREE and then lobbeth thou the holy handgrenade of Antioch at thy foe, who, being naughty in my sight, shall snuff it.");

returnfalse;

}

returntrue;

}

There’s a fair amount going on in the editFunc function, but none of it is very complex. This is another area I’d like to work on tweaking a bit.

As I said before, for the sake of this demo, I have hardcoded in an array to provide the choices for the dropdown:

//keeping it simple for the sake of this demo code - not really calling a service to get values

function getOptionsFromSimulatedWebServiceCall() {

return ["One", "Two", "Three", "Four", "Five"];

}

If you needed to make this dynamic, simply change the code in that method (and probably its name, too) to do whatever you need.

Technical Details: New Form

The New Form rendering is actually handled by the editFunc function. Notice back in the IIFE when we create the JSFieldText object that we pass editFunc in twice:

The first one is actually the New Form callback. If you needed to render your field differently on the New Form, simply pass in a unique callback here.

Technical Details: Display Form

Last but not least is the Display Form. Remember this shows the numeric representation of our field’s value (kind of lame, I know, but I wanted to show something different for each type of rendering and this was the best I could come up with). The displayFunc function handles this for us:

/// Function called to render the DisplayForm view of the field

function displayFunc(renderCtx) {

if (renderCtx != null && renderCtx.CurrentFieldValue != null) {

var fieldVal = renderCtx.CurrentFieldValue.toString();

switch(fieldVal)

{

case"One":

return"1";

break;

case"Two":

return"2";

break;

case"Three":

return"3";

break;

case"Four":

return"4";

break;

default:

return"5";

}

}

return'';

}

The Future’s So Bright…

I’m quite excited about the possibilities being opened up for developers with changes like the JSLink property in SharePoint 2013. While a number of folks are bemoaning the move to the cloud and the changes that brings, I’m quite bullish on the whole thing. Sure, there are some things I don’t like, but all in all, as a developer, I’m happy. At least in the 2013 release, I don’t really lose anything very important for my on-premises deployments and I pick up significant ground in my cloud implementations. A win all around.

Specifically for custom field types, I see this capability as incredibly powerful. A nice, easy, standard (and supported) way of customizing the end user experience to be more efficient and friendly opens up a lot of possibilities. I predict a small segment of the app store will be based around custom fields utilizing this capability.

Conclusion

That’s all there is to this. As I said at the beginning, with it all laid out and explained, there’s not a lot to this and it is pretty easy. I think that bodes well for this type of customization in the future. I plan on spending a lot more time on this once the RTM bits are available. Once I get it all working and cleaned up, I’ll integrate it into my JavaScript series and probably into a SharePoint 2013 version of the CKS:ScriptLibrary project as well.

Comments and ideas for additional functionality are welcome. I do realize that a lot of the code in this needs to be cleaned up and like I said, I’ll get to that with the RTM bits. If you have specific comments about something that doesn’t work, or could be refactored to make this better, I’d specifically like to hear those.

Synopsis and Key Take-Aways

This article introduces the JavaScript client-side object model (CSOM) added to SharePoint 2010, covering a discussion of when and why you would use the JavaScript CSOM and some important aspects that impact the performance and scalability of your application. It finishes with a handful of examples of the CSOM in action performing typical (and perhaps not so typical) actions as well as links to other examples and reference material.

Introduction

If you’re new to the JavaScript CSOM, the best place to start learning about it is a decent overview. You need to get an

Why Use the JavaScript CSOM?

There are a few key pieces of information about the JavaScript CSOM that will help to answer this question:

The code can be combined with other JavaScript libraries – most importantly JQuery

With that initial understanding, you can start to see benefits offered by the JavaScript CSOM: we can provide the rich, engaging user experience that our user’s want while still interacting with SharePoint in a way that maintains the integrity of our SharePoint server environment.

To give you an idea of what is possible before going into detail, here are a few things that can be done from the CSOM:

Why Would You NOT Use the JavaScript CSOM?

Not everything is available in the CSOM – some areas of SharePoint simply cannot be accessed by code utilizing the CSOM. More on this in a moment. In addition, some requirements are best met with server side code – see below for details.

What Can We Access?

In general, CSOM code can interact with SharePoint at the site collection level and below. This means that the following are available:

SPSite (although we can’t reach outside of our current site collection)

SPWeb

SPList

SPDocumentLibrary

SPListItem

SPFile

SPField

SPUser

web parts

security

etc.

The The Official JavaScript Object Model Reference lists out all of the available objects. It is important to note that not all aspects of some of these objects are available – some properties and methods available in the server side object model are blocked. For example, you can access the Site object to get to a site collection, but not the SPSite constructor, which would allow you to connect to a different site collection; it is blocked. One final note, the SPSecurity object is not available as well. This means that you cannot do any type of elevation of privilege from the JavaScript CSOM. All code runs as the current user.

When analyzing requirements to determine whether CSOM is an option, consider the level of access required. In general, if it is within a single collection, you can probably do it with CSOM.

What Scenarios Don’t Work?

Even if you are only using allowed objects in your JavaScript code, there are some scenarios that just don’t lend themselves to client-side processing. For example:

Processing large amounts of data – pulling all of that data across the wire down to the client may not be a good way to process it

Security concerns – pulling data across the wire to process it without some type of encryption (e.g. SSL) may present security problems

Code that needs to run whether users are connected to and browsing your site or not, for example workflows and timer jobs.

Code that must respond to server-side events, such as after an item has been added to the database

Code that must run, and must run exactly as written. This is a hard one. User’s can turn off JavaScript. Malicious users could also modify the cached version of the JavaScript file on their machine and run their modified code instead of the code you intended for them to run. What happens to your data integrity and environment if your JavaScript code doesn’t run, or doesn’t run the way you expect it to run? Understand that large portions of SharePoint will stop functioning or at least have a severely reduced level of functionality, but it is not impossible to use SharePoint with JavaScript turned off. What happens to your environment then? It seems to me that this is a largely ignored area, but that doesn’t mean it isn’t important. Honestly, I don’t have a really good answer for this yet. It’s just an area you need to think about – what are the implications?

Requirements

To make use of the JavaScript CSOM in your SharePoint site, you need to make sure that the CSOM library is loaded by your page. This will be done for you from the Master Page if you use one of the out-of-the-box Master Pages. If you use a custom Master Page, you must make sure that the sp.js file is loaded. Part 2: Deploy & Download covers some aspects of this. Additional material will be covered in later modules.

Advanced Topics

With a basic understanding of the JavaScript CSOM and what is and is not possible, we can move on to some more advanced topics.

Batch processing

All commands in the JavaScript CSOM are not executed immediately when the JavaScript engine in your browser evaluates the line of code. Instead, the command is added to a batch and executed along with the other commands in the batch at a point in the future. This is one of the fundamental paradigms of JavaScript CSOM programming that must be understood. The hardest part of this whole operation is understanding when you must commit a batch and how to do so.

At the most basic level, you must commit a batch of commands prior to interacting with the properties of any objects referenced in the commands. An example will help to clarify this:

1:function runCode() {

2:var ctx = new SP.ClientContext();

3://Store the Announcements list in a variable:

4:var list = ctx.get_web().get_lists().getByTitle('Announcements');

5://Store a list item in a variable

6: listItem = list.getItemById(4);

7://Load the list item so we have access to its

8://properties

9: ctx.load(listItem);

10://To this point, we have not left the client and

11://none of our code has actually "done" anything.

12://All of the commands have simply been added to a

13://batch. The next line sends the batch to the

14://server so all of our commands will execute.

15://After the following line has returned from its

16://call to the server, the list and list item we

17://"retrieved" are available for us to access and

18://work with its properties

19: ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

20: }

21:

22:function onQuerySucceeded() {

23: alert('Item Title is ' + listItem.get_item('Title'));

24: }

25:

26:function onQueryFailed(sender, args) {

27: alert('Request failed.');

28: }

Don’t worry about the details of what’s going on in that code sample for now. We’ll look into those details later. Read the comments to get an understanding of the code, but here are the things you need to know to understand the batch process:

All of the code from line 1 to line 18 runs on the client and has no interaction with the server. For example, on line 4, the call to getItemById does not actually connect to the server and grab a reference to the SPListItem. It simply adds the command(s) to the list of commands queued up to be committed (this is the “batch” referred to earlier)

Line 19 commits the batch to the server. When this line executes, the batch is packaged up and sent across the wire to the server. The server executes the code we’ve told it to run and returns a result to our page.

Line 19 also registers 2 callback methods – one for if the batch runs successfully (on lines 22-24), one if the batch fails (on lines 26-28). After the batch has returned form the server, the appropriate callback method is run and the items collected in lines 1-18 (list & listItem) are available for use.

Line 23 references one of the properties from the listitem (specifically the Title property) that is now available to us because we have committed the batch to the server and gotten a populated ListItem back.

Why Batches?

The JavaScript CSOM code runs on the client machine, inside their browser. As we saw above, it commits batches of commands to the server to be executed. It then receives data from the server. By operating in batches like this, the number of round trips across the wire is minimized. Sending data to the server and getting a response is the slowest part of the CSOM; traversing the network is inherently slow. The batch operation keeps our CSOM applications from being excessively chatty which would slow things down considerably.

Things to Know About Batching and Objects

The batch operations have a couple of quirks that are important to know.

Scalar and Collection Properties

Besides the fact that CSOM operates in batches like this, the only really important thing to know about the batched nature of CSOM operations is how we go about accessing objects and their properties in a CSOM application.

For example, consider the following code:

var ctx = new SP.ClientContext();

var web = ctx.get_web();

ctx.load(web);

This code is responsible for filling up the (in this case) web object so that we can access its properties. It is important to note, however, that not all properties of the object are filled up (populated) by this command. It is only the scalar (i.e. non-collection) properties that will have data after a load command and its corresponding executeQueryAsyc command. In other words, simple properties – ones with data types like string, int, bool, etc. are populated. Properties that contain collections are not populated; they must be explicitly filled up. In the case of the Web object, the following are examples of properties that would be populated after a call to load like we have in the code above:

id

rootFolder

serverRelativeUrl

title

etc.

These are all simple scalar properties. (Note that this is not an exhaustive list of the scalar properties, and each object has its own collection of scalar properties.)

On the other hand, collection properties such as the following are not populated by the call to load shown above:

lists

folders

features

fields

availableContentTypes

etc.

These properties all represent a collection of items. In order to populate these properties, we need to explicitly load the collection property:

var ctx = new SP.ClientContext();

var web = ctx.get_web();

var lists = web.get_lists();

ctx.load(lists);

ctx.executeQueryAsync(onQuerySucceeded, onQueryFailed);

We can now work with the collection of lists from the Web. Notice that we don’t need to load up the web object because we don’t access any of its properties, except the lists collection property, which we are explicitly loading.

If you don’t fill up a collection property and commit the batch before trying to access the collection property, you will get an error message that says The collection has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

Performance: Property/Column Values and RowLimit

A key tenet of querying SharePoint data server-side is to only pull from the database the minimum you need to accomplish your requirement. In other words, don’t pull back all of the columns on a list item if you’re only going to work with, for example, the Title and Body column values. A similar approach is important for retrieving content from SharePoint with the JavaScript CSOM, with one additional reason – as stated previously, sending data across the wire is expensive in terms of performance. The more data that has to traverse the wire, the greater the performance impact. Therefore, it is important to only extract and pull down the values we need to work with. In the JavaScript CSOM, this is done in one of two ways:

Listing the property names we need in the load statement

Trimming our queries to retrieve only the columns we need

The latter is the same concept as what we’ve done using SPQuery for years, although the syntax is slightly different, as you’ll see in a moment. The former is new to CSOM. Unlike the load statements shown above which do not specify the properties we need, and therefore pull down all properties, a more efficient load statement would look like this:

ctx.load(lists, 'Include(Title,DataSource)');

This would limit the properties that are populated on our lists object to just the Title of each list and the DataSource. If you do limit the properties that are populated in this way, you need to make sure that you specify every property that you need. Similar to the error above if a collection property is not populated, if you attempt to access a scalar property that has not been populated, you will get an error The property or field has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

The syntax for trimming the columns sent down to the client is similar to what we use on the server, but just slightly different syntactically:

You can see the way the ViewFields elements are set into the query, which is stored in the viewXml property of the CamlQuery object. This example will only retrieve the Title column value from the database and send it down the wire.

This example also shows how to limit the number of records that are sent down to the client. This is another important aspect of maintaining performance. In this example, the result set is limited to the first two records by use of the RowLimit element..

Examples of Real World Tasks

There are only a certain number of core operations that are typically going to be done via the JavaScript CSOM. There are dozens or hundreds of other operations to perform in CSOM, but the majority of the time, your operations will involve the following in some form and then add the other operations on top of these.

NOTE: I wrote some of these samples for MSDN back during the beta of 2010 and some of them are available in the SDK, although I tweaked some of them a little and tested them all for this blog post.

Get ListItem By ID

If you know the ID of a ListItem, this sample demonstrates how to retrieve it. It also demonstrates how to retrieve a List by name.

Synopsis and Key Take-Aways

This article covers important considerations for the mechanics of using JavaScript in your custom solutions and concludes with the recommendations for using a SharePoint Document Library as the repository for your script files, and the Custom Action approach to getting the files downloaded to the user’s browser. Other approaches are reviewed briefly and discarded

Deploy and Download

In the first part of this series, we did a very basic overview of JavaScript, not really touching on SharePoint at all. In this posting, we’re going start touching upon some SharePoint-specific aspects of working with JavaScript. So far, all we’ve done is type some JavaScript directly in to the Console of our browser, we haven’t needed a server. Obviously, that is not realistic for a production scenario.

In a real-world situation, we will have one or more JavaScript files, called libraries, which contain our code and need to be deployed to the server and then downloaded to the user’s browser when we need to access the functionality provided by the library. Typically these libraries will have a .js extension and will be included along with other files in our Visual Studio project. That project will generate a WSP (solution) file which will be deployed to our server environment at either the sandbox or the farm level. When the WSP is deployed, it will make our JavaScript file available for download. We then need to make sure that the file is downloaded.

Getting Started

For the sake of this post, I’ll assume that you have already created a VS project using one of the SharePoint project templates. It doesn’t matter whether you create a farm solution or a sandbox solution. I’ve used the Empty SharePoint Project template. Create a new JavaScript library by right-clicking on your project node in Solution Explorer and select Add | New Item on the context menu. In the dialog box that pops up, click Web in the template category listing on the left and then select JScript File:

This produces a blank JavaScript file in the root of your project, which is fine for now; we’ll move it in a little bit. The name of my file is the default JScript1.js and I’ll refer to this name throughout this post. If you use a different filename, you’ll need to adjust references later in the post as appropriate. Add the following very basic JavaScript code to the new file and save it:

function sayHello(){

alert("Hello World");

}

This is about the simplest JavaScript function you could imagine, but it is sufficient for our needs – we’re focusing on getting our JavaScript deployed and downloaded in this post, not on actually writing JavaScript.

Now that we have a file with some JavaScript in it, we can move ahead with deployment and then getting it downloaded to the user’s browser. First we need to explore some options for deployment to understand the best option for the majority of the scenarios we will encounter.

Deployment Options

In a production scenario, we need to place the file which contains our JavaScript functions on the server and have it downloaded to the client to be executed. The first thing this means is that we need to identify a location on the server to which our JavaScript file can be deployed. The only real requirement for this location is that it be URL-addressable so that the user’s browser can request it and download it. In SharePoint, there are several locations that could fit this bill:

The _layouts virtual directory

Ghosted site pages

A Document Library

We’ll touch upon each of those in the following sections, and rule out all but the last. A few notes about the criteria used to rate each option:

Performance is a consideration, though not necessarily a huge one thanks to browser caching

I’d prefer to keep my approach to JavaScript downloading as simple as possible so I’m looking for the broadest applicability of each approach. Approaches that work across a wide range of scenarios will rank higher.

Ease of maintenance is important

Supporting multiple versions of a library is important

Supporting metadata about the libraries is important

Flexibility is important – I don’t want my choice for deployment to dictate anything about the approach to usage of the libraries if at all possible

_Layouts

Deploying our .js files to the _layouts virtual directory is the most like a “regular” ASP.Net site. When the user’s browser requests the file, it is delivered to them straight from the file system of the server. In SharePoint, deploying to the _layouts virtual directory is easy – especially if we are using the Visual Studio Tools for SharePoint. While it is arguably the easiest location to which we can deploy our .js files, as well as the location which delivers the best download performance, it is unfortunately not the best location for our script files. For one thing, it is not sandbox compatible. It is not possible to deploy files to the _layouts folder in a sandbox solution. Another problem is that it offers the least capability from a metadata and management point of view. For these reasons, it is not a recommended location to which to deploy JavaScript files.

Site Pages

The next option is to deploy our script files as ghosted site pages using a SharePoint Feature. This overcomes some of the limitations of _layouts pages but not all of them. Unlike _layouts, site pages are fully sandbox compatible. That’s a good thing. From a performance point of view, in non-sandbox solutions (i.e. farm solutions), performance will be comparable to _layouts pages as the site pages will be delivered from the file system, even though we deploy them as site pages. In sandbox solutions, performance will not be quite as good as the content will be served from the content database instead of the file system. That’s not necessarily a bad thing, but it is worth mentioning. The one thing that site pages does not address is the manageability of the script libraries. Managing our files requires SharePoint Designer and there is still no effective means for managing metadata about our script files when deployed as site pages, so we will not be exploring this option any further.

Document Library **Recommended**

The final choice for deploying script files is to push them to a document library via a Feature. This is the recommended location for our script files. Primarily this is because it offers us the most capabilities for managing those script libraries, as we’ll see in just a moment. It is important to note, however, that deploying to a document library will have a slight performance impact when our files are requested and downloaded to the user’s browser. Normally a known performance impact is something we would try to avoid. However, in this case it is OK for one main reason – JavaScript files are cached on the user’s machine and so they do not get downloaded with every page. This means that we typically only incur that slight performance hit once per user. The rest of the time, when the browser needs our JavaScript file, it pulls it from the local cache instead of re-requesting it from the server. Thus the performance impact is avoided on all subsequent page loads. If the user clears their browser cache, the file will be automatically redownloaded.

This works so long as you have repeat visitors (so the JavaScript files will be cached on the client), such as in an intranet or perhaps an extranet. In Internet-facing sites, the performance implications may be more severe and the benefits of the document library somewhat lessened. In that case, the _layouts folder may be a better option.

Deploying to a Document Library

Deploying JavaScript files to a document library is trivially easy, as you’re about to see. We’ll begin from our very basic SharePoint project discussed above, containing just the single JavaScript library. In order to deploy this file to a document library, we first need to make sure that the document library exists in our site.

Note: If you know that this document library already exists in your site, you can skip this step.

Creating the Scripts library

In SharePoint 2010, we can very easily add an instance of the Scripts document library to our project:

Right click on your project name in Solution Explorer

Select Add | New Item

Select the SharePoint | 2010 category from the list of installed template categories

Select the List Instance template

Change its name to Scripts

Click Add

In the next dialog box, configure the options as shown and click Finish:

Add the JavaScript file to the Scripts document library

Whether we just created the document library or it existed already, we add our JavaScript file to it in the same manner:

Right click on your project name in Solution Explorer

Select Add | New Item

Select the SharePoint | 2010 category from the list of installed template categories

Select the Module template

Change its name to ScriptFiles

Click Add

In Solution Explorer, delete the Sample.txt file included in the ScriptFiles module

In Solution Explorer, click on the JavaScript file we created earlier, drag it and drop it on the ScriptFiles node.

If the Elements.xml file for the ScriptFiles module is not open, double click it in Solution Explorer to open it

Notice that the Elements.xml file has been updated to include a reference to the file we just added to the module

That’s all you need to do to make sure that your JavaScript file is properly deployed to your SharePoint environment. If you’d like to test it out, hit F5 and when the browser comes up, navigate to <site URL>/Scripts. You will see your JavaScript file listed:

Getting the file deployed is only half the battle. Now we need to get it downloaded to the users browser so it can be used.

Downloading Options

JavaScript libraries are useless if they never leave the server; they must be downloaded to the user’s browser in order for the code they contain to be executed. Similar to getting our library deployed, there are several options for getting it downloaded. We’ll explore each option, discussing its pros and cons before declaring a winner.

A couple notes about the criteria I’m using to evaluate these options:

Performance is a consideration, though not necessarily a huge one thanks to browser caching

I’d prefer to keep my approach to JavaScript downloading as simple as possible so I’m looking for the broadest applicability of each approach. Approaches that work across a wide range of scenarios will rank higher.

I’m looking for the broadest reach when deploying. I want one place to manage all of my JavaScript downloads. I am not interested in deploying specific libraries only to specific pages in my application or conditionally downloading libraries based upon some criteria. The vast majority of your JavaScript is likely to be used in multiple places so combining it together and downloading it in a small number of requests and taking advantage of browser caching makes sense. While it is true that some JavaScript will only be needed on certain pages, it is far better to use a delayed loading or script-on-demand approach (discussed in part 11 in this series) to control those downloads.

Compatibility with a delayed-loading scheme such as SharePoint’s Script On Demand is important

ScriptLink Control

The ScriptLink control was introduced in SharePoint 2007 and is still available in 2010. Its main responsibility is to cause a JavaScript file to be downloaded to the user’s browser. It does this by ensuring that the proper <script> tag is written into the HTML stream. While the ScriptLink control is often recommended, I am going to shoot it down for two reasons:

it is not sandbox compatible. Say what you will about the sandbox, and even toss in the fact that it is not a recommended approach for SharePoint 2013, it is an option in 2010, is the only option for server-side code in Office 365 and therefore important, at least for now. As SharePoint and SharePoint Online evolve, this will need to be revisited.

it is hard to maintain as needs change. In this way, it is similar to adding the <script> tags directly to the master page – changing it requires editing the master page in SharePoint Designer, which is something that should be avoided.

It is unfortunate that the ScriptLink suffers from these drawbacks, because it does provide value that we’re going to be hard-pressed to replicate.

Master Page Script Tags

Adding <script> tags to your master page is just not a good idea. They’re hard to maintain, hard to work with initially, don’t travel well if you switch master pages, require SharePoint Designer to edit and various other problems. While they may seem initially like a good idea, they’re just not in the long run. Don’t do it.

Web Part

This option entails using the dreaded Content Editor Web Part to add a <script> tag directly to a page. This is troublesome for a few reasons:

It is page-specific. It only impacts the page to which you add it. While this may seem like a good thing, it is not. There are many pages in SharePoint that you can not add a web part to – how will you get your JavaScript onto those pages? It also means that you need to add this web part on to every page upon which you want the JavaScript available. If your needs change, there are a lot of places that you will now need to change.

It does not version well, cannot be included in source control and makes debugging more difficult. These all fall under an umbrella of professional development disciplines and the CEWP fails miserably here.

Custom Action **Recommended**

Adding the HTML to our page via the SharePoint Custom Action element offers many advantages and no disadvantages. It is easily the best option:

It is fully sandbox compatible and fully farm compatible. This means that I can use the same approach no matter how my solution is being deployed.

It is global – all pages in the scope at which we activate its Feature (Site or Site Collection) will have the appropriate HTML added to it

It is flexible and easy to maintain. We take advantage of the SharePoint Feature framework to be able to turn the functionality on and off at whatever scope we need it

There are no performance penalties to using the Custom Action

It is fully compatible with SharePoint’s Script on Demand functionality and various other delayed loading schemes we’ll talk about in part 11 of this series.

Therefore it is easy to declare the Custom Action as the best option for getting our JavaScript files downloaded. The next section walks through how to accomplish this.

Downloading with a Custom Action

Adding the necessary HTML to our pages to cause our JavaScript files to be downloaded with a Custom Action element is easy:

Right click on your project name in Solution Explorer

Select Add | New Item

Select the SharePoint | 2010 category from the list of installed template categories

Select the Empty Element template

Change its name to MyJSFile

Click Add

Add the following to the Elements.xml file that Visual Studio creates and opens for you (placing it between the opening and closing Elements tags):

<CustomAction

Location="ScriptLink"

ScriptSrc="~Site/Scripts/JScript1.js"

Sequence="10"

Id="MyScriptLibrary"

/>

That’s it. SharePoint will now add the following to every HTML page in our site:

This will cause our custom JavaScript library to be download (if it is not already cached on the client) and made available for use. The functions in our library are now available to be called and executed. Notice in the Custom Action element the use of the token ~Site. When activated within SharePoint, this will be replaced with the URL of the site at which the Feature is activated. Another useful token is ~SiteCollection, which gets replaced with the URL of the site collection.

If you’re curious why SharePoint adds the tag using a document.write instead of simply adding the <script> tag, it’s a mechanism for overcoming some quirks in how browsers download and parse JavaScript files. Without going into too much detail (which I’ll do in another post), it prevents one script from blocking the download of other page elements (scripts, images, CSS, etc) while the script is downloaded and parsed. More on that in a separate post.

To test this out, simply hit F5 to deploy and run your code. After the browser opens to the homepage of your site, type the following into the address bar of the browser: javascript:sayHello(); and hit Enter. The results should look something like this:

Our JavaScript code in the library (the sayHello function defined earlier) ran and popped the alert box up on the screen. A very simple example, but sufficient to test out our process. We’ve successfully deployed a JavaScript library to SharePoint and had it downloaded to the client. Later in this series we’ll be developing more complex JavaScript libraries to begin offering real world functionality, but we’ll take advantage of the same mechanisms we’ve covered in this post for deployment and download.

Other Considerations

There are a number of tangential topics that need to be addressed in order to wrap up any discussion about deploying and downloading JavaScript libraries – improving download and caching, overall performance, etc. These topics will be addressed in parts 5 (Download & Caching), 6 (Efficiency & Performance) and 10 (Advanced Topics).

Wrap Up

This post has answered the question of “What is the best way to deploy my custom JavaScript to SharePoint and then have it get downloaded to the user’s browser?” We’ve discussed a number of options and whittled down to the best approach – document libraries and Custom Action elements. No matter what functionality we’re looking to introduce – form the simplest of examples (such as the project used here) to the most complex of scenarios, this approach will work equally well.

Introduction

As the title of this post states, this is an introduction to JavaScript and using it in SharePoint. It has some good information in it for any JavaScript developer, but it is very basic. If you are already familiar with JavaScript, go ahead and skim this post or skip it entirely. There likely won’t be a lot of new material here for you. If, however, you are just getting started with JavaScript, read on. You’ll probably learn a thing or two.

JavaScript is not a new technology, and yet it appears to be new to a lot of SharePoint developers. I honestly don’t know why. This series is going to take a look at JavaScript from a dev’s point of view and attempt to explain how it can be a very powerful tool in our toolbox. I’m going to repeat one point there just to make sure you got it: this is a developer’s point of view. That means no Content Editor Web Part (CEWP, the root of all evil). It also means Visual Studio, deploying Solution Packages, and generally approaching JavaScript as a coding language (which, after all, is what it is). If all you want to do is grab a script and toss it into a CEWP, this is not the right place for you. Please move along. We’re talking professional developer stuff here.

Starting right at the beginning – why would we use JavaScript in our SharePoint code when we have a rich server-side API available to us? That’s an easy one – because not everything can or should be done on the server. That’s not to say that the server side API is useless. Quite the contrary – both are necessary to deliver an effective, modern application (whether on SharePoint or not.) Despite what you may think, and what you may read elsewhere, neither JavaScript nor the server API can solve 100% of the business problems we face. Too, SharePoint is just a great big ASP.Net application so using JavaScript is not going out on the limb of unsupportability.

In recent years, JavaScript has grown up considerably. The addition of libraries such as JQuery, Dojo and MooTools have helped make JavaScript to become more popular. In the SharePoint world, SharePoint’s JavaScript Object Model has brought JavaScript to the foreground.

One quick point here about JQuery. Nobody anywhere has ever written a single line of JQuery. There is no such thing. JavaScript is a language. JQuery is a library. We use JQuery to make JavaScript easier to work with, but we don’t write JQuery. This may seem like a pedantic nit, but it is an important distinction for one reason: there is a wealth of JavaScript material on the web that you ignore at your peril. If you always look for a JQuery solution, you are almost certainly making your life harder. That’s not to say that JQuery isn’t a great library; it is. It’s just not always the answer. Sometimes a few well-crafted lines of JavaScript are all that is necessary. There’s not always a need to fall back on JQuery and it’s library of plug-ins.

What Can We DO With JavaScript in SharePoint?

Another easy one. Just about any UI manipulation we care to write the code for. This can range from pure eye-candy to something that provides actual business value. The only times you can’t manipulate the UI are:

Content delivered via a plug-in of some type (Silverlight, Flash, ActiveX, etc) (Although, some of these do offer a limited ability to manipulate them via JavaScript )

If JavaScript is turned off in the user’s browser

Anything else is fair game, and that is both boon and bane.

DOM Scripting

When you are manipulating the user interface, you are doing what is called DOM scripting or DOM manipulation. DOM stands for Document Object Model and it is the hierarchical representation of the HTML markup that the browser uses to render your page. Every object shown on the page (and also those not visible) are represented by a node nested inside the DOM. Changing the characteristics of those nodes, adding nodes and removing nodes all result in a changed interface being shown to the user – changed, that is, from what the server originally sent down to the browser.

What You Need to Know

At a very basic level, there are a few important things to know about using JavaScript:

JavaScript is an interpreted language and so there are no DLLs that need to be deployed to the server

JavaScript is downloaded to the user’s machine and parsed /executed there.

JavaScript is downloaded in clear-text (though it may be somewhat obfuscated). This is potentially a big deal as we’ll discuss in Part 12.

Although there is a standard definition of JavaScript (ECMA-262, version 5.1), different browsers will interpret your JavaScript code slightly differently. This has an impact on testing, as we’ll discuss in Part 8.

Once downloaded, JavaScript files are cached on the user’s machine to improve performance.

JavaScript is case-sensitive. HELLO is not the same as hello and not the same as Hello.

Another pedantic nit here: JavaScript is neither Java nor Java-lite. Other than some semantic similarities, it has nothing to do with Java.

Your First JavaScript Program

Enough introductory drivel. Let’s write some code. Fire up your browser of choice and navigate to any website. Assuming that you’re using a relatively modern browser (IE 7 or later , Firefox or Chrome) open up the developer tools: in IE and Chrome, simply hit the F12 key. In FF, hold down Ctrl and Shift and hit the K key.

In IE: Click the Script tag and then the Console button below it:

In Chrome, click the Console button:

In FF you don’t need to do anything, the Console is automatically shown:

In the browser/JavaScript console (where indicated in the above screenshots for your browser), type the following code:

alert("Hello World");

Now hit enter. You should see a small dialog box pop up that shows your message. Click OK to close the dialog box.

Congratulations, you’ve written your first JavaScript program. (We’ll revisit each of those Console windows later on in this series).

It’s time to get a little more advanced (but just a little).

Variables

Like any programming language worth it’s salt, JavaScript has variables. Declaring and using variables is easy:

var myVariable="Hello World";

alert(myVariable);

You can type that in the same Console and you should get the same results. The variable declaration (“var myVariable=”HelloWorld”) defines a variable which is then referenced in the following alert statement. At that point the value of the variable (in this case “Hello World”) is used by the alert. Go ahead and give it a try.

There are a few things that can get tricky about variables, but we’ll cover those later. For now, here are a couple of rules to remember:

JavaScript variables are not typed. In other words, you don’t declare the data type that the variable will store when you declare the variable, and changing data types is perfectly OK. The following is perfectly valid, although perhaps not a great practice:

var myVariable="Hello World";

alert(myVariable); //outputs HelloWorld, myVariable holds a string

myVariable=345;

alert (myVariable); //outputs 345, myVariable holds an integer

var is your friend. 99% of the time, you always want to preface your variable declarations with the var keyword. This declares the variable as local to your current scope. We’ll talk about this more later in part 5. For now, and until you understand the implications of not doing so, always use var

Null – variables are null if it contains an unknown value, which is not the same thing as undefined.

Undefined – undefined is a special value in JavaScript. It means the variable has not been declared (in the current scope – more on that in part 5), or has been declared but has not been given a value.

Functions

JavaScript is organized into functions. A function is a group of statements organized together in what essentially amounts to a named container. Here is an example of a function declaration and usage:

//declare the function:

function sayHello(){

alert("Hello World");

}

//call the function:

sayHello();

Passing a parameter to the function is equally simple:

//declare the function with a parameter:

function saySomething(thingToSay){

alert(thingToSay);

}

//call the function and pass in a parameter:

saySomething("Hello World");

We’ll touch on some advanced function declarations and usage in part 4.

Wrap Up

That’s a quick, high-level tour of the basic stuff you need to know to begin operating in a JavaScript world. There is obviously a lot more that needs to be covered – or else I wouldn’t have 14 more sections planned :-). We’ll get to all of that in later sections of this series. Hopefully this has gotten everyone on the same page. We’re pretty quickly going to move into the deep-end, so buckle up…

We all agree that SharePoint is a great application right out of the box, but is it more than that? Can SharePoint be used as a platform upon which we build web applications? The answer is most assuredly, YES. However, it’s not as cut and dried as we might like it to be. SharePoint has strengths and weaknesses that we must keep in mind. At times we may need to tweak our architecture a little to be able to take advantage of all that SharePoint offers. When does it make sense to do so, and when should we look elsewhere for a leg-up in our application building endeavors.

This post will begin to explore this scenario. It will not fully address every aspect, but it is a start. Future posts will continue to explore this thread, based on my own experience as well as comments left here and discussions I have with other developers.

One important point to make before going any further is that almost everything I cover in this post is available in SharePoint Foundation (SPF) – the free SKU (there are no server or client license costs for SPF). In some cases, I’ll talk about a capability of one of the SharePoint Server SKUs, but I’ll explicitly call those out. This means that you can use SPF and everything it gives you for no more than what it costs to buy ASP.Net – nothing.

To begin with, what are the common threads of most (if not all) web applications? What must any respectable web app include in order to be considered complete? A nice beginning list, in no particular order, would have to include:

Security – who are you and what are you allowed to do? AuthN and AuthZ

Data storage – a place for the stuff the app is responsible for managing

I’m sure there are more but those represent a nice general-purpose list of what web applications do. Let’s take a look at these one at a time from a SharePoint perspective.

Security

From an Authorization (AuthZ) perspective, SharePoint gives us a lot. It provides a whole collection of securable objects – from individual pages, to items to sites. In addition, it provides a collection of pages for managing security, assigning/revoking permissions, defining permission levels, managing group membership, etc), the concepts of roles and groups and a whole slew of other items. From an AuthN (authentication) perspective, SharePoint supports NTLM, Username/password and/or Claims based federated identity management out of the box. With little to no work, you can authenticate your users in whatever way makes the most sense for your application.

Data Storage

SharePoint provides a mechanism for storing data. Natively, it is not how most web applications store data, and perhaps not how most application architects “think” but it does store data. Instead of highly configurable relational tables, SharePoint stores rows and columns of data in lists. True, the structure of the database under the covers would make a DBA cringe, but we don’t deal with SharePoint data at that level. We remain higher up where the rows and columns analogy works pretty well.

With SharePoint 2010 you can get what I like to term “lightweight relations” between data points, but SharePoint is NOT a relational data store. I cannot emphasize that enough – SharePoint is not a relational data store. Don’t try to make it one. If you need relational data capabilities SharePoint is not the right place to store your data. That doesn’t mean, though, that SharePoint can’t present that data and allow users to interact with it. See the next section on Content Presentation for additional details on presenting external data.

From a purely “storing of data” point of view, SharePoint does it. You may need to rethink your data storage design and plans, but we’ll address that later.

Content Presentation

SharePoint can present data to end users like any other web application. Similar to just about any web application, the actual storage location of that data is irrelevant – which is why I broke this out into two sections. SharePoint can present data and whether that data is stored inside SharePoint or elsewhere doesn’t matter. Presenting data stored in SharePoint is easy and there are a number of out of the box ways to do that – Views, Web Parts, etc. Presenting data stored externally to SharePoint is still no harder than in any other web application (just write the database access code like you would in ASP.Net, PHP, whatever) and could be much easier if you can use Business Connectivity Services (BCS), which is now mostly available in SPF.

Much of this is possible out-of-the-box with no custom code. If the out-of-the-box content presentation capabilities don’t suit your needs, then you need to write some code – but if you’re not using SharePoint and instead developing the application in ASP.NET, you’re writing code anyway, so you’re no worse off.

Administration

Any non-trivial application requires administration to one degree or another. SharePoint provides a framework for delivering that administration in the model that it utilizes itself – application pages (which are really just standard ASPX pages). In addition, SharePoint provides an easy mechanism for securing those pages to allow only administrators to access them (RequireSiteAdministrator).

User Interface

Historically, SharePoint has been tarred with a reputation of delivering only “boxy” unfriendly user interfaces. All of that is over (beginning with 2007, but really coming into it’s own with 2010). Any UI you can deliver in ASP.NET, you can deliver in SharePoint. Period. End of story.

Performance

For anything but the most trivial of applications, performance is a concern. SharePoint provides some capability for cache management. For the most part, the actual caching is the same as what’s delivered in ASP.NET. SharePoint just adds a UI to allow you to administer the caching.

User Management

User management in SharePoint is primarily based around one of two areas – security and user information. Security is discussed above, and user information is handled via the User Profile Service (Server SKUs) or User Information List (SPF) of SharePoint. In either case, you have the means of managing information about your users that can be used as-is or enhanced (with or without code)

Additional Benefits of SharePoint

In addition to the above list of typical web applications elements, SharePoint provides a number of additional capabilities that can come in handy for some applications:

A fully baked Windows Workflow Foundation host to automate a business process A built in process timer manager that can execute tasks on a defined schedule A deployment management paradigm via Solution packages and Features

SharePoint Architectural Considerations

With all of that said, I don’t think there’s any doubt that SharePoint is a viable candidate for giving you a leg-up in your application building endeavors. However, like anything else, choosing SharePoint does require some trade-offs. Primarily I find that this comes into play in terms of interconnectedness. In other words SharePoint ties some things together that might not ordinarily be tied together. For example, when you define a Content Type, it has ramifications for the form used to present items of that Content Type. Sometimes this is a strength, sometimes it’s not. In addition, I try at all costs to avoid using some of the out-of-the-box columns that ship with SharePoint (like the ubiquitous Title column) as they just carry too much baggage. I find I’m far better off not using the default Title column and just creating my own.

SharePoint also has a tendency to force architectures down a certain path, that being a paradigm of multiple sites whereas a more traditional web application might be delivered in a single site. I reality, that’s just a semantic distinction. Sites in SharePoint are really just an organizational boundary and you don’t have to use multiple; you can often deliver everything within one site. In any case, your users never really need to know that they are visiting multiple sites. You can often hide it from all but the most observant of them (those that pay attention to URL strings).

Drawbacks of SharePoint as an App Platform

All is not perfect when using SharePoint as an app-platform, however. It does force you to rethink your applications somewhat and likely approach them in a new and different manner. In addition, SharePoint is a HUGE application and honestly some parts of it are better than others. In time, you get to know what parts to stay away from. Under the covers, there are some real head-scratchers and inconsistencies in the SharePoint object model. Again, you get to know SharePoint’s quirks as you work with it more. SharePoint also tends to lag about one release behind ASP.Net in access to the new things in the .NET Framework so if you find yourself constantly chasing those shiny objects, you’ll be forever tilting at the SharePoint windmill.

There are more drawbacks, and I’m sure each of you would come up with your own list. Honestly, I’ve been approaching SharePoint this way for so long that I forget what I use to rant and rave about. I look at what some of my ASP.Net friends have to do to build their applications and I just shake my head and say to myself – thank god I don’t have to deal with that anymore…

Conclusion

So that’s it. My take on SharePoint as an app-platform. Obviously I’m biased as I’ve already (mostly) made it over the hump and am comfortable with this model. I won’t say it was always an easy battle, but the end result has been well worth it.

Every so often you come across a discussion amongst developers that talk about eking every last drop of performance out of your code. I understand this mindset. It’s natural to want your code to perform as quickly and efficiently as possible. People don’t like to wait, and if we make them wait, they won’t like our application. If they don’t like our application, it’s kind of like we failed. No one likes to fail. Too, wait time is one of the easiest low-hanging fruit by which our code can be measured – “that must be bad code, look how long it takes to load a page.”

It’s time to call bullsh*t.

Performance is important, sure, but it’s not as important as many other things we, as developers, need to be concerned with. Once you reach a threshold of “acceptable” performance (more on this later), any time spent optimizing your code for faster load times is likely wasted effort. There are far more important things to spend that time on.

Other Factors

What else do we need to be concerned with? Lots of stuff:

Security

Application maintainability
Business Requirements

Overall end user experience (of which acceptable performance is just one piece)
etc.

This list isn’t in any particular order, and I’m sure you can come up with a few more things to add to it, but it gives you an idea of the types of things that you should be thinking about once your performance is “good enough.”

Where I’d rather spend my time

Of all the items in the list above, one area where I think people don’t spend enough time by a wide margin is application maintainability. Hopefully your application has a long life. Hopefully people use it for a long time and ask for new functionality or enhancements. In that time, they’ll likely come across a couple of bugs as well.

When this lifetime is measured in years, the hygiene and developer-friendliness of the original codebase are critically important. Too often, in the rush to hit a deadline, we cut corners in our development. Unfortunately, this makes our applications harder to maintain. How often have you inherited code from another developer, or had to revisit your own code 6 months after you wrote it, only to realize what a steaming pile of doo-doo you’ve got on your hands? I find myself in this situation fairly often and it’s rarely fun. How often have you had to spend an exorbitant amount of time debugging the codebase to find out exactly what is going on and the exact conditions that cause a bug to surface before you can even begin fixing it?

Wouldn’t it be better if instead of being dense and hard to follow, our code opened up before us like a road map and we could easily discern exactly what was going on? Wouldn’t it be better if, instead of spending hours tracking down exactly where in 20,000 lines of code we had to make a change to fix a bug, we could simply examine the logs and have them lead us exactly where we need to go?

That’s code that was written with an eye towards maintainability.

Here’s an example of what I’m talking about (click the image to enlarge):

This is an excerpt from the ULS logs of a sample application I wrote to demonstrate application maintainability. I added some color highlighting to the image to help you understand what is being written to the log:

Green is simple messages written into the logs to help a troubleshooter know what is going on and where in the code base this code came from
Blue is variable and parameter values. How nice is it to see actual values that can help you understand that a bug only shows up when a variable is greater than 100 and less than 200, for example. Seeing actual runtime values in the log can be invaluable.
Yellow is process trace message – one for each method I enter and again when I leave the method. Notice, too, that I get input and return parameter values for each method entry/exit (highlighted in blue).
What does all of this logging cost me in terms of performance? Roughly a tenth of a millisecond or so for each message. In other words, the 13 messages written for each iteration of my code (one iteration is shown above) cost me about 1.4 thousandths of a second:

In exchange for this miniscule amount of time, I get the rich details about the processing of my application. I’ll make that trade any day of the week and twice on Sundays.

I’ve been using this approach for projects lately and my current client is impressed with the speed with which I am able to resolve bugs. They almost don’t care that the bugs are showing up because I can get them resolved so quickly.

I’m not saying that this is perfect and I’m still tweaking the code that manages all of this for me (to be released as part of the CKS:API project I started last year) but I wanted to present it as an example of what is possible to give you an idea of what I’m talking about when I talk about maintainability as being more important than performance.

One final note on logging. To avoid filling up my log and slowing things down just a little more during routine production run time, I make use of the secret-squirrel VerboseEx logging level on the TraceSeverity enumeration. This level cannot be set through the UI so you need to do it in code. If a bug surfaces, I crank up the logging level and have the user repeat the steps that produced the bug. From my code’s point of view, nothing is different so we don’t have any issues with Heisenbugs.

More than Just Logging

The example above is simply logging and that is an important part of code maintainability, but it is not all there is. There is also:

Code comments – the oft-maligned part of a developers job. Code comments are important to help understand a codebase. I’m finding that, while I still comment my code, I do so less because the extra logging messages I’m adding can stand in for code comments. One benefit of this is that as my codebase evolves, I’m far more likely to update the inline process trace messages than I ever updated my comments. My comments tend to stay focused on big-picture elements of describing what a method does overall rather than the details of how it does it.

Defensive Coding – making sure that my code can continue running before attempting to do so and that the environment hasn’t changed in such a way that assumptions made by my code are no longer valid. Some of this is hinted at in the image above – the last green highlighted line says Contract satisfied – continuing with update. In my code, I’m checking both pre-conditions and post-conditions for my methods on entry/exit, as well as in the midst of processing to make sure that, for example, the list I need to work with hasn’t been deleted, or that my SPQuery has, in fact, returned items:

Don’t worry about the syntax there (the TrueIf object is another part of CKS:API that will be released publicly as soon as I can finish it). Just understand that I’m making sure I have a valid list item collection that contains items before I attempt to do anything with it. Does this cost me a few processor cycles? Sure, but it makes my code far more bulletproof and maintainable. While I don’t have numbers for this, I’d guess that all of this checking costs me no more than a handful of milliseconds.

Standards – I don’t particularly care what standards you code to, and honestly, no one outside your team should, either. The only thing that matters is that you as a team have standards and you adhere to them as closely as possible. BY standards, I’m talking about things like variable and method naming, approach to testing, commenting, etc. Within a project, I would recommend that you support the same standards. Cross project, it doesn’t so much matter. But it sure would be nice if all of your variables were named following the same pattern within a project. It just makes maintenance SO much easier.

Acceptable Performance

So what is “acceptable” performance? The answer is invariably “it depends.” Mostly it depends on the content you’re serving up. If you are the only source for that information, which is often the case with intranets – SharePoint’s most typical use, then you’ve got some leeway. This doesn’t mean you have license to abuse your users, you still need to stay within the realm of reasonable response times, but you’ve got more time than, say, a public facing website.

Your thoughts on this matter will almost certainly be different, but here are the metrics I shoot for:

Site home page: 2-3 seconds

Interior landing pages: 3-4 seconds

View pages: 3-4 seconds

Item pages: 2-3 seconds

Documents: depends on the size of the document

Again, let me repeat myself on a few points:

These are general guidelines I use, not hard-and-fast rules.

These are for internally facing sites. Public facing websites are a whole different ballgame

I use these guidelines to help determine how to architect my solutions; how I know that I can have my code spend a little more time making the applications easier to maintain; how I know that I should optimize things a little bit more because I’m not hitting them. Everything related to performance is a trade-off with something else. Knowing when I have some leeway is important.

Fixing Performance Problems

So far, everything I’ve mentioned so far falls into the bucket of slowing down your site (if only by a few hundred milliseconds). If you already have a performance problem, what options do you have?

Performance problems are fixable

But not always via code fixes. Sometimes that means adding more hardware – servers, RAM, CPU, faster disks, etc. Think about it this way – if you pay a developer $150/hour to fix performance problems and you estimate it requiring 80 hours, that will cost you $12,000. Depending on the nature of the problem, that money may be better spent simply increasing the RAM in your web servers or database servers.

Not all problems can be fixed simply by throwing more hardware at the problem. The trick is figuring out whether your particular performance problem is one that can. In general, more hardware will help if your servers (web or database) are heavily utilized and it is legitimate utilization. More hardware probably isn’t the right answer if your performance problems are the result of simply shoddy coding practices – not disposing of objects, poor use of collections, extra database hits, etc.

Perception vs. Reality

In few other places in our line of work is the distinction between perception and reality more important than a discussion about performance. The perception of speed is far more important than actual speed. If your users think your site is fast, then it’s fast; even if a stopwatch says differently.

What does this mean for us? Easy. Make your site seem fast, even if its not. This is where things like the JavaScript object model, partial page rendering, JQuery and asynchronous postbacks come into play. These are all tricks that can make your site seem faster, even when in reality they often slow things down.

Conclusion

Am I saying that you shouldn’t make your code run efficiently and as quickly as possible? No. Absolutely not. I’m saying that you shouldn’t be a slave to performance. There are more important things to worry about.

Call to Action

The next time you need to write some code, think about performance in light of everything else. Think about how adding some code that will reap huge benefits during maintenance or bug fixing are well worth the 100 or 200 milliseconds of processing time they take on page load. Actively work to make your code more maintainable and don’t worry quite so much whether a page take 2 or 2.5 seconds to load. No one will notice, except the developer who has to figure out what’s going on in the code 6 months later – the developer that may be you.

Welcome to OnSharePointDevelopment. I’m going to assume that you’re here for one of two reasons:

You’re a SharePoint developer who is interested in improving his/her craft

You’re an insomniac and this inane drivel helps you to finally fall asleep.

If the latter, feel free to keep reading until you nod off.

If the former, then let’s get down to business.

Why Are We Here?

I conceived of this site in the Fall of 2011 and started preparing presentations for the SharePoint user group I help to run. At the time, I was seeing a couple of trends develop in the SharePoint space, some of them encouraging, some of them discouraging. In either case, it was something that interested me and I felt like it was an area in which I could help. In no particular order, here’s what I noticed:

We were at a watershed moment in SP Dev – a cusp or turning point. SharePoint is an extremely powerful product and a game changer in many ways. It brought about significant changes in the way IT operates and developers weren’t keeping pace

Too much SharePoint development work was really, really horrible. Bad code is bad code, period. Unfortunately, the complexity and popularity of SharePoint have combined to produce an onslaught of really shoddy code and coding practices.

SharePoint was not slowing down. I really thought that SharePoint would have peaked and begun slowing down by 2011. It hasn’t, and isn’t showing and signs of it, either.

The Rise of the End User – SharePoint allows non- or less-technical end users to perform a dizzying array of customizations. While this is undeniably a good thing overall, it is not without its problems at the edges. The mantra of “just because you can doesn’t mean you should” applies, and is often the cause of friction.

Extreme polarization – The dividing line between end user, developer and administrator is, oddly enough, simultaneously blurring and becoming more pronounced. SharePoint allows a significant amount of customization without requiring a single line of managed code. This allows non-developers to do things which historically required a developer. Instead of causing an amalgamation of professionals into a more homogenous “SharePoint Pro”, this has, for some reason, caused the stratification of SharePoint professionals and users into distinct “camps”, and anyone in a different camp is generally wrong. It’s remarkably similar to the dividing line drawn between Republicans and Democrats in the political arena.

We were suffering an identity crisis – what did it mean to be a SharePoint developer? Did it mean traditional development in managed code? JavaScript? CSS? Cloud? The answer is a schizophrenia-inducing “all of the above – and more”.

A Journey of a Thousand Miles…

Paraphrasing Lao-tzu, the Chinese philosopher from the 6th century BC, a journey of a thousand miles begins with a single step. This is our collective first step… On SharePoint Development is a discussion on the what and why of SP Dev. It is my small attempt to help make us all better at our craft. There are dozens of decent sites/blogs/articles/books that can help you to figure out how to do anything you need to do in SharePoint. A simple Bing/Google search will return hundreds of results for any SharePoint-related query – some of them garbage, some of them gospel; most of them somewhere in between. What has been sorely missing, however, is an understanding of other aspects of getting the most out of this product, how to approach a problem, how to control process, how to get everyone on board and get the job done.

Rather than approach this in a scattershot manner, I’ve broken the journey down into sections:

Development Process – a large portion of the beginning of this journey is going to be focused on this section. I’m doing this for two reasons – it’s currently the area that interests me the most, and it’s also, I believe, the area most sorely lacking. Writing code is easy. It’s all the other stuff of making that code good, efficient, reusable, repeatable, testable, manageable, fixable, etc that is hard.

The SharePoint toolbox – SharePoint is the Prego of business software. (Just about) whatever you need…It’s In There. Understanding how best to use this tool and all that it gives you, however, is often where the problems come in. Too often, the immediate answer to the question “with SharePoint, how do I…” is it depends, followed be a litany of questions. Unfortunately, too often those questions are never asked and the developer runs off in the direction with which they are most comfortable, whether it is the right direction or not, only to find out later that their initial direction was flawed and they need to go back and start again. We need to fix this…

I intentionally sub-titled this blog “Insights from a Journeyman Coder” as it’s important to acknowledge that we’re all still learning. I expect to learn as much in the process of writing the content on this site as you will from reading it.

What To Expect

The posts on this site are, for the most part, prose versions of the lectures I’m giving at my user group and other venues; not word-for-word transcripts as they’re augmented with additional detail, but close enough. All of the main concepts come across intact. As appropriate, I link to existing content elsewhere on the web to help drive home points.

I’ll warn you right up front that you’re not going to get a whole lot of “tips & tricks” on this site – there are FAR too many of those types of sites elsewhere and that’s part of the problem

What I’m hoping to Achieve

My goal for the content on this site is to make you think differently about what you’re doing. No longer is “make it work” an acceptable end-goal. I want us all to think about where we’re trying to get to and figure out the best route to get there.

David Mann is a founder and director at Aptillon, Inc as well as the Owner and President of Mann Software, a boutique software consultancy located outside Philadelphia, PA. He has been a SharePoint MVP since 2007 and is a part time trainer for Critical Path Training. As a developer, software architect, author and trainer he has focused on Microsoft’s Information Worker and Collaboration stack, working with portal, collaboration, and content management technologies for over 15 years helping businesses to be more productive. Dave presents regularly at major and minor conferences and runs the SharePoint User Group in Philadelphia.