Few weeks ago I had a nasty bug on customer premises where a combination of SharePoint 2013 and ASP.NET UpdatePanel control resulted in some very weird behaviour.

THE SYMPTOMS

Our project consisted of several full-trust code ASPX pages, dutifully placed in _layouts folder. Two of these pages had ASP.NET AJAX UpdatePanel controls to give “a kind of” smooth user interaction experience. The reason to use it was the simplicity of wrapping the existing SharePoint server controls such as grids and buttons in the update panel to prevent page reloads on postbacks. (Yes, I know it’s legacy technology but still, sometimes it’s the best and simplest solution for a given problem).

However, these pages gave random errors as if their event was somehow lost between the UpdatePanel client-side code and the server-side postback. Randomly, we had errors in our pages, with no discernible pattern. The only clue was in the ULS logs, where there were no user claims associated with the request, as if it were a non-authenticated call. What’s worse, as it had no clear cause, our reproduction procedure was to mindlessly click the buttons in the hope of a random error. Needless to say, it was way too inefficient.

Googling around, I found a way to reproduce the errors using Fiddler HTTP debugging tool. The sequence of steps is to make the first page load with Fiddler running in the background. Then, I deleted all the captured traces in Fiddler and clicked the button inside UpdatePanel. Immediately, the UpdatePanel threw the dreaded error.

THE CAUSE

According to the Microsoft KB3062825 article, the cause is the mismatch between the HTTP protocol implementation in UpdatePanel and SharePoint 2013, where instead of returning a 401 code in the handshake, SharePoint returns 200 with the login redirect and the UpdatePanel loses its changes. It only applied to a certain combination of SharePoint 2013 and .NET Framework, and even in these combination it wasn’t certain that the error would happen.

Unfortunately, we were in that situation, with the right combination of SharePoint and .NET versions.

THE SOLUTION

The solution is simple: applying a specific .NET Framework hotfix (for Windows Server 2012 R2, for Windows Server 2012) that’s meant to patch the HTTP protocol handling causing the error. The patch is available via Microsoft support, but it’s not publicly available. I think that it’s because the error isn’t deterministic enough to be included in a regular cumulative update or service pack.

My session was about connecting IoT to Office 365 (via Azure). I used an Intel Galileo prototyping board with a Passive Infrared sensor (PIR). The sensor data was used to determine if a meeting room was empty or occupied. The raw data is uploaded by Galileo to an Azure Notification Hub. A continuously-running Stream Analytics job is then used to translate the raw data to 1-minute resolution of the room availability and to insert this data into Azure Table Storage. Finally, a provider-hosted Office 365 SharePoint application is used to visualize the room availability.

Another weird SharePoint app bug happened yesterday. The solution was fairly easy once you know what’s going on, but it’s just weird altogether.

SYMPTOMS

You have a custom app in your SharePoint 2013 App Catalog.

A Custom App Inside App Catalog

You want to add this app to a SharePoint site. You can’t find your app in the “From Your Organization” section when you click at “Add an app” in a site.

The App Is Missing From “Your Apps”

CAUSE

I first suspected that the current user doesn’t have permissions to add an app. However, the user is the site collection administration and thus has the permission to install an app.

Yet…a slight detail. The App Catalog site is, well, a SharePoint site. With its own permissions. And, by default, containing only the user who created the catalog in the first place (the system admin).

So, the current user, although a site collection admin, doesn’t have permissions to read from the app catalog. (This is the weird part, as I expected SharePoint to do the reading using a system account behind the scenes.)

SOLUTION

Add the users that should be able to install your custom apps to the site permissions of the App Catalog site, with Read permission level. In my case it was “Dani Alves” (yes, I’m a Barcelona fan).

Now, the app is visible in “Your Apps” when you try to add it to a site. Yeah!

A puzzling SharePoint search alert behaviour was keeping our team of three scratching our heads for days. If you speak Spanish, you can check my teammates’ blog posts about this same error: Ignasi and Miguel.

SYMPTOMS

You have a SharePoint search configured correctly in SharePoint 2010/2013. You search for a keyword and create a search alert for the results.

You make more than 10 changes that should trigger the alert. However, the search alert email you receive lists only 10 changes. The rest of the changes are not sent as alerts.

CAUSE

The Search Alert mechanism in SharePoint 2010/2013 is described in great detail in the following MSDN article. In a nutshell, the process is as follows:

The alert is invoked once a day ("daily summary") or once a week ("weekly summary").

The alert runs the query again with the user-supplied search criteria (in my example case: "Hello").

If there are no new results (the results that are more recent than the last time the alert was run), the alert finishes.

If there are new results, they are formatted according to the search alert email template and sent to the user.

There is one tiny bit of missing information here. The step 2, where the search query is run again, has a hidden parameter that limits the number of results that are returned from the search index. If you look inside the SharePoint code that processes the alert, you will find a RowLimit parameter supplied to the query:

Another interesting issue arose the last week. I was tasked with implementing a BCS .NET connector to a OTRS web issue tracking service, as I mentioned earlier. The icing on the cake was extracting the binary data (issue attachments and images) and showing them in SharePoint leveraging Business Connectivity Services (BCS) in SharePoint.

I found out a post on how to use BCS to expose SQL Server data, which was not applicable in my case. I also had the extra difficulty of having the attachments in a child entity. The OTRS Ticket was the primary entity in my model, with OTRS Article being a child entity with 1:N relation in between). The attachments were properties of the article in OTRS but in the model I attached them to the Ticket in order to be more accessible.

In this post I will show you how to achieve these goals step by step with Visual Studio.

What we want to achieve?

This is the model that I’d like to end with. It has a main entity called Product and a set of child entities called Photo. The child entity has a compound key of both the ProductId and it’s own PhotoId. It also has a binary field called PhotoData together with MIMEType and FileName fields that will govern how to expose the photo to the browser.

Those are the minimum three components for binary data BCS compatibility: MIME type, binary content and a file name.

We will model these two entities in a custom NET assembly connector. For brevity, I will fake the external service and return hardcoded data read from picture files embedded inside the connector.

Building the Connector

The first step is to create a new Business Data Connectivity Model named ProductModel in Visual Studio.

Visual Studio will create a new entity called Entity1 and will implement the entity sample methods X and Y, together with a NET implementation of the entity. The main problem with BCS development is that the metadata has to match hand-in-glove with the implementation, and also the BCS metadata has to match internally.

We will begin filling the Entity1 and changing its name to Product. We’ll also change the name of the model from BdcModel1 to ProductModel. After the renaming of several files and nodes in the BDC Explorer, we’ll have something like this.

As you can see, the Product entity has ReadItem (gets a product by its ID) and ReadList methods (gets a list of products). The methods are declared in the BDC model (left side) and their code will reside in ProductService.cs class.

We will model the Product entity operations first, as every change in the model triggers a change in the code that is generated in ProductService class. First, we’ll change the Identifier1 field of the Product entity into a ProductId of Int32 type.

Modeling Mayhem

Then, in the BDCM editor we’ll select the methods and fill their details in the BDC Method Details pane. This is the tricky part of modeling BDC: it’s very easy to do this wrong. Luckily, the BDC Explorer lets us copy and paste metadata to save time. First, we will model a ReadList operation. It will take one no parameters and will return a "Return" parameter of type "Collection of Product", which will be our entity metadata. Take a look.

When we edit the metadata, we have the following BDC Explorer tree:

Here we have to change the type and the name of Entity1List, Entity1, Identifier1 and Message. They should be: ProductList, Product, ProductId (Int32)and ProductName (String). The change is done in Properties window (F4) and we should change the Name and Type Name property. When changing the Type Name you have to choose the entities from "Current Project" tab. For collections (such as ProductList) you should select the entity and check "Is Enumerable" list.

Note: when modeling the ProductId, you also have to specify that that property maps to the Identifier (and the entity that it refers to, i.e. Product).

Now, we have to change the same thing (without the collection, of course) for the ReadItem method. It should take one parameter (mapped to String) and return a Product. The good news is that we can copy and paste the Product node from ReadList into ReadItem method.

The underlying code class Product.cs and the service ProductService.cs have to be changed to include "hardcoded" data:

At this moment we have a workable connector that exposes products and product details, but nothing else. We will do a quick check by deploying the connector and creating a new external list in SharePoint.

Well done! Now we have to model the photos 🙂

Adding the Photo Entity to the model

First, we’ll add a new class to the project, with the following simple properties:

Then, we add a entity in the BDCM model canvas right-clicking it and choosing "Add / Entity":

Of course, we have to change the name and add the properties of the entity. We have to add both identifiers, the PhotoId and the ProductId. They both have to refer to the Photo entity, and in the association we will let BDC know that it will provide the value of ProductId when the association is navigated.

I have also added a ReadItem method.

Even if the association is necessary, you still have to model the ReadItem method in advance and add an instance of that method, which should be of SpecificFinder type. We will take 2 In parameters with the 2 identifiers of the Photo entity and we will return an instance of Photo class, with all its fields.

We’ll add the association between Product and Photo entities now, right-clicking again on the BDCM canvas:

In the dialog, we’ll make sure that the association is correct and in this case we will only have the navigation from Product to the Photo, not the other way around. We’ll remove the extra navigation method (the last one) and we will uncheck the Foreign Key association, as the ProductIds are returned in the code for the association method in the Product class.

Now we have a new method called ProductToPhoto in the Product entity that returns a list of photos for that product.

We still have to do the "boring" stuff of mapping the return types in the BDC Explorer pane:

After that, we have to write the code for the ProductToPhoto method. At the moment we won’t be showing the photo yet, so we can set the BLOB array to null.

Ready to roll! Deploy the solution to SharePoint and create External Content Type Profile pages in the BDC Service Application (Central Administration). It will automatically add the related Photos to the Product in its profile page.

We have to delete and recreate the external list. Now we can go to the View Profile action and see the details of the product and its photos:

Reading the photos

The only thing missing is the link to see the actual photo (the BLOB content). We have to add a StreamAccessor method and a method instance.

We can’t add this method in the entity designer. We have to open the BDCM file as an XML file and then add the Method and MethodInstance nodes to it.

As you can see, we return a Stream with the data. We have two additional instance properties that specify which entity property is the MIME type and which one is the file name.

Check the mappings: both identifiers should be mapped to Photo entity and both as parameters and return values in the entities (for ReadItem method). If not, it will complain in runtime about "Expected 2 identifiers and found only 1". It took me some time to solve that one!

In our PhotoService.cs class we have to add the method that returns a Stream with the data. In my case I use a Base64 string with a small sailboat image in PNG format, using the excellent web site that allows you to encode the image into a string http://www.base64-image.de/step-1.php. I use the Convert .NET class to convert that string into the original array of bytes. (In this snippet I have shortened the string for legibility):

If you have custom JavaScript file loaded in your master page, as we usually do in SharePoint, you might have stumbled upon the problems with custom JS and SharePoint 2013 new Minimum Download Strategy (MDS).

By default, MDS is enabled on Team Sites in SharePoint 2013 and allows for refreshing data on SharePoint pages without causing reloads. But, in order to do so, all the content and JavaScript in MDS pages must play along nicely. If not, the symptoms include:

blank page on loading custom JS in a MDS-enabled page

custom JS script not loading

The solution

The first part is to use ScriptLink control in the master page, instead of using script tags directly. Specify "LoadAfterUI" attribute in order for the script to be loaded after the page is loaded in MDS.

I just had a strange error the other day, deploying Business Connectivity Services (BCS) model arranged around a NET assembly. When accessing the external list data, I found the following error:

Assembly was requested for LobSystem with Name 'Namespace.LobSystem', but this assembly was not returned. SystemUtility of Type 'Microsoft.SharePoint.BusinessData.SystemSpecific.DotNetAssembly.DotNetAssemblySystemUtility' requires the assembly to be uploaded.

Of course, I checked the assembly and it was loaded in the GAC. So, where’s the error coming from?

Well, our friend BCS registers the assemblies for your external content type when you activate the feature containing your BCS Model and Assembly. This feature is made automatically when you create a new BCS project in Visual Studio. The feature has a custom feature receiver and also has a custom entry in the feature.xml declaration.

My error was renaming the model in some point of time. It went well for the model XML, but the old name ("BdcModel1") was still remaining in the feature.xml. After manually editing the feature.xml and pointing it to the new name of the LOB System, the error was gone:

As you probably know from my previous posts, I have been configuring a wide-scale document management solution using Content Organizer feature of SharePoint. The idea is to use managed metadata to tag the document with information about the business unit and region it originates from (in my case Region and Section metadata columns) and let SharePoint classify it to the correct site and document library. I wrote about how to expose the cross-site content organizer hubs a few months ago.

As I had many regions and many sections to configure, I had to manage to build the whole hierarchy with a PowerShell script instead of doing the work by hand. In this post I will share with you the things I learnt by doing so.

Anatomy of the Content Organizer Rules

When you click the Content Organizer Rules in Site Settings, it will show you the contents of a hidden list called "Content Organizer Rules". You can see it adding /RoutingRules to the site URL.

Each rule is a list item with several fields of importance:

RoutingEnabled: this column should be set to 0 or 1 in order to disable or enable the rule.

RoutingPriority: a number from 1 to 10. 1 is the highest priority and 10 is the lowest one. A rule with higher priority will run BEFORE any rules with lower priority.

RoutingRuleName: a string with the rule name.

RoutingContentType: if the rule applies to a specific content type, here you should put the content type name (not the ID).

RoutingContentTypeInternal: this is the content type ID for the content type specified in the RoutingContentType field concatenated with the content type name using the pipe ‘|’ character as separator

RoutingRuleExternal: if the content organizer rule should route to another site, this field should be set to 1. Set it to 0 if the routing is done in the same site as the rule.

RoutingTargetLibrary: the destination library if the document is routed in the same site.

RoutingTargetFolder: the destination folder if the folder classification is used.

RoutingTargetPath: if you are routing to another site, this should be the name of the content organizer source (set in Central Administration).

RoutingConditions: this is an XML string with the routing condition. The XML syntax will be explained later.

In order to programatically create these rules, SharePoint server object model exposes a class named EcmDocumentRouterRule. This class simply surfaces the underlying list columns as class properties.

In order to create a new rule, just instantiate the EcmDocumentRouterRule passing the SharePoint site for the rule in the constructor (the SPWeb object). Populate the properties and call the Update() method on the rule. Job done!

Rule Condition XML

The most complex part of the creating the rule in code is how to correctly construct the condition XML. It should be in the form of:

All the children nodes of Conditions are evaluated together (i.e. it’s an AND of all of them). Each condition specifies the Column that is evaluated, the operator and the value to compare against.

The column is comprised of several pieces: field GUID|field internal name|field display name. (in my case it was ‘5bc078e1-bcf6-4475-aadf-2b567726c696|Region|Region’)

The available operators are:

IsEqual

IsNotEqual

GreaterThan

LessThan

GreaterThanOrEqual

LessThanOrEqual

BeginsWith

NotBeginsWith

EndsWith

NotEndsWith

Contains

NotContains

EqualsOrIsAChildOf

NotEqualsOrIsAChildOf

IsEmpty

IsNotEmpty

ContainsAny

ContainsAnyOrChildOf

ContainsAllOrChildOf

The value is the literal value to compare against. If you are comparing against managed metadata columns, the value specified as ’16;#Austria|1953bfe9-95d0-4ec8-8b9f-7a58169a9a53Region’. The left part is the underlying lookup value for the managed metadata field and the right part is the Term ID for the selected term.

Note: all the managed metadata columns in SharePoint are implemented as lookup columns to a site-collection root site hidden list. Every time a new managed metadata value is added to a list item, SharePoint adds a new entry to this hidden list. The consequence of this implementation is that you must get the lookup value IDs to make a CAML query against managed metadata using TaxonomyField.GetWssIdsOfTerm method.

Putting it all together

So, let’s say that we have the text value of the managed metadata column and we want to create a content organizer rule that will route the document with the content type "My Content Type" to a different site when the value of that column matches our text value or its children (in my case the column name and the term set name is Region and its value is ‘Spain’). How should we construct our PowerShell script to do it?

First of all, we have to retrieve the Term ID that corresponds to the taxonomy node for the term set "Region" with the value of "Spain". In the $web variable we have the SPWeb object for the site we want to create the rule for.

Now we have the term ID that corresponds to ‘Spain’ entry in the ‘Region’ term set in the ‘Group Name’ term set group. Now we need to construct the full literal value of the managed metadata column (the lookup part and the GUID part). In order to do so, we use the TaxonomyFieldValue.PopulateFromLabelGuidPair method. It consists of the text value (‘Spain’) and its Guid separated by the pipe character (‘|’).

Now we can create the content organizer rule. Remember that the column name for the condition XML is the field Guid, static name and display name. In this case we route the document to another content organizer called ‘Spain’, that’s why the rule is marked as External. The value for the taxonomy field is retrieved using the ValidatedString property of the TaxonomyFieldValue object.