In this article we will discuss implementing “Alternate Access Mapping” or commonly known as “AAM” in SharePoint 2016.

If you are not aware of AAM or you have some misconceptions about it, I would highly recommend you to read through an excellent blog Alternate Access Mappings (AAMs) *Explained by “Brain Pendergrass” from Microsoft and would like to thank him for such an awesome blog to make me understand this concept in depth.

This article will focus on guided steps to configure AAM in SharePoint 2016 and we won’t discuss AAM as a concept.

Create New Web Application

Step 1: To start the demo let go to SharePoint Central Admin Site and click on “Manage Web Applications” as shown below-

I am creating a new Web Application for demo purpose. In real environments we can use any existing Web Application to perform these steps.

If operation completed successfully we can see popup window show success message

We can see this new Host added to the existing list

Add/Update IIS Bindings

Step 6: Add “IIS” Bindings

Now we have to add IIS bindings for the new Host. Search for “IIS” and select “Internet Information Services (IIS) Manager”

Select the Web Application that we have created in above steps from the “Connections” Panel on the left under “Sites” node

Click on “Bindings…” on the right to Add/Edit Web Application Bindings

In the “Site Bindings” screen select the default binding and click “Edit”

In the “Edit Site Binding” screen enter Host Name value and this should match the entry that we have created earlier in DNS

Click OK

Once saved the existing binding will look like as shown below-

Configure Alternate Access Mappings Using Central Admin

Step 7: Configure Alternate Access Mappings (From Central Admin)

Go To Central Admin -> “Application Management”

Under Web Applications Click “Configure alternate access mappings”

Click “Edit Public URLs”

In the “Edit Public Zone URLs” Screen and enter “Default Zone” URL with “Host Name” configured earlier as shown below.

Click “Save” once you entered the default Zone URL to save the data.

Now try accessing SharePoint Web Application using Public Zone URL and if the configuration goes well, the access will be granted to you

Test Alternate Access Mappings

Configure Another Mapping

Step 8: Repeat Step 5 to add another DNS entry to add a new Host Name

Step 9: Repeat Step 6 to Add Web Application Bindings

Configure Alternate Access Mappings Using PowerShell

Step 10: Configure Alternate Access Mappings (Using PowerShell)

Now we will add this new Host Name to the default zone for the web application

Launch SharePoint 2016 Management Shell

“New-SPAlternateURL” cmdlet gives us option to add new Alternate URLs to the required zone for web application

This cmdlet take following parameters

New Alternate URL – URL that you need to register as alternate URL

Web Application Path – URL that represent Web Application

Zone – Represents a zone that you need this alternate URL to add to

Once this command executes successfully we can a new Alternate Access Mapping added to the list

Test Alternate Access Mapping

Now if try to access this web application using this new mapping it still be translated to the same Public URL for the Web Application.

By following above steps we can enable a web application that receives the request from an internal URL in one of the five authenticated zones to return pages that contain links to the public URL for the zone

In this article we will understand how utilize a famous developer productivity tool called fiddler as REST API Test Client for SharePoint (though the target system could be anything with a valid REST API Endpoint)

Fiddler is primarily used as a Web Proxy that can allow you intercept REST API Request – Response Cycle. The usage of this tool has increase with shift in modern SharePoint development paradigms that favors more if Client Side Development Techniques/Strategies/Platforms rather than traditional Farm Solutions.

In this upcoming section of this article I will guide on how to use Fiddler to test REST API Call against SharePoint Data.

In this article we will explore only GET type of Requests only.

To start with this demo launch Fiddler and go to “Rules” Menu and Select “Automatically Authenticate”, this will let Fiddler to authenticate you against SharePoint based on the User Token stored once.

If this setting is not enabled you might encounter “401 UNAUTHORIZED” as shown below-

Also notice the request headers that are required to execute the SharePoint REST API Endpoint

Uploading large files to SharePoint On-Premise or Online is an obvious problem during data migration from any external systems like Lotus Notes.

Here is one of such errors which we might encounter while trying to upload a file of size greater than 250 MB-

In this article I will explain a data upload strategy where we can split a large file into multiple chunks of smaller size.

Solution Architecture Diagram

For better understanding we can refer to the following solution architecture diagram-

Based on this diagram we can conclude the following facts-
1. This solution can be hosted on multiple servers to launch parallel uploads
2. This solution can consume data from Network File Shares
3. Once data file is retrieved (say of size 300 MB), this solution will split the file (100 MB) automatically based on the pre-configured chunk size (which should not exceed the size limit of 250 MB)
4. Each chunk then appended to the file uploaded in multiple iterations

In order to start with this demo we would need a SharePoint Document Library in SharePoint Online (or On-Premise) Site as shown below-

Another prerequisite to demo is to have files of various sizes that we can use to upload to the document library.

I made use of following command line utility to generate files of various sizes. This utility takes destination folder path and size of the file in KBs as input.

Step 3: Loop through all the files and pass each file to the “UploadLargeFiles” function along with the document library name

Step 4: Generate unique upload id & get file name of the file to be uploaded

Step 5: Get handle on document library object and load the root folder (or any target folder) with in the document library

Step 6: Calculate the block size to be uploaded and total file size (as shown in the architecture diagram)

Step 7: Read the bytes from the source file and set the read buffer based on the block size

Step 8: Read the bytes based on the buffer limit that we set in earlier steps

Step 9: Check if this is the first chunk that is being uploaded, if yes then add a new file to SharePoint Document Library, get the file content based on the buffer size for the chunk and call “StartUpload” function that is defined under “Microsoft.SharePoint.Client.File” class. This will add the file to the document library but with small bunch of content only.

Step 10: Check if this is not the first chunk that is being uploaded, if yes then find the file in document library and get the handle on it

Step 11: If this is another chunk of data which is not the last chunk, this chunk will be appended to the same file by using “ContinueUpload” function that is defined under “Microsoft.SharePoint.Client.File” class. This will append the content to the file identified by Upload Id that we have initialized in earlier steps.

Step 12: If this is last chunk of data, this chunk will be appended to the same file by using “FinishUpload” function that is defined under “Microsoft.SharePoint.Client.File” class. This will append the content to the file identified by Upload Id that we have initialized in earlier steps and commits the changes to the file. Once this function completes successfully the changes will be made persistent to the file.

During one of my assignments I have come across a situation where we need to fix data issues in SharePoint Lists.

One of the issues that we found was presence of duplicate data. In order to fix that problem in hand I had developed a Powershell Script to find out duplicate data based on a specific or a group of columns.

For the sake of demo, I have added a SharePoint List with some duplicate records in it as shown below:

Now let’s look into the code to understand implementation details-

In Step 1 we are getting references of the Site and Web where the SharePoint List resides

In Step 2 we are splitting the list of columns based on which we want to find out the duplicate data

We can see there are two input variables “ColumnToValidate” and “ColumnToDisplay”. “ColumnToValidate” provides columns based on which duplicity needs to be checked while “ColumnToDisplay” contains the list of columns that needs to be the part of data export.

In Step 3 we are creating the export folder that will hold the CSV files exported with duplicate records

In Step 4 we are creating the list object that will give the handle on the list which needs to be validated

In Step 5 we are getting list of Items from SharePoint List and grouping them based on the validation columns

In Step 6 we are creating the directory for export files

In Step 7 we are exporting all the groups which is having item count greater than 1 (this logic identifies the duplicate items)

That is all for the code.

Now we will see the variation in outputs depending on the columns specified for duplicacy check

In Step 8 we specify the validation and display columns, for the first execution we will check duplicate values in “Title” column

In Step 9 we are calling the “Get-DuplicateListItems” function to find the duplicate values

After the function executed successfully we can see the following output.

In Step 10 we can see the output of this excution and can see 6 items found which duplicate in Title Column

In Step 11 we can see the CSV file that is exported by the execution considering “Title” Column to be validated.

In Step 12 we can see the output file and can notice duplicate values in “Title” Column

In Step 13 we have changed the list of columns to be validated. In this second execution I have added another column “Role”.

Now the list will be validated for duplicity based on the combination of “Title & Role” Columns

In Step 14 we can see the output of this excution and can see 4 items found which duplicate in “Title & Role” Columns

In Step 15 we can see the CSV file that is exported by the execution considering “Title & Role” Column to be validated

In Step 16 we can see the output file and can notice duplicate values in “Title & Role” Column

In Step 17 we have changed the list of columns to be validated. In this second execution I have added another column “Location”.

Now the list will be validated for duplicity based on the combination of “Title & Role & Location” Columns

In Step 18 we can see the output of this excution and can see 2 items found which duplicate in “Title & Role & Location” Columns

In Step 19 we can see the CSV file that is exported by the execution considering “Title & Role & Location” Column to be validated

In Step 20 we can see the output file and can notice duplicate values in “Title & Role & Location” Column

This is a very simple technique that can be used to fix one of the issues with SharePoint List data.

In this article we will discuss another obvious performance issues with SharePoint Solutions involving large volume of data transactions surfacing SharePoint Custom Pages.

This could become more prominent if we have strict governance in place and we are not allowed to make use of advanced server side options (Custom Web Service End Point, MTA Enabled Modules etc.).

In one of the recent assignment I came across a similar scenario where I need to crawl data from an external Web Service end Point and surface data on SharePoint Pages. Since the anticipated data volume was huge and traditional caching approaches like Cookies wont’ work due to size limitations.

In pursuit of the solution I have gone through the “HTML5 Web Storage APIs” that allows you to setup an In-Browser Transactional Database System called “IndexedDB”.

Here is a quick introduction of IndexedDB for details I must recommend you to visit IndexedDB

“IndexedDB is a transactional database system, like an SQL-based RDBMS. However, unlike SQL-based RDBMSes, which use fixed-column tables, IndexedDB is a JavaScript-based object-oriented database. IndexedDB lets you store and retrieve objects that are indexed with a key; any objects supported by the structured clone algorithm can be stored. Operations performed using IndexedDB are done asynchronously, so as not to block applications.”

I also want to thanks to “Raymond Camden” for his detailed research on Storage Limits for IndexedDB and believe you must refer this link to understand the limits carefully before getting into concrete implementations.

Now let’s try to understand the implementation details by using following diagram:

Solution Architecture Diagram & Explanation

In this solution the SharePoint Page will try to look for the required data in Local Indexed DB created to support this page. If data is not found in local database, page will issue the request for data from SharePoint List.

Since we are dealing with “100,000” Items present in SharePoint List, I made use of “REST API + OData Continuation” data access strategy to overcome SharePoint List Threshold Limits. This mechanism will access only 100 List Items at a time and it is safe to extend this limit up to 2000 items per fetch.

Each fetch will a JSON Object that will be persisted into Indexed DB as individual record. I opt this strategy to reduce the page load time. If the number of items are not much you can add each item as separate record.

Every subsequent data call will be automatically diverted to the local database as primary source.

Additionally we can add “Auto Refresh Modules” to keep the local database fresh with SharePoint List Changes and sync the changes with Indexed DB “Asynchronously”.

Ideally speaking for a complete solution “Auto Refresh Modules” are must to have.

So this all about execution summary for this solution.

Now let’s have look at implementation details as follows-

I have created a SharePoint List with two columns and “100,000” Items added to it as shown below.

Demo

This list will be acting as data source for the page. In actual scenarios this source could be a Web Service End Point which can provide voluminous data on demand.

Before getting into code let’s see how this Page will behave on execution. Demonstrating the page in action will be helpful later when we get a deep dive in code.

If we run the page we will see this page took about “3 minutes” to get execution completed.

The first execution cycle will include the following actions:

Initialize IndexedDB Database

Query SharePoint List

Add REST API Response to IndexedDB

Load page with data from IndexedDB

Since we are adding data to the store asynchronously, overall application will remain functional even it is taking 3 minutes to complete.

Following screen shot showing data adding to IndexedDB asynchronously

We can also review the Indexed DB initialized as the part of this request using “Developer Tools or F12 Key” with in the browser as shown below-

We can explore each item in the each of the JSON Object as shown below-

Now refresh the page to see the execution again and we can see roughly “1 second” to complete the page request.

The subsequent execution cycle will include the following actions:

Query IndexedDB for data

Load page with data from IndexedDB

So we can see how we can trim the execution path by using a well-defined strategy.

Code Analysis

Let’s do the code analysis to understand the concrete implementation.

In Step 1 we are enclosing some of the literals as variables and will refer theses variables later in the code

In Step 2 we are checking if respective Indexed Database is initialized already or not and if not Initialize the Database. In this demos let’s call this database as “Products”

In Step 3“onsuccess” event handler will get executed and database object will get stored in a global variable “SharePointOptimization.sharePointStore”. This variable will be acting as start point for all the operations on the database in future.

In Step 6 we are making use of OData Continuation Techniques to overcome SharePoint List Threshold restrictions.

In this step we also call “AddDataToStore” function that will add SharePoint List Items coming as JSON Object to the Local Indexed Database. It is important to recall that in this demo I am storing 1 JSON Object as 1 record in database and each object contains information for 100 List Items.

In Step 7 we are adding JSON Objects to IndexedDB. In order to do that we need to perform following operations-

Initialize Transaction with Read Write Operation Permissions

Get Handle on “Products” Database inside IndexedDB Data Stores

Call asynchronous “add” method to add JSON Object to “Products” Store

In Step 8 we are calling “QuerySharePoint” function to query data from SharePoint List in case data is not available in Local Database.

Steps 9, 10, 11 explains about “ReadSPStore” function where we will read the data from Local Data Store (IndexedDB)

In Step 9 following operations are performed-

Initialize Transaction with Read Operation Permissions

Get Handle on “Products” Database inside IndexedDB Data Stores

Call asynchronous “count” method to get total number of JSON Object available in “Products” Store

If success read the record from IndexedDB and add to the local array variable

Call “continue” function as long as there are items left in local store

Once all data is read and save to the local array pass this array to “RenderUI” function to render this data on the interface as required

In Step 12 we can plug any UI engine to produce more intuitive UI as applicable, for the sake of this demo I am writing out the Count of Store records * 100 (since each record contains 100 Items) to show the total number of items stored in the local store.

Steps 13, 14, 15 show you a helper function to check if local store contains required data or not. It helps to decide if we need to read data from Local Store or SharePoint List

“GetProductCount” function is quite similar to the “ReadSPStore” function except it perform a lesser number of operations

In Step 16 we will initialize Local SharePoint Store by calling “InitializeSharePointStore” function

In Step 17 we can see some of the UI elements to build a basic UI for this demo

Point of caution

Before implementing this mechanism make sure you have identified all the compatibility issues around this corner.

I would recommend you to refer the following site every now and then to make sure you are using features supported by the targeted browsers.

This demo is about another tool that I worked out during an assignment while working with an integration scenario using web services supporting Basic Authentication.

Problem with basic authentication is that you have to keep username and password stored somewhere in order to generate the authentication token.

Since I was integrating external web services with SharePoint so I felt to delegate the Token Generation Process to an external tool and consume the Authentication Token directly with out keep user name and password to be stored in the code itself.

To run this demo I have created a simple interface that takes username and password as input and generates the Basic Authentication Token and will display it in “Authentication Token” section

Now when we click “Get Authentication Token” button we will see the authentication token in the “Authentication Token” section.

This token can be used for any Web Service supporting basic authentication, and this strategy can be merged with other functionalities too in order to generate this token on the fly.
Hope you find it helpful.

Recently I have developed a couple of PowerShell based components that will serve as data crawlers for federated data sources like External Web Services, SQL Server Databases, and Excel Workbooks & SharePoint Lists.

In order to authenticate the Service Accounts against all of these sources I had no choice but to embed the User Name and Passwords with in the PowerShell Code in plain text. It gets even worst when few of the Web Services could support only “Basic Authentication”.

Saving passwords in plain text to code files could lead us to the Compliance Issues and could get the solutions rejected eventually.

In order to fix this issue I have implemented a couple of mechanism to deal with each type of Authentication requirements.

In this article I will discuss the mechanism to authenticate the requests to SharePoint Lists.

In order to simplify this demo let’s consider a simple scenario where I am having a list “MyLocations” as shown below and I need to export its metadata using a PowerShell based component.

To keep the content crisp I will walk you through the specific section from code and skipping all the CSOM specific code which you can refer in my earlier articles if you like.

I have intentionally divided this implementation into two separate code files in order to keep the passwords safe from the developers. Intent is to get the Encryption File generated by the SharePoint Admins and provided these files to developers for so that they can use it in code directly as shown below.

In the following code snippet you can see the commands to encrypt password “12345678” and export it to a text file “BANSALP.txt”

This file would look like as shown below:

This way you can store passwords for all required service accounts in different text files without violating Security Compliance.

Now in order to pass this encrypted password to SharePoint for authentication we can make use of “System.Management.Automation.PSCredential” Class as shown below.

Here “Get-Content” Command let is used to read the content from “BANSALP.txt” file and “ConvertTo-SecureString” Command let to get the encrypted password as secure string

Once credential Object has been created we can assign this credential object to SharePoint Client Context “Credentials” Property

With this Client Context SharePoint Authenticates the incoming request based on the ACL of the requestor

Following is the outcome of the call that we have send to SharePoint:

I have exported the metadata to a “csv” file as well that would look like this.