Category Archives: Performance

Post navigation

The history of a task is tracked by events. These events can be viewed in Task Scheduler for each task to track when the task was registered, run, and when it completed or failed. The progress of a task can be monitored through its history. A task can be controlled by running or stopping the task manually (on-demand).

EventDetails

Product: Windows Operating SystemID: 111Source: Microsoft-Windows-TaskSchedulerVersion: 6.1SymbolicName: JOB_TERMINATIONMessage: Task Scheduler terminated the “%2” instance of the “%1” task due to exceeding the time allocated for execution, as configured in the task definition. Increase the configured task timeout or investigate external reasons for the delay.

Resolve

Fix task configuration settings

The task was stopped due to a configured setting. Possible causes include:
•The task ran for longer than the maximum configured run time.
•The task was configured to stop when the computer switched to battery power.
•The task was configured to stop when the computer is no longer idle.
•The task was configured to stop when a new instance of the task is triggered.

This behavior might be as expected. However, if the behavior was unexpected you can reconfigure the task configuration settings.

To update the task settings and conditions:

Click the Start button and type Task Scheduler in the Start Search box.

Select the Task Scheduler program to start Task Scheduler.

Select the task to configure by locating the task in the task folder hierarchy. Right-click the task, and select Properties.

On the Settings and Conditions tabs, update the task settings and conditions.

Click OK.

Verify

To verify that the execution of a task has completed as expected:

Click the Start button and type Task Scheduler in the Start Search box.

Select the Task Scheduler program to start Task Scheduler.

Select the task to run by locating the task in the task folder hierarchy.

On the Actions menu click Run. You can also click Run in the Actions pane.

Click the History tab for the task to verify that it contains events indicating the task was registered successfully. Also, ensure that the task completed successfully or that the task timed out as expected.

The history of a task is tracked by events. These events can be viewed in Task Scheduler for each task to track when the task was registered, run, and when it completed or failed. The progress of a task can be monitored through its history. A task can be controlled by running or stopping the task manually (on-demand).

We can access and analyze several query and crawl health reports, logs and usage reports from the Search service application in the SharePoint Central Administration to monitor the health of the search system.

The health reports and logs only contain information after a full crawl has completed. To run a full crawl, we have to set up a Search service application, add at least one content source, and then start a full crawl.

To view the health reports and the crawl log, one have to be an administrator of the Search service application. Alternatively, an administrator who is a member of the Farm Administrators group can grant user accounts Read permissions on the Search service application. A user account that has Read permissions can only view the Search service application status page, the health reports and the crawl log.

Query health reports:

Trend

Overall

Main Flow

Federation

SharePoint Search Provider

People Search Provider

Index Engine

To view query health reports:

Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.

In Central Administration, under Application Management, click Manage service applications.

On the Service Applications page, click the Search service application.

On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Query Health Reports.

On the Search Service Application: Query Latency Trend page, click the query report that you want to view.

The following table shows which reports are available.

Crawl health reports:

SharePoint 2013 provides the following reports about crawl health:

Crawl Rate

Crawl Latency

Crawl Queue

Crawl Freshness

Content Processing Activity

CPU and Memory Load

Continuous Crawl

To view crawl health reports

Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.

In Central Administration, under Application Management, click Manage service applications.

On the Service Applications page, click the Search service application.

On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Crawl Health Reports.

On the Search Service Application: Crawl Reports page, click the crawl health report that you want to view.

The following table shows which reports are available.

Crawl log:

The crawl log tracks information about the status of crawled content. This log lets you determine whether crawled content was successfully added to the index, whether it was excluded because of a crawl rule, or whether indexing failed because of an error. The crawl log also contains information such as the time of the last successful crawl and whether any crawl rules were applied. You can use the crawl log to diagnose problems with the search experience.

To view the crawl log

Verify that the user account that is performing this procedure is an administrator of the Search service application, or has Read permissions to it.

In Central Administration, under Application Management, click Manage service applications.

On the Service Applications page, click the Search service application.

On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Crawl Log.

On the Crawl Log – Content Source page, click the view that you want.

Additional columns in the Content Source, Host Name and Crawl History views:

Usage reports (search report):

To view usage reports

Verify that the user account that is performing this procedure is an administrator of or has Read permissions to the Search service application.

In Central Administration, under Application Management, click Manage service applications.

On the Service Applications page, click the Search service application.

On the Search Administration page, in the Quick Launch, in the Diagnostics section, click Usage Reports.

On the View Usage Reports page, click the usage or search reports view that you want view.

The SharePoint Server 2016 environment might require configuration of the diagnostic logging settings after initial deployment, after upgrade, and if a change is made to the environment, such as adding or removing a server.

The guidelines in the following list can help you form best practices for the specific environment.

* Change the drive to which the server writes logs:

By default, SharePoint Server 2016 writes diagnostic logs to the same drive and partition on which it was installed. Because diagnostic logging can use a large amount of drive space and compromise drive performance, you should configure SharePoint Server 2016 to write to another drive on which SharePoint Server 2016 is not installed.

You should also consider the connection speed to the drive on which SharePoint Server 2016 writes the logs. If verbose-level logging is configured, the server records a large amount of data. Therefore, a slow connection might result in poor log performance.

* Restrict log disk space usage:

By default, the amount of disk space that diagnostic logging can use is unlimited. Therefore, restrict the disk space that logging uses, especially if you configure logging to write verbose-level events. When the disk reaches the restriction, SharePoint Server 2016 removes the oldest logs before it records new logging data.

* Use the Verbose setting sparingly:

You can configure diagnostic logging to record verbose-level events. This means that SharePoint Server 2016 records every action that it takes. Verbose-level logging can quickly use drive space and affect drive and server performance. You can use verbose-level logging to record more detail when you are making critical changes and then reconfigure logging to record only higher-level events after you make the change.

* Regularly back up logs:

Diagnostic logs contain important data. Therefore, back up the logs regularly to ensure that this data is preserved. When you restrict log drive space usage, or if you keep logs for only a few days, SharePoint Server 2016 automatically deletes log files, starting with the oldest files first, when the threshold is met.

* Enable event log flooding protection:

When you enable this setting, SharePoint Server 2016 detects repeating events in the Windows event log, and suppresses them until conditions return to a typical state.

You can set the level of diagnostic logging for the event log and for the trace log. This limits the types and amount of information that are written to each log.

The following tables define the levels of logging that are available for the event log and trace log.

Configure diagnostic logging by using Central Administration :

In Central Administration, on the home page, click Monitoring.

On the Monitoring page, in the Reporting section, click Configure diagnostic logging.

On the Diagnostic Logging page, in the Event Throttling section, configure event throttling as follows:To configure event throttling for all categories:

Select the All Categories check box.

Select the event log level from the Least critical event to report to the event log list.

Select the trace log level from the Least critical event to report to the trace log list.

To configure event throttling for one or more categories:

Select the check boxes of the categories that you want.

Select the event log level from the Least critical event to report to the event log list.

Select the trace log level from the Least critical event to report to the trace log list.

To configure event throttling for one or more subcategories (you can expand one or more categories and select any subcategory):

Click the plus (+) next to the category to expand the category.

Select the check box of the subcategory.

Select the event log level from the Least critical event to report to the event log list.

Select the trace log level from the Least critical event to report to the trace log list.

To return event throttling for all categories to default settings:

Select the All Categories check box.

Select Reset to default from the Least critical event to report to the event log list.

Select Reset to default from the Least critical event to report to the trace log list.

In the Trace Log section, in the Path box, type the path of the folder to which you want logs to be written.

In the Number of days to store log files box, type the number of days (1-366) that you want logs to be kept. After this time, logs will automatically be deleted.

To restrict the disk space that logs can use, select the Restrict Trace Log disk space usage check box, and then type the number gigabytes (GB) you want to restrict log files to. When logs reach this value, older logs will automatically be deleted.

After you have made the changes that you want on the Diagnostic Logging page, click OK.

Configure diagnostic logging by using Windows PowerShell :

Verify that you have the following memberships:

securityadmin fixed server role on the SQL Server instance.

db_owner fixed database role on all databases that are to be updated.

Administrators group on the server on which you are running the Windows PowerShell cmdlets.

An administrator can use the Add-SPShellAdmin cmdlet to grant permissions to use SharePoint Server 2016 cmdlets.

On the Start menu, click All Programs.

Click SharePoint 2016.

Click SharePoint 2016 Management Shell.

To change the drive to which the server writes logs, at the Windows PowerShell command prompt, type the following command:

Set-SPDiagnosticConfig -LogLocation D:\DiagnosticLogs

To restrict log disk space usage, at the Windows PowerShell command prompt, type the following command:

Set-SPDiagnosticConfig -LogMaxDiskSpaceUsageEnabled

Or assign the maximum disk space for logs:

Set-SPDiagnosticConfig -LogDiskSpaceUsageGB 500

To view the current logging level, at the Windows PowerShell command prompt, type the following command:

Get-SPLogLevel

To change the logging level, at the Windows PowerShell command prompt, type the following command:

Set-SPLogLevel -TraceSeverity Monitorable

To set all categories back to default levels, at the Windows PowerShell command prompt, type the following command, and then press ENTER:

SharePoint Server 2016 provides three types of caches that help improve the speed at which web pages load in the browser: the BLOB cache, the ASP.NET output cache, and the object cache.

The BLOB cache is a disk-based cache that stores binary large object files that are used by web pages to help the pages load quickly in the browser.

The ASP.NET output cache stores the rendered output of a page. It also stores different versions of the cached page, based on the permissions of the users who are requesting the page.

The object cache reduces the traffic between the web server and the SQL database by storing objects such as lists and libraries, site settings, and page layouts in memory on the front-end web server. As a result, the pages that require these items can be rendered quickly, increasing the speed with which pages are delivered to the client browser.

The monitors measure cache hits, cache misses, cache compactions, and cache flushes. The following list describes each of these performance monitors.

A cache hit occurs when the cache receives a request for an object whose data is already stored in the cache. A high number of cache hits indicates good performance and a good end-user experience.

A cache miss occurs when the cache receives a request for an object whose data is not already stored in the cache. A high number of cache misses might indicate poor performance and a slower end-user experience.

Cache compaction (also known as trimming), happens when a cache becomes full and additional requests for non-cached content are received. During compaction, the system identifies a subset of the contents in the cache to remove, and removes them. Typically these contents are not requested as frequently.

Compaction can consume a significant portion of the server’s resources. This can affect both server performance and the end-user experience. Therefore, compaction should be avoided. You can decrease the occurrence of compaction by increasing the size of the cache. Compaction usually happens if the cache size is decreased. Compaction of the object cache does not consume as many resources as the compaction of the BLOB cache.

A cache flush is when the cache is completely emptied. After the cache is flushed, the cache hit to cache miss ratio will be almost zero. Then, as users request content and the cache is filled up, that ratio increases and eventually reaches an optimal level. A consistently high number for this counter might indicate a problem with the farm, such as constantly changing library metadata schemas.

You can monitor the effectiveness of the cache settings to make sure that the end-users are getting the best experience possible. Optimum performance occurs when the ratio of cache hits to cache misses is high and when compactions and flushes only rarely occur. If the monitors do not indicate these conditions, you can improve performance by changing the cache settings.

The following sections provide specific information for monitoring each kind of cache.

Monitoring BLOB cache performance:

Note:
For the BLOB cache, a request is only counted as a cache miss if the user requests a file whose extension is configured to be cached. For example, if the cache is enabled to cache .jpg files only, and the cache gets a request for a .gif file, that request is not counted as a cache miss.

Monitoring ASP.NET output cache performance :

Note:
For the ASP.NET output cache, all pages are cached for a fixed duration that is independent of user actions. Therefore, there are flush-related monitoring events.

Monitoring object cache performance :

The object cache is used to store metadata about sites, libraries, lists, list items, and documents that are used by features such as site navigation and the Content Query Web Part.

This cache helps users when they browse to pages that use these features because the data that they require is stored or retrieved directly from the object cache instead of from the content database.

The object cache is stored in the RAM of each web server in the farm. Each web server maintains its own object cache.

You can monitor the effectiveness of the cache settings by using the performance monitors that are listed in the following table.

You may have noticed that the hardware requirements for SharePoint 2013 Server are quite hefty. Many SharePoint 2013 performance issues have been attributed to lack of resources. Although meeting the minimum performance specs is highly recommended, you can tweak SharePoint 2013 to work with less resources.

The following tips are some that I have collected while looking for ways to improve SharePoint 2013 performance. Some of these will help SharePoint’s performance however, my experience has been that unless you have a server that meets the minimum SharePoint 2013 requirements, the Search function will still bring your SharePoint server to a crawl.

If however, you want to try and tweak SharePoint before completely turning off the search service, be aware that performance results will vary depending on your server’s RAM and CPU speed.

Before You Begin: Stop/Disable the SharePoint 2013 Search Service

If your SharePoint 2013 is running like a 3 legged turtle, temporarily disable the Search Host Controller and the Search Server. This will render your SharePoint Server usable until you complete these tasks. Note: The SharePoint Timer Service will re-start both the Search Host Controller and the Server Search service, so you may want to temporarily disable the Timer Service as well.

The services to disable are SharePoint Search Host Controller and SharePoint Server Search 15. You can find these services by running services.msc from a command prompt. Once you have finished, don’t forget to enable them again.

Update SharePoint with March 2013 Patch Update KB2767999

Updating SharePoint with the March 2013 Patches fixes search-related performance problems. Note that you should disable the SharePoint Timer Service first, then SharePoint Search Host Controller and the SharePoint Server Search before installing the updates. (Detailed instructions here)

How to Reduce the Search Feature Crawl Time Interval and Configure SharePoint 2013 Search

By default, the search feature is set to crawl every 20 minutes. This is nice if you have a monster server, but if you are not so lucky, reducing the crawl rate will yield much better performance. If the SharePoint server hosts a large content database, it may take more than 20 minutes to index. By the time the crawler is finished, it’s time to start again! This leaves you with a perpetually crawling indexing function.

Click on Search Service Application and then on the Content Sources menu link.

Locate your site, click on the drop down menu and then select edit. In the start address, make sure that you have entries for your FQDN, your default Web as well as the SPS3 Protocol Handler. For example: if your SharePoint server’s NetBIOS name is SP01, then you should have an entry for http://SP01 and SPS3://SP01 in addition to your Web site FQDN. This is necessary for the search feature to properly work. If the search index does not properly work, continual searches will damper performance. Once search is working properly, server performance will greatly increase.

Next, set a full and incremental crawl schedule so that the crawl takes place during off-peak usage times. The older your server, the less frequent you may want to make the crawls.

If you cannot see noderunne.exe.config, click on the drop down next to file name and select All Files (.)

Modify the following line:

Noderunner.exe is set to “0” by default, which means unlimited memory usage. Change the “0” to a number in megabytes to limit the total amount of ram that it can use.

For example:

Click save and exit. Restart the SharePoint server.

You can greatly increase SharePoint 2013 server performance by stopping the following services:

· SharePoint Search Host Controller

· SharePoint Search Server 15

Of course, this should be done if you are not using the search service. If you are, this may not be an option.

If that’s the case, your best option may be to increase your server’s available RAM to meet or exceed the minimum requirements. You can also mitigate the performance effects caused by the search service if you reduce the amount of Ram available to noderunner.exe however this is less desirable than stopping the search services.

Reset the Search Index and Initialize a Full Crawl

Finally, re-initialize the search index to clear out the database and re-initialize the crawl. To do this, go to Central administration –> General Application Settings –> Farm Search Administration –> Search Service Application. Select Index Reset from the crawling menu and then click on the reset now button.

Once you have reset the index, click on content sources and select start all crawls.

Here are five storage-related issues in SharePoint that can kill performance, with tips on how to resolve or prevent them.

Problem #1:

Unstructured data takeover. The primary document types stored in SharePoint are PDFs, Microsoft Word and PowerPoint files, and large Excel spreadsheets. These documents are usually well over a megabyte.

SharePoint saves all file contents in SQL Server as unstructured data, otherwise known as Binary Large Objects (BLOBs). Having many BLOBs in SQL Server causes several issues. Not only do they take up lots of storage space, they also use server resources.

Because a BLOB is unstructured data, any time a user accesses a file in SharePoint, the BLOB has to be reassembled before it can be delivered back to the user – taking extra processing power and time.

Solution:

Move BLOBs out of SQL Server and into a secondary storage location – specifically, a higher density storage array that is reasonably fast, like a file share or network attached storage (NAS).

Problem #2:

An avalanche of large media. Organizations today use a variety of large files such as videos, images, and PowerPoint presentations, but storing them in SharePoint can lead to performance issues because SQL Server isn’t optimized to house them.

Media files, especially, cause issues for users because they are so large and need to be retrieved fairly quickly. For example, a video file may have to stream at a certain rate, and applications won’t return control until the file is fully loaded. As more of this type of content is stored in SharePoint, it amplifies the likelihood that users will experience browser timeout, slow Web server performance, and upload and recall failures.

Solution:

For organizations that make SharePoint “the place” for all content large and small, use third-party tools specifically designed to facilitate the externalization of large media storage and organization. This will encourage user adoption and still allow you to maintain the performance that users demand.

Problem #3:

Old and unused files hogging valuable SQL Server storage. As data ages, it usually loses its value and usefulness, so it’s not uncommon for the majority of SharePoint content to go completely unused for long periods of time. In fact, more than 60 to 80 percent of content in SharePoint is either unused or used only sparingly in its lifespan. Many organizations waste space by applying the same storage treatment for this old, unused data as they do for new, active content, quickly degrading both SQL Server and SharePoint performance.

Solution:

Move less active and relevant SharePoint data to less expensive storage, while still keeping it available to end users via SharePoint. In the interface, it helps to move these older files to different parts of the information architecture, to minimize navigational and search clutter. Similarly, we can “unclutter” the storage back end.

A third-party tool that provides tiered storage will enable you to easily move each piece of SharePoint data through its life cycle to various repositories, such as direct attached storage, a file share, or even the cloud. With tiered storage, you can keep your most active and relevant data close at hand, while moving the rest to less expensive and possibly slower storage, based on the particular needs of your data set.

Problem #4:

Lack of scalability. As SharePoint content grows, its supporting hardware can become underpowered if growth rates weren’t accurately forecasted. Organizations unable to invest in new hardware need to find alternatives that enable them to use best practices and keep SharePoint performance optimal. Microsoft guidance suggests limiting content databases to 200GB maximum unless disk subsystems are tuned for high input/output performance. In addition, huge content databases are cumbersome for backup and restore operations.

Solution:

Offload BLOBs to the file system – thus reducing the size of the content database. Again, tiered storage will give you maximum flexibility, so as SharePoint data grows, you can direct it to the proper storage location, either for pure long-term storage or zippy immediate use.

It also lets you spread the storage load across a wider pool of storage devices. This approach keeps SharePoint performance high and preserves your investment in existing hardware by prolonging its useful life in lieu of buying expensive hardware. It’s simpler to invest in optimizing a smaller SQL Server storage core than a full multi-terabyte storage footprint, including archives.

Many organizations haven’t yet explored these externalization capabilities, however, and are missing out on significant storage and related performance benefits. However, native EBS and RBS require frequent T-SQL command-line administration, and lack flexibility.

Solution:

Use a third-party tool that works with Microsoft’s supported APIs, RBS, and EBS, and gives administrators an intuitive interface through SharePoint’s native Central Administration to set the scope, rules and location for data externalization.

In each of these five problem areas, you can see that offloading the SharePoint data to more efficient external storage is clearly the answer. Microsoft’s native options, EBS and RBS, only add to the complexity of managing SharePoint storage, however, so the best option to improve SharePoint performance and reduce costs is to select a third-party tool that integrates cleanly into SharePoint’s Central Administration. This would enable administrators to take advantage of EBS and RBS, choosing the data they want to externalize by setting the scope and rules for externalization and selecting where they want the data to be stored.

Rule Name: Accounts used by application pools or service identities are in the local machine Administrators group.

Summary: A user account that is used by application pools or services must have permissions of a domain user account and must not be a member of the Farm Administrators group or a member of the Administrators group on the local computer. Using highly privileged accounts for application pools or services poses a security risk to the farm, and could allow malicious code to execute.

Cause: Accounts that are used by application pools or services are members of the Administrators group on the local computer.

Resolution: Change the user account to a predefined account, or to a domain user account that is not a member of the Administrators group.

Verify that the user account that is performing this procedure is a member of the Farm Administrators group.

On the Central Administration home page, in the Security section, click Configure service accounts.

On the Service Accounts page, in the Select the component to update list, click the application pool or service that uses the credentials of a member of the Administrators group on the local computer as its security account.

In the Select an account list, click an appropriate account for this component — for example, the predefined account Network Service — or click Register new managed account, and then on the Register Managed Account page, specify the credentials and the password change settings that you want.