Sitecore caching – an overview

Sitecore caching isn’t documented very well and it is quite intransparent how it works; still it is extremely important for the performance of your site. Therefore this article will try and explain the different caches in Sitecore, how they work and when they are cleared.

Tools

To view the different caches, the size and the usage of them, use the cache tool provided with Sitecore. It can be accessed by entering the URL http://yoursite.com/sitecore/admin/cache.aspx. Here you can see all the caches, how much memory can be allocated to it and how much of it is in use. If you are in doubt whether a certain cache is getting cleared, use this tool and refresh it after you cleared it. You will be able to see the delta of the used cache, giving you an indication of just how much has been cleared.

The different caches

There are a lot of different caches in Sitecore. Here I will only describe the most important ones in detail.

The prefetch cache

There is a prefetch cache for each database. The cache contains items of the Sitecore class PrefetchData, which is more or less item data pulled out from the database. The items are pulled out of the database at Sitecore startup. The items which should be loaded at start up are specified in the config files specified in /app_config/prefecth folder. The items which are loaded are a combination of the items specified in the config file for the specific database (for instance master.config) and the common.config file.

The obvious purpose of this cache is of course to trade longer startup times for faster load times at runtime.

The data cache

There is also a data cache for each database. The cache contains items of the Sitecore class ItemInformation, which also is more or less the item data pulled from the database. The cache is populated whenever they are requested. The data is pulled from the prefetch cache, if it present there; else it is pulled out of the database, put into the prefetch cache and into then the data cache.

The purpose of this cache is to minimize the amount of requests to the database. This is extremely important for performance, as requesting items from the database is rather expensive.

You can specify the maximum size of the data cache in the web.config. You can either alter the DefaultDataCacheSize setting:

<settingname="Caching.DefaultDataCacheSize"value="10MB" />

Or you can set it per database in the /databases/database section:

<cacheSizeshint="setting">

<data>20MB</data>

<items>10MB</items>

<paths>500KB</paths>

<standardValues>500KB</standardValues>

</cacheSizes>

Here you can set the size of the data element to whatever you like.

The item cache

As with the data and prefetch cache, there is an item cache for each database. This cache contains objects of the Sitecore class Sitecore.Data.Items.Item, which are the one you use in your code most of the time. The items are built from the ItemInformation objects in the data cache. So if you request an item, which aren’t present in the item cache, it will get it from the data cache and then populate it in the item cache as an Item.

The reason for this cache is to gain performance so the ItemInformation objects don’t need to get parsed into an Item each time they are requested.

I have performance tested the item cache several times, and there are some gains of using it, but it is not anyway near the gain you get from the data cache.

You can set the size of the item cache in the same place as the data cache.

The HTML cache / web cache

The HTML cache - also referred to as the web cache - caches the actual HTML generated from your sublayouts and renderings. There is a cache per site in your solution. You can configure your HTML cache per presentation control. You can either set this generally on the control or where it is assigned. You can specify the HTML cache to vary on different parameters. For instance you can set it up so it varies by querystrings; then there will be stored multiple versions of the HTML depending on which querystring have been specified.

In this image you can see that I have opened the configuration of a given sublayout. I have specified that it is cachable, so it will create an HTML cache for the given page. I have also specified that the cache should vary by data, which means that the HTML should be different if the Sitecore.Context.Item is different. In this way the HTML cache will be different depending on which item you are trying to access. Further I have set up that the cache should vary by querystrings, which will cache different HTML if a querystring is different. This is great when you for instance are using paging with querystrings.

The HTML cache is really important; especially if you have renderings or sublayouts, which are rather slow. An example can be a menu which iterates over a large amount of items. When a rendering like this is HTML cached, it renders in constant time no matter how many items are to be rendered in the menu.

Please note that your site should be able to perform in some way or another without HTML cache, as this is cleared whenever you publish, which will be describe later in this article.

You can configure the allowed size of the HTML cache and whether it should be enabled or not for each of your sites. This is done in the sites section of the web.config. See the following line:

Overview of requests and caches

So what and when are things actually cached? Well look at this graph, which shows how a request is handled.

If the item which is requested has presentations which are cached, the HTML from the HTML cache is returned and the request is ended.

If there isn’t any HTML cached all items needed by the presentation is requested from the item cache.

If the items aren’t in the item cache it is retrieved from the data cache and parsed into an Sitecore.Data.Items.Item and stored in the item cache.

If the item isn’t present in the data cache, it is retrieved from the prefetch cache and the ItemInformation is stored in the data cache.

If the item isn’t in the prefetch cache it is retrieved from the database and stored in the prefetch cache as PrefetchData.

Clearing of caches

Sitecores caches are cleared differently depending on your environment. A clearing of the cache can be brutal to the performance of your site until it is rebuilt.

The different caches are cleared like this:

The HTML cache: The HTML is always cleared on publish. So whenever you publish even a single item the complete cache is cleared. This is due to the fact, that parts of the HTML can be built from multiple items, so the cache has no way of knowing if the item being published has influence on the presentation.

The Item cache: The item cache is updated incrementally in an atomized way, so if an item is updated during a publish it is updated in the item cache as well. If you publish an item which might influence other items, these items are cleared as well. For example if you publish standard values or a template, that clears more than a single item.There is currently a bug in Sitecore 6.1.0.090630, which clears the item cache completely, if you publish a single item. Further the cache is completely cleared by default if you use the staging module.

The data cache: Also the data cache is incrementally updated, when changes take effect after a publish. The cache is completely cleared by default if you use the staging module. It is rebuild incrementally when the items are requested again.

The prefetch cache: Is cleared in the same way as the data cache, but the items specified in the config files are prefetched from the database.

The important part to notice here is that publishing can make quite a big impact on performance; especially if you are running in a staged environment; so you should consider your publishing strategy carefully. My experiences is that you often get the best result by using scheduled publish.

Please rate this article

11 rates / 4,55 avg.

About the author:

Jens Mikkelsen

Jens Mikkelsen is a partner at Inmento Solutions a Sitecore consulting firm. He works as a Sitecore specialist and consulting helping clients architect and build quality Sitecore solutions using the newest modules and tools.

Further he has been deeply envolved in various complex solutions and has built up a strong knowledge of Sitecore architecture and best practices. He has especially focused on and is specialized in debugging and analyzing Sitecore solutions.

Jens is very interested in the technical mechanisms in the new marketing products such as Sitecore DMS and Sitecore ECM.

6 responses to "Sitecore caching – an overview"

If you have defined multiple sites in the sites section of the web.config, you should also add these sites to the definition of the HtmlCacheClearer in the web.config. If you forget this step, the html caches of your additinal sites are not cleared when an item is published.

I think that event handler should read the elements, which should have an attribute indicating whether to clear the cache or not, instead of hard-coding the site names in the handler. It's not hard to implement an extension to the site info object to do this, but Sitecore should really do this itself.

Thanks for providing the above information. After reading your post I have a question that if my Sitecore database is down for 2 or 3 min and If a user requests a page at the same time(when database is down) will Sitecore render the page from cache or is it going to throw an error saying database is down.