Mark's Sitecore Bloghttps://sitecoreblog.marklowe.ch
Sun, 30 Dec 2018 16:46:01 +0000en-UShourly1https://wordpress.org/?v=4.5.16Debugging xConnect with xConnectHelperhttps://sitecoreblog.marklowe.ch/2018/12/debugging-xconnect-with-xconnecthelper/
https://sitecoreblog.marklowe.ch/2018/12/debugging-xconnect-with-xconnecthelper/#respondSun, 30 Dec 2018 15:00:21 +0000http://sitecoreblog.marklowe.ch/?p=517Setting up and testing xConnect can be a challenge – especially in on premise setups. Are all the certificates set up? Are all databases up and connection strings correct?

Initiially, tracking, processing and reporting can be somewhat of a black box because to see it working correctly i.E. to test experience profile functionality, you’ll have to first write code to identify a contact through a form submit, start a session with interactions, trigger identification and wait for your session to expire and wait for processing / reporting in order for your contact to show up in the expirence profile manager.

How does the helper help?

View status information on the current Tracker config and collection service connection

View basic contact facets

Set one or multiple identifiers for the current contact

Set basic contact facets

Flush current session which immediately triggers processing

Some possible use cases:

I want to check the xConnect connection:
If there is a connection problem, this will be immediately shown on the xConnectHelper page. Also, if the tracker is not active becaues of config or license issues.

I want to check if data is written to the collection and reporting db:
Interact with your website and then hit flush current session. Processing of your contact and interactions will immediately start and data should be visible in the databases within seconds.

I want to test contact search / experience profile:
The experience profile manager only lists identified contacts. You can set an identifier for your contact, then flush session. Your contact should show in the list within a few seconds.

Important Note: Because this tool is made to debug current contacts and interactions in-session, it can not be used as a regular Sitecore admin page and is therefore publicly accessible (ony restricted by your access key). You are responsible for setting appropriate measures to regulate access to this page.

]]>https://sitecoreblog.marklowe.ch/2018/12/debugging-xconnect-with-xconnecthelper/feed/0Watch your IDTables when upgradinghttps://sitecoreblog.marklowe.ch/2018/12/watch-your-idtables-when-upgrading-sitecore/
https://sitecoreblog.marklowe.ch/2018/12/watch-your-idtables-when-upgrading-sitecore/#respondSun, 30 Dec 2018 01:49:38 +0000http://sitecoreblog.marklowe.ch/?p=501Recently after a major Sitecore upgrade, we ran into issues with items that have been synced from a DAM through a data provider. These items suddenly had different Guids than before which led to many broken links.

The reason for this took some time to figure out:

This dataprovider was using the IDTable to maintain consistent Guids. While on the old version, the IDTable was stored in the master database, on the new CM server, it was pointing to the web database.

Look out when you are upgrading or switching roles in conjunction with DataProviders! I hope this saves you some time.

]]>https://sitecoreblog.marklowe.ch/2018/12/watch-your-idtables-when-upgrading-sitecore/feed/010 issues you might run into when upgrading to Sitecore 9https://sitecoreblog.marklowe.ch/2018/12/10-issues-you-might-run-into-when-upgrading-to-sitecore-9/
https://sitecoreblog.marklowe.ch/2018/12/10-issues-you-might-run-into-when-upgrading-to-sitecore-9/#commentsMon, 03 Dec 2018 07:00:56 +0000http://sitecoreblog.marklowe.ch/?p=471In this post I’ll list 10 issues that – depending on your particular setup, you might also run into when upgrading to Sitecore 9. They are all easy to fix, so I hope this post saves you some time.

1. Proxy Data providers

Could not find configuration node: proxyDataProviders/main

If you have configured extra databases other than core, master or web i.E. additional publish targets, your old config will need to be updated. Remove the proxyDataProvider section from your database config node. Additionally you’ll need to define PropertyStoreProvider and eventing/eventQueueProvider. See this Stackexchange post for details

2. Compilation errors because of ProxyDisabler()

Anything to do with proxy items has been deprecated and removed in Sitecore 9. Any occurrence of ProxyDisabler() in your code has to be removed.

using (new ProxyDisabler())
{
... this will fail
}

Look out for third party libraries that were not compiled against the Sitecore 9 kernel. You might run into runtime errors if they include ProxyDisabler anywhere. Suggestion is to obviously only use third party libraries that are compatible with Sitecore 9.

3. WFFM Failing Upgrade Post Steps

If you’re upgrading WFFM from before 8.2 you might run into an exception because the upgrade post steps are not compatible with Sitecore 9 API.

To continue the WFFM upgrade, remove Sitecore.Forms.Core.dll and temporarily disable all WFFM configs in /App_Config/Modules to continue with the next update package.

If this happens after you’ve installed the final WFFM update package, check that you have the correct version of Sitecore.Forms.Core.dll inside your bin folder and also check, if the WFFM configuration files are correct in /App_Config/Modules

5. Compatibility issues with new versions of System.IdentityModel.Tokens.Jwt

If you are using the System.IdentityModel.Tokens.Jwt library, you’re most likely using a recent one (5.1+ at the time of writing). Sitecore uses version 4.0.4 for validating the license and there are breaking changes between 4.x and 5.x in that library.

Luckily, both assemblies can be used at the same time by adding them to a different folder than /bin and setting the following in the assemblyBinding section of web.config:

6. SolrSearchIndex. No matching constructor was found on ContentDelivery

In case the Update Installation wizard has created a Sitecore.ContentSearch.Solr.Index.Master.patch.config file during the upgrade process, this file will reference the master index which is disabled for the ContentDelivery role. Review the config changes in the .patch.config and if they are needed, apply them to the Standalone or ContentManagement role only.

7. Query.MaxItems affecting query results

This is a hard one to find: The default limit for items returned by Query.MaxItems is 100. In some previous versions, this had been patched to to 260 through Sitecore.ExperienceExplorer.config. If you happen to upgrade from one of those previous versions and are using queries, check your code if it still works as expected.

In some cases your queries might only return 100 items now instead of 260 and this might result in different behaviour. Thanks to Christoph Brunner for finding this one.

Just the usual reminder around queries: It is usually a bad idea to increase the Query.MaxItems value as this can drastically reduce performance. Always consider carefully if it is a good option to use Queries.

8. SessionProvider issue when Tracker is enabled

On our solution, the UpdateInstallationWizard had created a Sitecore.Analytics.Tracking.patch.config as follows:

Exception: System.NullReferenceException
Message: Object reference not set to an instance of an object.
Source: Sitecore.SessionProvider.Sql
at Sitecore.SessionProvider.Sql.SqlSessionStateProvider.Initialize(String name, NameValueCollection config)
at Sitecore.Configuration.DefaultFactory.GetProviders[TProvider,TCollection](List`1 nodes)
at Sitecore.Configuration.DefaultFactory.GetProviders[TProvider,TCollection](String rootPath, TProvider&amp; defaultProvider)
at Sitecore.Configuration.ProviderHelper`2.ReadProviders()
at Sitecore.Configuration.ProviderHelper`2.get_Provider()
at Sitecore.Analytics.Data.Dictionaries.DictionaryData.Session.SessionDictionaryData..ctor(SharedSessionStateConfigBase configuration)

As we wanted to use the default SQL Session state configuration, the simple solution was to just delete this .patch.config file.

9. Good performance but slower startup

Overall performance of the 9+ versions is very good but the startup time of Content Editor, Experience Editor seems to be a bit longer than with previous versions. Also on our solution, WFFM forms take quite some time when loaded for the first time after an App pool restart. The subsequent loads are quick again.

To provide the best experience for editors and visitors of your sites it is a good idea to implement a warmup of ContentEditor, Experience Editor and your forms on the ContentDelivery instances.

]]>https://sitecoreblog.marklowe.ch/2018/12/10-issues-you-might-run-into-when-upgrading-to-sitecore-9/feed/1Customize Solr Managed Schema in Sitecore 9+https://sitecoreblog.marklowe.ch/2018/10/customize-solr-managed-schema/
https://sitecoreblog.marklowe.ch/2018/10/customize-solr-managed-schema/#commentsSun, 28 Oct 2018 22:07:34 +0000http://sitecoreblog.marklowe.ch/?p=459In Sitecore 9+ we can now populate Solr managed schema through the Sitecore control panel. It is still possible to use the legacy schema.xml files, but it is actually a lot more comfortable to be able to just deploy the schema with one click.

Sitecore currently has some basic schema which it populates and has created the contentSearch.PopulateSolrSchema pipeline as extension point. Mikael Högberg has written a post describing this extension point and I’ve built the following simple processor based on that. It reads Managed Schema update commands as XML and applies them to the indexes you have specified (pipe-separated)

Feel free to use and extend it.

Caveat: Solr only accepts the add-field-type command if the field is not contained in the managed schema yet. The replace-field-type will only work if it is already there. This is a bit annoying and could be worked around by checking if the field exists in the schema in code and then changing add to replace and vice versa.

Also, the schema update will fail silently if there are any errors in the update commands. I’ve found it helpful to first test the update commands as JSON against Solr’s API directly and then convert that JSON to XML.

Update 05.11.2018:
Added a add-or-replace-field-type command which is automatically replaced to “add-field-type” or “replace-field-type” depending if the field type already exists in the schema.

Back to topic. Debugging xDB issues on a remote environment can be very difficult. Is the MongoDB connection OK? Is tracking configured correctly? Are contacts being created and identified? Is my data pushed to the correct places, when my session ends? Why is GeoIP detection not working?

Luckily, there is a nice little helper page named xdbHelper initially created by Ivan Buzyka. I have added some extensions to it to make it even more useful. Here is what i looks like *ugly UI alert!*

Flush your session

The most important feature of xdbHelper is to allow you to terminate your session immediately. This will make Sitecore store your temporary session data in the analytics database and also helps to check whether data is being aggregated in the reporting db tables without wasting time waiting for sessions to expire.

After clicking the “flush session” button, check if the number of interactions in MongoDB has gone up. If not, there is an issue with session state configuration or processing data. Check the Sitecore log for details.

Check if Analytics Tracking is running

The page will display a warning at the top if Tracker is not active. If not this can be due to several issues

Incorrect configuration

No connection to MongoDB

Invalid license

Check the Sitecore log for more detailed information. Usually it states right at the start why Tracker was not able to be initialized.

Check MongoDB Connection

Often it is not immediately evident through logs that there is no connection to MongoDB. xDB Helper will show you if there is no connection and why if failed.

Check GeoIP resolving

Ever so often I’ve run in to issues with GeoIP resolving in production. Mostly related to Loadbalancers not forwarding client IP addresses or special formatting of forwarding headers. xdbHelper will display the received client IP and also the X-Forwarded-For header if present.

If no Country is displayed, geo IP lookup is either disabled or not configured correctly. If your loadbalancer forwards the client IP through a header (i.E. X-Forwarded-For), make sure you have the appropriate header set in

<setting name="Analytics.ForwardedRequestHttpHeader" value="" />

Test identifying contacts

In case you identify contacts on your site (i.E. through form submissions), you can use this page to verify that identification works correctly. It is also possible to manually set identification and name / surname to help searching for your contact in the Experience Profile dialog.

I have found this to be very useful to find the correct datasets inside my analytics database.

How to set up

Compile and deploy the files to your solution to a hard to guess subfolder in your webroot. (not /sitecore/admin)

Access through https://yoursite/somefolder/xdbHelper.aspx?key=*AccessKey*

A word of caution:

Make sure you take appropriate measures to prevent public access to this page. Especially because the internal X-Forwarded-For http header is exposed.

]]>https://sitecoreblog.marklowe.ch/2017/12/santas-little-xdb-helper/feed/0Allow lookup field queries to respect site roothttps://sitecoreblog.marklowe.ch/2017/11/allow-lookup-field-queries-to-respect-site-root/
https://sitecoreblog.marklowe.ch/2017/11/allow-lookup-field-queries-to-respect-site-root/#respondWed, 22 Nov 2017 09:00:47 +0000http://sitecoreblog.marklowe.ch/?p=425This post shows two ways of using relative lookup field queries in a multisite environment.

The issue

Imagine you want to let your users pick sub Pages of the current site in a multilist. You can set something like the following as source in the template builder:

Notice that the site root /sitecore/content/home is hard coded. This will not work if you have multiple site roots in your content folder:

sitecore

content

Site A [Template: Site Root]

Home

Site B [Template: Site Root]

Home

Solution 1: Relative queries

Of course you can just set /sitecore/content as root in the query but that is not very user friendly. To automatically restrict the items to the current site, there is a solution out of the box using relative queries:

query:./ancestor-or-self::*[@@templatename='Site Root']

This works fine but you loose the ability to use the IncludeTemplatesForSelection and IncludeTemplatesForDisplay parameters.

Fortunately, it is possible to hook in to the getLookupSourceItems pipeline and modify the DataSource query at runtime. I wrote a simple processor that replaces a $siteRoot token with the path of the parent site root item. Once included in your solution, you can then rewrite your query to something like

Only downside here is that Droptree fields for some reason bypass the getLookupSourceItems pipeline, so you’ll have to use solution 1 for those fields.

Check out the source code and config here:

]]>https://sitecoreblog.marklowe.ch/2017/11/allow-lookup-field-queries-to-respect-site-root/feed/0Things to look out for when setting up xDBhttps://sitecoreblog.marklowe.ch/2017/11/things-to-look-out-for-when-setting-up-xdb/
https://sitecoreblog.marklowe.ch/2017/11/things-to-look-out-for-when-setting-up-xdb/#respondTue, 21 Nov 2017 07:30:29 +0000http://sitecoreblog.marklowe.ch/?p=375This post highlights some potential pitfalls when setting up xDB on a Sitecore 8.x environment.

1. Alternate DefaultLanguage

When you have set a DefaultLanguage other than “en”, you might run in to the following issue:

No Contacts appear in the Experience Profile app

To fix the this issue, you need to set the Analytics default language (matching your DefaultLanguage) in order for the contacts to be queried correctly from the analytics index:

2. No Timeline is showing in Experience Profile

This might happen after upgrading from a previous version due to missing channel images. There is an error in the Ajax response related to missing media items. (Could not get media stream for image with id)

Solution: Go to a fresh install of Sitecore 8.x (corresponding to your version). Create a Sitecore package containing all sub-items from the following path:

/sitecore/media library/System/Channel Images

Install this package on the instance that is not working and it should now show the timeline again.

]]>https://sitecoreblog.marklowe.ch/2017/11/things-to-look-out-for-when-setting-up-xdb/feed/0Why silent saving with a processing server is a bad ideahttps://sitecoreblog.marklowe.ch/2017/11/why-silent-saving-with-a-processing-server-is-a-bad-idea/
https://sitecoreblog.marklowe.ch/2017/11/why-silent-saving-with-a-processing-server-is-a-bad-idea/#respondMon, 20 Nov 2017 07:30:21 +0000http://sitecoreblog.marklowe.ch/?p=392This post highlights why silent saving, EventDisabler or using BulkUpdateContext can lead to inconsistent data on a multi-CM environment i.E. when using a processing server.

Silent saving

Silent saving or using BulkUpdateContext will prevent firing an item:saving and item:saved event when Items are written through code. This can for example speed up imports where many items are written to the database.

This will update the field and will not trigger any index update nor any other item:saved / item:saving pipeline events.

The first issue here is that the updated field will most likely not be visible on your site or in content editor. This is because your data is already cached in one of Sitecore’s several caching layers. After silent saving, the item won’t be cleared from these layers.

A hacky workaround is to manually clear the items from cache after saving. This might work on a single-CM-environment but as soon as multiple CM instances are in place, there will be trouble.

Cache clearing across multiple instances

Sitecore uses an event queue to communicate item changes with all servers connected to the database. When CM1 changes an item, it is cleared from cache of CM2 through the item:saved:remote event. This is not the case when silent saving! CM2 will never learn of any changes on CM1 and if an author changes the item on CM2, the changes from CM1 will be overwritten.

This is just a simplified example. I saw these issues at a client where the layout field was being updated through a batch job on the processing instance. When authors would update these items it lead to completely corrupted layout fields.

How to get around these issues

In most cases, I would just simply not use silent saving, EventDisabler or BulkUpdateContext when more than one CM instance is present. This will make bulk updates a bit slower but data consistency is usually a higher priority.

In cases where indexing needs to be disabled during bulk updates, try

IndexCustodian.PauseIndexing();
IndexCustodian.ResumeIndexing();

but bear in mind that this will disable indexing on the entire instance.

Conclusion

In a setup where a processing instance or generally where more than one CM instance are present, these issues need to be considered.
If you need to add a processing instance to an existing setup, it is important to analyze the existing code base and make alterations if silent saving, EventDisabler or BulkUpdateContext are used.

]]>https://sitecoreblog.marklowe.ch/2017/11/why-silent-saving-with-a-processing-server-is-a-bad-idea/feed/0Using IP Geolocation and SSL on Azurehttps://sitecoreblog.marklowe.ch/2017/01/using-ip-geolocation-and-ssl-on-azure/
https://sitecoreblog.marklowe.ch/2017/01/using-ip-geolocation-and-ssl-on-azure/#respondWed, 18 Jan 2017 09:00:23 +0000http://sitecoreblog.marklowe.ch/?p=358I want to share some lessons learned from a recent migration of a Sitecore infrastructure to Azure. If you are running a load balanced environment over SSL and relying on IP Geolocation then you need to look out for the following gotchas:

Azure Load Balancer doesn’t support SSL offloading

Its 2017 so by now you better be running your site over SSL. Unfortunately, the Azure Load Balancer doesn’t support SSL offloading. This is a problem as you need to terminate the SSL connection at the load balancer. You are relying on the X-Forwarded-For header to contain the visitor’s IP address. The load balancer cannot inject this header if the connection is encrypted all the way down to IIS.

To get past this problem, Azure offers Application Gateway which has more advanced features such as SSL offloading. Keep in mind that it comes at a higher price than the basic load balancer though, but there is essentially no way around it.

Handling the Application Gateway’s X-Forwarded-For header

The format which Application Gateway writes IPs to the X-Forwarded-For header differs from most other load balancers I’ve encountered so far as it also adds port numbers behind IPs. I.E. 192.168.0.155:1234
To allow Sitecore’s UpdateGeoIpData processor to be able to work with these, the port numbers need to be stripped off. Otherwise you’ll end up with errors in your log like:

Cannot parse a valid IP address from X-Forwarded-For header

Grant Killian has written this post on how to patch the XForwardedFor processor in the createVisit pipeline which handles this.

Enforcing SSL

Neither the basic Load Balancer nor Application Gateway currently offer a way of redirecting requests to https. You will need to take care of this yourself on IIS. A handy way of doing this is through a rewrite rule which listens to the X-Forwarded-Proto header:

]]>https://sitecoreblog.marklowe.ch/2017/01/using-ip-geolocation-and-ssl-on-azure/feed/0Is there a difference between master and web?https://sitecoreblog.marklowe.ch/2016/11/is-there-a-difference-between-master-and-web/
https://sitecoreblog.marklowe.ch/2016/11/is-there-a-difference-between-master-and-web/#respondWed, 23 Nov 2016 15:43:59 +0000http://sitecoreblog.marklowe.ch/?p=346This question has been in the back of my mind for as long as I’ve been working with Sitecore. Is there an internal difference in the way items are stored to and retrieved from these databases? I put some time into investigating:

Versions

The main difference is the way versions are stored. While the master database will contain every version of each item, only the latest version of an item is published to the web database. This is described in more detail here.
An exception to this applies to Sitecore 8.0 and upwards where all versions that are used in a content test will also be published to web. This is done in the newly introduced publishVersion pipeline. More details in this post.

Database schema

The next thing to investigate is the database schema. Are the tables the same? Are the same indexes set?

To find out, I installed several vanilla SC versions and had SQL management studio script the database and all objects for me. The scripts could then be compared using a diff tool reveiling the following:

Version

Differences

7.2 Rev. 151021

‘uniqueidentifier’ column of table ‘ClientData’ is the last column instead of the fist. Apart from that: no differences.

8.1 rev. 160519

No differences at all.

8.2 Rev. 160729

No differences at all.

The result surprised me a little because I did expect some differences around indexes, but both database schemas are actually exactly the same.

What’s with core?

The core database schema on the other hand differs significantly from the other databases as it also contains the asp.net membership provider tables.