UPDATED: Integrating Google Analytics with a CRM

A while back I wrote a post about how you can integrate Google Analytics with a CRM system. That posts referenced the old urchin.js code and is in need of an update. Plus a lot of people have been asking me about different integration options and I thought it would be a good time to revisit the topic.

So here it is, an update. I’m not going to re-do the entire post, just a few key points.

Why Do This?

This entire hack configuration is based on one simple idea: to build a more robust view of customers/prospects/leads/whatever-you-call-them. The more information we know about people that we sell to the more we can adjust our marketing. This technique let’s us combine where the visitor came from (referral, campaign, direct, search engine) and how often they’ve been to the site with CRM data.

How It Works

We’re going to extract data from the GA tracking cookies and add it to a lead generation form. The GA data will be stored in hidden form elements that the visitor can not see. When the visitor submits the form the GA will be connected to the other information that the visitor entered into the form (usually name, contact information, etc.).

Here’s the data that we’re going to pull from the GA cookies:

* Current referral information (i.e. where the visitor came from)
* Custom segment value (if it exists)
* How many times the visitor has been to the site

You’re probably wondering why we’re not extracting the data via the GA API. More on that in a minute.

The Code

Here it is, the updated code:

The new code differs from the original code in three ways:

First, the inclusion of ga.js. No explanation needed there.

The second change is the addition of the _uGC() function. This function was in the original urchin.js, and because I don’t include that file I need to include the code in the script.

The function is used to parse pieces of data and return certain parts. For example, I use to to parse the cookies (which are just a string) and return the value of the campaign cookie (__utmz), the custom segment cookie (_utmv) and the visitor identification cookie (__utma).

Finally, I added some new functionality to include this visitor’s number of visits. This data is pulled from the __utma cookie (which identifies the visitor). The last integer in __utma is actually a visit counter. I think it is interesting to understand behavior based on frequency so I added this nugget of data.

If you want to see the code in action try this:

In my previous post I included a reference section that documents the format of the campaign cookie and the _uGC() function. Check it out if you’re looking for more technical information.

An Alternate Approach

Remember, all the data we’re pulling is in first party cookies. This means that you could extract the info at the server level rather than the browser level (first party cookies are available to server side code).

You don’t need to do this in JavaScript, as I did. You could use ColdFusion, PHP, .NET or some other server side language.

I’m just trying to provide some inspiration here :)

What About the Google Analytics API?

I’m sure there are a lot of people out there wondering about the GA API and why I don’t just use it to pull data out from GA. The reason is the data in GA is all anonymous, aggregated data. There’s no way to reach into GA and say, “tell me where John Doe came from.” That’s why we pull data from the cookie, because we can combine it with a visitor’s info when they supply it.

Sure, it is possible to pull out GA data and place it in a CRM (like traffic from an email campaign, bounce rate from paid search, etc.) but it’s tough to identify individuals in that data.

Is it impossible? No, not from a technical standpoint. You would need to place a unique identifier in GA to link the data.

But we’re getting into a risky area here.

Remember, it is against the GA terms of service to capture data in GA that identifies an individual. This could be an email address or database ID number. Not my rule, Google’s, and they will crack down.

I’d be interested to hear people’s thoughts on this. Feel free to comment!

Thanks a lot for this post! One of my clients had some concerns about implementing this kind of script on their website, but I would love them to do so because it would enable me to link leads ans actual sales, providing an ROI for every marketing initiative.

Their concern is:

As a publicly traded company, they are worried about privacy issues and they have raised some questions about the fact that we would be collecting data about a user without him/her knowing it. Every Analytics tool does it, but as you mentionned in this article, they do not relate keywords or sources to specific individuals. This script goes a step further.

My question is: are large organizations using that level of precision when tracking online forms? If so, is there any privacy issue, or should it be mentionned somewhere that this kind of data can be collected by the website?

Since I would really like to convince them to use this script, any argument “in favor” or examples of large organizations using this kind of method would be very appreciated!

Hi Justin
Thanks for all the great posts.
If I knew that a customer bought a certain product on a certain day, I could use that product ID to link the web analytics data to back end data, often to individual level. So in many cases the capacity is there already to link to PII. I personally do not have strong feelings about companies tracking my behaviour but I acknowledge and abide by the fact that many people are bothered by this. However I find the GA privacy policy vague and contradictory, even without storing down personal details with my analytics tool I often have the capacity to link behaviour to back end demographic data at individual level anyways.
Best wishes
Niamh

Would really be neat to see more writing on CRM and Google Analytics. Maybe a list of some really popular CRM solutions and how the matching is being done. I assume it’s based on logged-in/registered users so it’s first name/last name or a unique key identified to them.

Would also be neat to see a table showing the pros’s and cons of server-side extraction vs. browser level in-terms of accuracy, how complicated, etc.

Finally, would like to see some articles showing how the Google Analytics data is being fed into the CRM for metrics such as engagement, etc.

Justin as always your posts rock man and I am going to totally suck for doing this but I need some help on this one and cant find the expert advices elsewhere.

Google Website Optimizer Technical Question

Situation is this:

Have a site with product pages that dont see enough traffic to each individual product page for a multivariate test (Google website optimizer) without waiting months for results. For example lets say I have 100 product pages each seeing about 200 views a week. I want to setup a simple multivariate test with different checkout buttons and locations. For this example lets use total of 10 combination’s. If I run that multivariate test on even my most popular product page with only 200 views it will take forever to get solid results and to me Google was just never have enough solid data to calculate. Now if i could aggregate all the product pages into a test 100×200 so 20k views a week that is the kind of data i need GWO to see. So my questions is this:

Does the Javascript really care about the Test Page URL or even Conversion Page URL for that matter as long as the scripts are referenced. In other words I want to make 100 product pages look like one test page they will all have the same scripts on top of page and at bottom of page and then the checkout will always have the same conversion script. Has anyone tried something like this or any expert in JavaScript out there know if this will work? Any insight would be much appreciated.

My 2 cents for GA API ! If you put unique identifier , lets say the value of a cookie (or part of it), does it uniquely identify individuals ? Also, fromt a privacy standpoint,with cookie or unique value , isnt it just hard to trace down individuals (unless they did a transaction on site?

does it really uniquely identify human attached to that cookie ??!!

On second thought, a good revenue stream for adwords is their API,as every server call makes money for google. If Google Analytics to go that path, it would be great for Google to make money out of the API server calls from Analytics ! However to make that happen this privacy thing really needs a great clarification i believe.

Thanks for the great blog! Just a comment about the GA TOS. I would say the reason GA has that TOS is that it is against the privacy law (at least here in the Netherlands but I assume in most countries) to exchange data that identifies an individual between two organizations, without the expliciet consent from the end user. As the statistics data from GA is owned by Google, not by the original site, this means this is an exchange of privacy data, hence not allowed, and will never be. So it’s not Google’s wish to have the TOS like that, they really have no choice.

I think to (legally) have real identifiable data (detailed profiles of users for example), the only way to do that is to have software in your own datacenter, not in a SaaS/hosted environment where you don’t own the data.

I would think that if you DO create(from a technological viewpoint) identifiable data in GA, you could get in trouble not only from Google itself (violating the TOS), but also from visitors of your website (if they find out), as you would be tresspassing the privacy law of exchange privacy data with a third party without explicit consent of the enduser (visitor).

I am no legal expert by any means, just my interpretation of how privacy legislation works.

Thanks Martijn for the thoughtful analysis! I’m not sure, but I believe that the TOS for various geographic regions is different and the European TOS is more strict. I could not agree more that adding personal visitor data to GA is a bad idea. Plus, in my opinion, it’s just not actionable. That’s why I like this method a bit better.

I would assume that submitting a form, that has a disclaimer, would constitute “explicit consent”.

The long term issue from my point of view is that the industry (at least the industry I work in: Web Content Management) is working towards a complete customer profile, to build up as much knowledge about the visitor as possible. Only then you can provide the visitor the content/products/information he/she needs, or in the case of CRM, provide sales with the best information about their customers. But with the current IT trend to ‘outsource’ parts of your applications to SaaS solutions like Google Analytics, you cannot legally put identifiable data to these solutions. I am curious how the IT world will solve this the coming years. Asking for expliciet permission is one way to solve it.
But this is all a bit offtopic for this blog entry of course ;)

End Function
Private Function ReadCookie(ByVal strName As String) As String
Dim strCookieValue As String = “”
Dim ga_cookie As HttpCookie = Request.Cookies(strName)
If Not ga_cookie Is Nothing Then
strCookieValue = ga_cookie.Value
End If

Thanks for the great script! I’m testing it at one of my websites and can see the organic/cpc breakdown. However I can’t see any keywords for cpc traffic, just got “-” all over the place; same is for Campaigns. Any ideas how I can improve this?

Hi Justin, I know this is an old post, but it was the only thing I could find right now! As of the other day (6/22/2010) or so, this script (as you’ve detailed here) stopped working on all clients (using ga.js tracking code). Very weird, since we have been using it successfully to pull the info into form submittals for a while (tks!). I mention this here, because I notice that the results of your form here (in the “The Code” section) are now looking like mine (i.e. pretty much blank)

I believe this code is different from the code you put on your GA Analytics book.

I have one qustion about tracking the campaign name. Does it mean that we wont be able to get the campaign name if we have Auto tagging enabled? If we are not able to do so do you know if there is a solution for tracking the campaign name as well. I believe campaign name is an important information.

@kaan – Correct, if you have auto-tagging enabled you won’t be able to get the campaign name. That info is obscured. The only thing we can infer from auto tagging is that the visitor came via Google AdWords.

@Adrian – I’ll try to get an async version out soon.

@Taylor – No, this does not violate the TOS. Google prohibits you from tracking any PII in Google Analytics. We’re just taking the cookie data and sending it to a CRM when a visitor tells us who they are.

@MarekTus – Things have been busy! With the merger, and the new version of GA, there has not been a lot of time to work on an async version of the CRM code. But it’s still on the to-do list. Sry for any delays!

Can you use this method to extract Google Analytics Campaign names? We recently started using the multi-channel funnels feature of the analytics. Ideally, what we would like to do is have a hidden form field that shows all of the traffic sources that have caused the visitor to fill out the lead form. For example, they might have initially visited the website from a campaign called 2012 Expo. They didn’t fill out a form at the time. But a week later, they came back to the website after clicking on another campaign called 2012 Join Expo Magazine Ad. At this point they fill out the inquiry form. With multi-channel funnels, you can see the full history of campaigns / sources that lead up to someone fill out the inquiry form (at least for the past 30 days). Can the hidden field contain both “2012 Expo” and “2012 Join Expo Magazine Ad”? Or can it only extract one campaign / source? If it can only extract one (definitely not ideal), which one does it use? The last campaign that brought them BACK to the website is important but the initial campaign they had clicked on is equally important. I’d love to know how much is actually stored in the cookies and how much can be pulled out. Thanks!

@Russell: Yes, you could include more than one piece of campaign information. However you would need to store that information in a cookie on the visitor’s machine and then send it along with the form submision. The code would need to do the following:

1. When someone visits the site copy the utmz information into a new cookie
2. On every subsequent visit append the utmz information to the existing cookie value
3. When a form is submitted send the information stored in your custom cookie to your CRM

Is it possible to parse the info stored in “Session / Page” level custom variables, just like the one you mentioned in custom segment cookie (_utmv) ? Say I want to know if the visitors viewed certain page/content before posting a form?

I’m looking for an update as well. This would be extremely helpful for me. I’ve tried to implement, but the best that I can do so far is for my form to return “undefined” for all variables. Do I have to set “UA-1-1″ to be the tracking ID of my google analytics account?

[…] the code relies on the old version of the google analytics code (urchin.js update: you can find updated javascript for the new version of the tracking code at Analytics Talk, however I have not yet integrated it with wufoo) and I’m using jquery for […]

[…] Now that the back end of our call tracking application is working, we need to build the logic that controls which phone number to display. Rather than reinventing the wheel and coming up with our own code to detect the referrer and other information about the visitor, let’s use what Google Analytics already knows! The code we use to read the Google Analytics cookie is based on the code written by EpikOne and published in their blog post “Integrating Google Analytics With A CRM“. […]

[…] Now that the back end of our call tracking application is working, we need to build the logic that controls which phone number to display. Rather than reinventing the wheel and coming up with our own code to detect the referrer and other information about the visitor, let’s use what Google Analytics already knows! The code we use to read the Google Analytics cookie is based on the code written by EpikOne and published in their blog post “Integrating Google Analytics With A CRM“. […]

[…] form. A more detailed description of this technique can be found in the original blogpost UPDATED: Integrating Google Analytics with a CRM written by Justin Cutroni. The code below is a shortened version of Cutronu’s […]

[…] source = ‘google’; medium = ‘cpc’;. For further reference, head on over to Justin Cutroni’s blog post for his great post about the integration. Form Assembly do now offer this as a standard feature […]

[…] Fortunately, people smarter than myself have found a way to solve this problem, and this solution for CRM integration by Justin Cutroni has become my gold standard for how to pull information out of Google […]

[…] acquisition source. Now we just need to know how to extract the data. Luckily, Justin Cutroni has previously described how this encoding works, and shared some javascript code to extract the key bits of […]