Clicky

This is the last analytic topic in my extended series on SEM Analytics. In the course of this series, I’ve covered topics ranging from getting setup for measurement, selecting the proper optimization points, measuring the cross-channel effects of PPC and SEO, and a variety of different techniques for measuring and optimizing specific parts of a SEM program. A good many of these analytic techniques are somewhat different than the traditional view of web analytics as a hunt for solutions to particular site or campaign problems.

There’s a good reason for that. Much of the basic optimization in SEM programs – especially PPC programs – happens outside the web analytics solution. Your bid management solution is the natural locus for most of the basic optimization. Web Analytics is often supplementary. This is especially true since web analytics tools often lack the essential data (cost per click, click rate, impressions, position, etc.) necessary to make sense of the actual marketplace. Even where this data is available, it’s not clear that the web analytics solution provides reports and analysis that are particularly useful for really understanding and optimizing this data.

The topic I’m taking up today is similar – though for slightly different reasons. A great deal of effort in the past two years has been focused by serious PPC buyers on the optimization of the Landing Page. It’s become something of a specialty – with whole companies devoted to this particular art. Multivariate testing, in particular, has grown up as a discipline largely driven by the particular needs of Landing Page optimization.

What’s somewhat ironic is that multivariate testing is done largely outside traditional web analytic packages. That seems ridiculous, but if it is ridiculous, it’s not the fault of users – who really haven’t had much choice in the manner. Multivariate testing capabilities have tended to be siloed solutions that integrated their own tracking capabilities. This made it easier for MV vendors to sell their stuff to companies – and it has been hard to argue with since WA companies lacked credible capabilities in this space. Nor was it possible to generate good MV tests with most CMS solutions and then piggyback measurement on top.

This may all be changing, of course, with Omniture’s recent acquisition a very positive sign. But then again, it may not be changing as much as we all hope. It seems to me that the proper place to put MV serving capabilities has always been the CMS – not the WA solution or any 3rd Party solution. Ideally, I’d like to have good MV serving capabilities in the CMS and then be able to integrate my analytics solution easily on top of that to track the testing. This would subsume multivariate testing into the same basic content generation/serving/analytics paradigm that is used for every other part of the web site.

Any other type of solution seems jury-rigged.

But for now, that’s not the world we live in. Most companies Semphonic works with have either turned to a 3rd Party Landing Page optimization vendor deploying custom multivariate testing or have deployed the Google Site Optimizer and are managing multivariate testing there.

In either scenario, you won’t be using your base web measurement reports to track landing page success – you’ll be using your multivariate test suite to both serve content and measure outcomes.

This doesn’t mean, however, that there aren’t some interesting aspects to Entry Page analysis using web analytics.

One fairly interesting type of analysis that is part of understanding the overall relationship between PPC and SEO is to measure the incremental value of controlling the Landing Page. With organic search optimization, the pages that are most relevant (according to the Search Engine) become the de facto landing pages for your sites. These may not, however, be pages that are particularly strong in driving to the core business goals of your site.

There’s a pretty good chance that nearly every page on your site will show up somewhere in the Google rankings for key terms. So, in that sense, nearly every page is a Landing Page for many key terms. But in the real world, only pages ranked in the top 30 or so listings will get any volume. And only listings in the top 10 will usually get enough volume to matter.

When you buy PPC ads, on the other hand, you control the landing page [Can you multivariate test on heavy organic Landing Pages? You can, and it’s certainly an option worth exploring. I can’t make head-nor-tail of the Google rules when it comes to this practice and I have no idea what would constitute abuse – if anyone can explain this I’d welcome the information!].

To really understand the impact of cannibalization, you need to understand the differential in effectiveness between your organic and paid landing pages. If there is a significant differential, then you might want to ENCOURAGE cannibalization. In fact, it’s conceivable that you might be better off REDUCING the organic position of the page. Otherwise, you can’t keep people from going to the wrong place on your site.

This isn’t an issue I’ve heard discussed – and I know of no SEO professional who has ever been asked to reduce a page rank (for an owned site, at least). But the logic is impeccable.

The measurement of differential effectiveness is reasonably straightforward. You can simply measure conversion (or appropriate optimization measure) for PPC sourced visitors to the PPC Entry Page vs. conversion for the Organic Entry Page.

As with so many web analytic techniques, however, there is a danger of self-selection in this analysis. It may be that visitors who use PPC or organic listings have a built-in differential in quality. If so, then your straightforward comparison of conversion will miss the actual significance of controlling the Landing Page. Here’s where piggy-backing this analysis on top of a cannibalization analysis is interesting. Because when you measure the shift in traffic from cannibalization, you can also track whether or not there is any change in measured conversion effectiveness.

One aspect of PPC and SEO integration that shouldn’t be overlooked is simply taking learnings from your PPC optimizations and applying them to popular SEO landing pages for the same keywords. This isn’t a task that demands any high-level of analysis – but it is the kind of thing an analyst should always be prepared to do. Not all of web analytics requires a web analytics tool. If you know that certain messages and link paths work well for PPC-sourced entries, there’s a very good chance they’ll work well for SEO sourced entries as well. And many times, this type of knowledge can be applied to Entry pages without significantly altering their SEO profile.

In a similar vein, studying the differential effectiveness of SEO Landing Pages (Bounce Rate and Conversion Rate for SEO Entry Pages Compared) AND the SEO to non-SEO differential effectiveness of Pages (Click-Thru and Conversion Rate when SEO sourced and when accessed in any other manner) can yield considerable insight into two classes of SEO misoptimization. First, you may find pages that simply don’t perform very well relative to other SEO entry points for the same or similar keywords. When you find these cases, you can try to either figure out design cues (links or content) that you can borrow or you can focus SEO efforts on the better performing page.

When you find pages that perform much worse when accessed via SEO, you need to consider whether the design template is appropriate for a Landing Page and when types of content might be necessary to improve it. Keep in mind that the bare fact of a page performing worse when accessed via SEO isn’t all that interesting. What’s interesting is comparing the differential between Page X accessed via SEO and otherwise and Page Y accessed via SEO and otherwise.

In general, I’ve found that SEO is less carefully measured, less optimized for conversion and less tested than PPC. So the web analyst is quite likely to find numerous places where SEO optimization is misguided. The frequently monomaniacal focus on traffic generated by position in SEO has created an atmosphere where vendors get rewarded for driving traffic whatever its quality. As I’ve mentioned before, this practically guarantees bad traffic or – in the case of Entry Pages - bad site entry placement or bad page performance.

This last point sums up so much of what the web analyst needs to keep in mind when it comes to SEM analytics. The simple fact of source is one of the key behavioral facts the analyst has available. And it turns out that it regularly makes a big difference. But SEM Analytics lives in a peculiar place – both in terms of web analytics and Search Marketing. As a web analyst, you aren’t generally responsible for optimizing your SEM program – either PPC or SEO. And there’s a pretty good chance that the people that are in charge of that optimization aren’t going to welcome outside measurement. That makes life a lot more difficult and challenging than it really ought to be.

I’ve covered many aspects of SEM analytics from a practical perspective. In my final post in the series, I’m going to address this peculiar political position and what organizations can do to make web analytics a more effective part of their Search Engine Marketing Effort.

I’m going to weave in some posts on learnings and discussion resulting from X Change while I finish my SEM Analytics series, and I thought I’d start with one that came out of my SEM Analytics Huddle and so fits right in!

We spent a good chunk of time in the SEM Analytics Huddle talking through some basic issues in getting setup for measurement and structuring PPC campaigns to manage and monitor them effectively.

One thing that struck me in the discussion was that most of the Agency experts agreed that when they took over a program they usually found themselves rebuilding it from scratch. There’s an element of ethnocentrism here, but I think it’s more than that. PPC programs tend to start fairly clean and become increasingly messy as they are managed over time. So unless a PPC firm is really on top of things, a program that’s been running for awhile is often a mess.

In an earlier post, I mentioned that one pretty good way to see if your Agency is managing your campaign well is to review your Ad Groups and look to see if they make sense. If search terms that are fundamentally different in business concept are grouped together, then your buyer isn’t doing a very good job.

But as we talked through this issue at X Change, I was struck by the fact that the process as described was almost 100% subjective. Buyers basically seem to rely on their personal understanding of the business and the search terms to make these judgments. That’s fine as far it goes, but there are some simple analytic techniques that can help test these judgments – and I get the feeling they are rarely used.

Here’s a simple real-world example. Should “stock quote” and “real time stock quote” be grouped together? If you know the trading industry, you’d probably say no. The addition of real time to the phrase is a significant addition – one that constitutes a demand for particular type of service and that is fundamentally different from the search for a quote on a specific stock. But while subjectively I’d make this distinction, I might not be right. And there are other cases where I’d have no particularly strong judgment. Take “stock ticker,” for example. Should “stock ticker” be treated the same as “stock quote” or “real time stock quote” or is it a different beast entirely? How about “free stock quote” being grouped with “stock quote?”

These simple examples are repeated hundreds of times in most programs and each represents a potentially significant empirical question – one that can’t necessarily be resolved by any amount of learned discussion.

In many cases, the correct answer may depend not on abstract considerations of how the words are used in the language but on the real-world products, services and web pages a particular business deploys. “Stock Ticker” might best map to “stock quote” for one company but “real-time stock quote” in another. The PPC buyer doesn’t work in a world where every distinct concept is distinct to the business. What’s more, the buyer frequently has to group words together to achieve any measurable statistical significance in the tail and to provide any level of management. So there is always a cost to splitting words out – and that cost needs to be constantly weighed against the potential benefits.

These practical considerations mean that you can’t realistically just expect to treat every word as a distinct entity. But if you have to group search terms, how do you know your groups are any good?

There is actually a simple way to test your groups. Keep in mind, that what you are striving for in a group of words is homogeneity. The more similar the words, the better the grouping, and the more disparate the words, the more likely the grouping is to show significant differences in internal performance.

So to check the quality of your groupings, you can review the search term performance within the group to look for significant variations. There are two places to do this. First, you can check within your bid management system for variations in click through. These variations are often indicative, but they can also be misleading because the results are shaped by a surrounding marketplace that isn’t necessarily constant.

On the web analytics side, however, you can get a purer read. If visits sourced from particular search terms within a group behave significantly differently, then the group isn’t homogenous. You can track at several levels here – but I don’t generally recommend focusing on distant measures like conversion. Instead, I expect problems in homogeneity to show up almost immediately on the web site.

So what I’d recommend looking at are two simple measures by keyword within a group. The first is % single access pages (SAP). If a search term within a group has significantly different SAP performance, then it probably doesn’t belong in the group.

Secondly, I’d look at 1st click to see which link path is chosen. If the visitors for a specific term tend to jump in a different direction than those from the rest of the group, then the search term probably doesn’t belong.

By measuring the variation in these two measures within an Ad Group, you can quickly test your hypotheses about Search Term homogeneity within an Ad Group. Having this empirical data is especially important for Agency buyers who are taking over new accounts and may not have learned all the subtleties of the business yet.

In our experience, these two simple measures will catch the vast majority of Ad Group mismatches.

If you’ve setup your measurement to include Ad Groups, you can generally automate a significant amount of this in reporting. It should be possible to capture the core statistics (Single Access Pages and Top Next Pages/Links) for both Group and individual terms. With a snapshot like this, you should be able to check your assumptions quickly and adjust your Ad Groups appropriately.

Since truly homogenous Ad Groups are absolutely essential to effective PPC Management and optimization, this is much bigger deal than most people realize.

There are some arguments that web analytics just won’t help solve. There are others where appropriate web measurement can effectively end the discussion. You probably wouldn’t have thought that deciding on the subjective placement of search terms in Ad Groups was one of the latter – but it’s just one of the many surprising ways that web measurement can support better Search Engine Marketing.

Part 11 in a Series on Web Analytics and Search Engine Marketing Programs

June Dershewitz has scheduled a special Web Analytics Wednesday meeting in Napa right before X Change. If you are coming and are in town on Wednesday, stop by and have a drink. Web Analytics DeMystified, Red Door Interactive and, of course, Semphonic are all co-sponsoring. Drinks and snacks are on us – but June swears she’s closing the tab early enough so that everyone will be okay for Thursday morning!

I’ve actually spent nearly the entire day trying to sort out Huddle assignments for the X Change Conference. It is without doubt one of the most headache inducing and complicated tasks I’ve done this year! So I’m rather happy to be returning to something as simple as SEM analytics…

And this next analysis – measuring conversion by ad creative - is conceptually one of the simplest imaginable. But it is an analysis that's ignored by many SEM practitioners to their considerable cost.

In several earlier posts I’ve talked about the critical importance of using the correct optimization point when doing an analysis. If you don’t, then you are generally worse off after optimization than before. This simple fact is particularly appropriate to the analysis of ad copy.

Unlike keywords, which are now routinely optimized to some measure of conversion or value, many SEM programs still optimize ad copy to traffic. The reasons for this are three-fold. First, Google provides an extremely convenient ad rotation system that lets marketers test creative versions easily. However, it only tests against traffic not conversion or value. Second, unlike keywords, ad copy never show up in your Web Analytics solution without some extra coding. So th analysis of ad copy versions doesn’t just fall out of the measurement system it has to be specially set up. Third, I think many SEM Managers incorrectly assume that the job of ad copy is to simply to deliver traffic – so that conversion isn’t appropriate as an optimization measure.

Let’s tackle each of these in turn.

The fact that any easy system exists for optimizing the wrong thing isn’t an argument in its favor. Google optimizes on traffic for two very good reasons: they have no reasonable alternative and doing so optimizes their revenue not yours. So if you decide you need to optimize on something other than clicks you should just turn off Google Ad Rotations and do ad copy testing manually.

The second issue – the fact that WA solutions report on keyword seamlessly but not ad copy is a practical stumbling block. But it is possible to setup your PPC campaigns so that Ad Creative is trackable.

Which brings us to the crux of the problem – the perception that ad copy is responsible for driving traffic and the web site is responsible for converting it. If this were true about ad copy, it would probably be true about keywords as well. But as it happens, it isn’t true about either. Keywords and ad copy need to be optimized for driving “qualified” traffic. And ad copy is every bit as likely to drive variation in qualified traffic as is keyword.

In studies we’ve done that compared the optimization of ad copy by conversion vs. optimization by traffic (as done by Google rotations), we found that optimizing to traffic wasn’t much better than a coin flip for picking the best ad. The optimization problems are most likely to appear in ads that generate only a modest differential in traffic. Over time, Ad Rotations will strongly favor ad copy that is only a few perctage points better in driving traffic. It doesn’t take much difference in qualification level to more than offset such small traffic differences.

In short, there is no good reason to stick with traffic as the optimization measure for ad copy. What’s more, if you do start measuring ad copy by conversions, you should go back and re-visit old ad copy that you discarded in prior optimizations. Look especially for solid traffic drivers that were marginally outperformed by “hero” versions and ended up being dropped or shown only rarely. There’s a pretty good chance that one or more of these old versions will test better than your current “traffic” hero once you factor in conversions!

Part 10 in a Series on Web Analytics and Search Engine Marketing Programs

Those who have known me for a while generally realize that I am not the most organized person in the world. I’d like to plead age, but I’ve pretty much always been a bit ragged around the edges. So it’s no surprise that in putting together the list of huddles I made a mistake here or there. But I have to admit, it was a pretty big DUH moment when I realized I had forgotten to list my own huddle! Oh well. I am, as I’ve said before, planning on doing one on SEM Analytics.

Which brings me back to the subject at hand – SEM Analytics. We’ve moved through a number of stages of analysis – many at quite a high-level. And you may find that much of the most valuable web analytics for Search Engine Marketing has already been covered. That’s especially true if your site has specific conversion points and you’ve already been optimizing conversion or revenue or lifetime value as appropriate. If that’s the case, then your Bid Management System is already handling most of the necessary day-to-day optimization, and web analytics will be primarily useful in understanding channel issues.

If that’s not the case, then you may well be using a Web Analytics package as your primary optimization tool. For the most part, this isn’t particularly challenging – it’s actually more of a reporting issue than an analysis issue.

However, here are few tips that might make this task quite a bit easier.

Most WA packages will let you track all of the following from a Campaign: responses, visitors, return visitors, conversions, eCommerce values. For SEM, these statistics are usually available at the campaign-level (which should correspond to Ad Group) and at the Keyword level. This is pretty good reporting – and if you have specific conversions and eCommerce data it will provide you pretty much everything you need for most the common optimization strategies.

However, if your site doesn’t have specific end-points and you are trying to measure the value of multiple events or of engagement then you’ll probably find the basic campaign numbers to be insufficient. You don’t typically get good measures of engagement for a campaign.

There are two strategies for handling this. The most obvious is to build a visitor segment based on the campaign. If you do this, you’ll have all the behavior for that segment and the engagement numbers (and the total value of events triggered) can usually be calculated quite easily. The drawback, of course, is that this requires a segment for every campaign. Not only is this a lot of work (and work that must be constantly maintained as new campaigns roll-out) but not all tools provide unlimited segmentation.

A better strategy in most cases is to build a segment for each level of engagement your interested in. By running the campaign report against all visitors you get the total of responses. Divide that by the number of campaign responders you get when you apply the engagement segment and you’ve got the percent of campaign responders that “engaged” at any level. So one segment can be used to calculate engagement for EVERY campaign. This technique preserves segments and saves lots of work – since the engagement segments need only be setup once and new campaigns will automatically be included in the campaign report.

For situations where you want to accumulate values for lots of actions (especially page views) and you want to dynamically optimize for them, this segment approach won’t work. Where every page view should record a certain value, it’s pretty much impossible to build an Engagement Segment that will measure total value. Every land has some value – and there is no single “cut-off” point that will measure engagement value.

In this situation, creating segments by campaign would work – but it still has the drawbacks mentioned above (lots of segments, lots of work), and are some alternative approaches. First, some tools (like WebTrends’ latest version) will let you build a score for a visitor. That score can capture discrete events like Page Views and can then be used to report average scores by campaign. That’s a beautiful approach.

In Omniture, you can use events and record a specific monetary value for every event. This works extremely well for sites that have a multiplicity of events they want to track – because you can track campaigns by the total and average value of all the events triggered. It’s a little clumsy for tracking page views, however, since it requires that you pass an event with every single view.

For other tools, you might choose to handle this situation with either custom variables or by passing eCommerce events. The former approach will work pretty well in some tools but not in others. The key is the degree of flexibility you have in reporting around a custom variable. The second approach will work for most tools, but has the same inherent clumsiness that I mentioned about Omniture events (in fact, it’s usually quite a bit worse) and, if you have real eCommerce events mixed in with your site will generally create too much confusion to be viable.

If you don’t mind the extra expense and some custom tag coding, you can use an extra report suite for doing this work. In the report suite, you send the event type (such as page view) and the campaign name concatenated together as the fully qualified page name. This approach is, I admit, quite weird. But it will give you straightforward reporting of the total value of engagement from each campaign! If you’re a media site and you can’t get reasonable reporting of engagement by campaign, a little weirdness may be worth enduring to get the answers you need.

Here’s one final tip about basic SEM optimization in web analytics tools. Regardless of whether you are using score based systems or levels of Engagement, it is often quite useful to report on multiple levels of engagement and – if scoring - on the distribution of engagement scores by visitors as well as the total and the average.

By multiple levels of engagement, I mean segments that are designed to capture different aspects of engagement or different tiers. At the simplest level, if you are measuring engagement by Page Views, you might have a segment of 2+ Page Views and a segment of 7+ Page Views. If you measure campaigns by the % of visits that are “engaged” at each level, you’ll usually see a fairly even relationship between the two. Most campaigns that score higher by the first measure will score higher by the second. And the rate of drop-off will be fairly consistent. But you’ll also find instances where specific campaigns don’t follow the site-typical pattern.

These campaigns almost always require special study. In many cases, they deviate from the expected pattern because of particular navigational issues on the site. If that looks to be the case, you’ll have to find the optimization rule you think is most applicable for that particular campaign. If you can’t find a navigational explanation, then the findings may indicate peculiarities in the distribution of visitors who are sourced from the campaign. In this case, it’s worth looking at the individual keywords that make up the campaign and also the common navigational paths to see if you can identify a sub-segment that is driving the unexpected performance.

[X Change is quite a limited venue – and with a lot of people signing up last week we are getting close to our limit – which is great! But there is space remaining for another ten to twelve people - so if you are a procrastinator, you can still register at http://www.semphonic.com/conf].

Part 9 in a Series on Web Analytics and Search Engine Marketing Programs

[September is here! That means X Change is getting close. The time to Register is now - and we have even more great speakers. Don't miss this chance to work in small groups with our new Expert facilitators - Rand Schulman, Clint Ivy, Anil Batra and Judah Phillips! They join amazing group including Eric Peterson, Matt Belkin, Olivier Silvestre, Aaron Gray, Marshall Sponder, Manoj Jasra and many more. Details and registration on the Semphonic web site.]

In an earlier post, I described one of the most useful tricks in an analyst’s kit-box – assigning time to various events on the web. I’m going to re-copy that discussion here because it’s the essential infrastructure to what I’m going to talk about in this post.

“… If you can create campaigns in your tool, you can setup a campaign for a specific time period. You can then use Response to that campaign as the criteria for a VISITOR segment that includes significant chunks of time before and after the Response. Now, you can compare the types of Ad Groups, Search Terms and Sourcing mechanisms before a particular campaign response (or conversion) AND after.

If you can’t create ad hoc campaigns in your tool, we suggest tagging key events (like campaign sourcing) with a date in a custom variable. Adding a date in a custom variable allows you to use that date or a date range to build a visitor segment identical to that discussed above.”

By using date-based campaigns or adding the date of source to a custom variable, you can create visitor segments based on WHEN a particular group of visitors were sourced. Suppose, for example, you create a segment of visitors whose first visit was in January 2007 and whose first source was PPC.

Using this segment, you can track January PPC visitors over time. There are quite a few interesting aspects of this over time tracking. The first, and most basic, is to understand the mere fact of return behavior. To show return behavior, you can trend the number of return visits by month – both for the whole campaign, for each engine and for ad groups (and even key search terms).

Once you have this basic data, the next step is to repeat the trend using conversions and/or conversion value.

Why is this important? For a PPC Manager, understanding the amount of return behavior helps define the window against which you have to track. The trend for conversions and conversion value is essential for understanding how to model the lifetime value of PPC sourced visitors. And this information can help you define the sales cycle – the typical durations between first acquisition and first sale.

Of course, all of this data needs to be carefully vetted in light of the known issues with cookie deletion. One of the most basic impacts in cookie deletion is the loss of tracking on groups over time. So if you measure a 10% return rate in February, there is an excellent chance that you are significantly undercounting the true rate. Worse, this effect is not cumulative (the fact that 30% of your visitors delete their cookies each month doesn’t mean that in 3-4 months all of your visitors will have deleted their cookies). Instead, it is often the same 30% of visitors deleting their cookies. So to adjust month out numbers, you can’t assume a flat rate of cookie deletion or a pure extrapolation.

Depending on your web site, you may be able to ascertain the true cookie deletion curve (at least for your customer population) by measuring the percentage of a specific set of customer logins or ids is associated with a “new” WA cookie over a period of time. With this curve, you can adjust the raw return visit counts given in the analysis above.

Using this curve for everything carries with it some dangers (if you think your customer base has significantly different cookie deletion patterns than your prospect or SEM pool) – but at least it’s a start.

These trends for return visits, conversions and value are all useful in and of themselves. They provide the SEM Manager a deep understanding of the overall impact of sourcing a visitor (and how it varies by Campaign and Keyword). I don’t think anyone would doubt that this is good to know. This knowledge can also shed light on how much information is getting lost when 3rd Party time-limited cookie based tracking is being used (as is usually the case with Bid Management tools).

For many sites, the biggest impact of this analysis will likely be around the assessment of lifetime value (and the cross-channel studies discussed in previous posts). Measuring lifetime value can raise serious optimization issues; because if lifetime value turns out to be significantly different for “converters” from different campaigns, then you can't use initial conversion as a good optimization metric. You also can’t optimize a PPC campaign based on waiting six months to see how lifetime value plays out. This problem is especially telling for ad-based sites who need to predict lifetime value based on projected return visits and page consumption. You need to optimize campaigns immediately. And while an analysis like this could spur you to make one-time immediate adjustments to a campaign – how do you go about the business of day-to-day optimization?

Fortunately, this is one place where you probably don’t have to worry so much about cookie deletion. Unless there is some reason why converters (or page consumers) sourced on one campaign are more likely to delete their cookies than visitors sourced on another, then the lifetime value numbers derived from our analysis of behavior over time are comparable (and therefore useful).

To tackle this long-term optimization issue, you need to carve out yet another time-period segment. For this analysis, you’ll want to segment based on a narrow time window of sourced visitors (often a single day). Now, you need to search for “lifetime-value” proxies in either the first visit or a short window after initial sourcing. The idea here is to look for patterns of behavior (conversions, pages, visits, etc.) that correlate with the long-term life-time value classifications.

Sometimes, you’ll get lucky and these will be relatively obvious. If so, it means that you can optimize your PPC programs based on the lifetime-value proxies you’ve discovered. This makes it possible for your PPC buys to be optimized against very short-run data – a big advantage. Sometimes, of course, you won’t. There just aren’t always good short-run behaviors that are predictive of lifetime value – and when they don’t exist you’ll have to use a combination of short-run and study-based optimizations when doing PPC buys.

Whenever you use proxies (be it for conversion or lifetime value) it’s also a good idea to check yourself with a deep-dive study every so often (perhaps twice a year) to make sure that the patterns you’re using still hold. Organizations tend to take the results of these studies and institutionalize them – making them into a kind of ritualized knowledge that nobody questions. But no matter how accurate these proxies may be for a given snapshot, they can age and become increasingly less useful as the business environment changes.

Tracking SEM visitors over time is a powerful and under-utilized method for SEM Analytics. It can provide PPC Buyers with a better understanding of channel interactions; a good sense of the true length of the sales-cycle; better perspective on the degree of error inherent in the PPC tracking from Bid Management or the Search Engines; and a much better measure of the lifetime value (and cost) of visitors sourced by PPC.

In the next post, I’m going to start looking at some types of measurement that are focused on true SEM optimization.

Part 8 in a Series on Web Analytics and Search Engine Marketing Programs

The last post in this series covered several different types of SEM channel analysis: channel cannibalization (the extent to which one online channel borrows or supplements traffic in another channel) and studying a specific channel as a process (the manner in which visitors use Search over-time). Each of these can fundamentally re-shape your understanding of the type of visitors being sourced by Search, how effective your Search Program is and which pieces of your Search Program are most important and cost-effective. This, however, hardly exhausts the subject of channel analysis.

In one way or another, most channel level analysis is ultimately about the allocation of resources. For every company, both time and money are fixed assets that need to be allocated as efficiently impossible to drive long and short-term returns. This means that one of the key problems that every company faces is how to make more intelligent decisions about what to do and how much to do it.

In the real world, it’s often hard to fathom how large companies actually make resource allocation decisions. Particularly in the realm of Search Engine Marketing, it can be baffling to understand why, for example, PPC programs are sometimes aggressively funded while SEO programs languish. The processes that drive such decisions are a compound of political and cultural decision-making factors – where likely optimum points are more often missed than not.

Now good analysis can’t necessarily pinpoint optimum resource allocations. It isn’t possible to know with any certainty what an additional allocation of time and money will accomplish in a channel. Programs do not, under almost any real-world conditions, scale perfectly. In truth, resource allocation decisions should come attached with that rider you always see on Mutual Fund performance claims: "Past Performance is Not Necessarily a Guide to Future Returns!"

However, there is a method by which the analyst can help decision-makers identify programs that are at or beyond their ideal scale. The idea is a simple one – if you track a good Conversion Proxy or Engagement Measure by Channel over a significant period of time, you can measure the point at which the percentage of unqualified visitors begins to rise sharply. When you hit this point, there’s a pretty good chance you’ve begun to max out the opportunity in a channel.

This is a very simple analysis – it’s simply a trend-line of the percentage of unqualified visitors by channel over an extended period of time (ideally since program inception). When you build this trend line, you’re tracking – in the most common case – two countervailing tendencies. Programs tend to get somewhat more efficient over time. Landing Pages are improved. Navigation paths get better. Ads are tuned. All of this tends to improve engagement and results. On the other hand, if you’re expanding within the channel then your program is growing larger. And scale almost always is accompanied by some diminution of quality.

It’s easier to focus on the percentage of unqualified visitors because it is much harder to change the behavior of unqualified visitors than qualified visitors. By looking at unqualified not average qualification or percent qualified, you’re screening off some of the effects of optimization (such as improved Landing Pages).

This isn’t the sort of analysis that should be followed with a hair-trigger. Declines and variations will happen in the course of regular business. It’s important to establish that you’ve got a consistent drop in quality before thinking about the consequences.

When this does happen, I think two courses of action are appropriate. The drop in performance may be indicative of a SEM program either losing freshness or a change in the competitive environment. Either way, a careful survey of competitive programs, competitive advertising, the keywords being used and the creative approaches may all be in order.

If the results of this freshening aren’t particularly positive, I’d be inclined to think that a program may have reached the limits of its current scale. If so, it would mean either exploring alternative scaling mechanisms (like video or local search for example) within a channel or simply shifting new resources into different channels.

A basic analysis of the percent of qualified visitors can also help a company just beginning to explore new channels. If a channel is going to prove fruitful in the long run, then there’s a good chance that your initial cherry-picking efforts will yield a good percentage of highly-qualified traffic. Comparing qualification rates for new channels can help you make a decision about which channel might be more promising for your company to explore first.

The next analysis I’m going to talk about is really and truly a channel analysis. It uses the power of measurement on the web to help optimize mass media offerings. Once again, the idea is fairly simple though its execution can be tricky.

When you run mass media campaigns, this will almost always have a strong impact on your web traffic. Everybody knows this. Effective mass media will impact your direct traffic and your Search Traffic. Indeed, Search is so ubiquitous as a finding method that most online marketers have given up trying to track with vanity pages. Customers simply don’t use them.

But although every web marketer whose company does mass media is keenly aware of the traffic effect, very few have ever taken advantage of the superb measurability of their medium to evaluate the effectiveness of mass media offerings.

The key to doing this (as in most analytic cases) is establishing a control. There are some great ways to do this. I think the best is to use geographic segmentation of your web traffic. Suppose you have two TV commercials for your product. You can run them in one of the several common testing media markets in the country. Before you do this, setup rigid baselines on your web site tracking all of the following for the test DMAs, some control DMAs and the entire site:

Traffic

Traffic Variation

Traffic by Source

Traffic Variation by Source

Qualification Level

Qualification Level Variation

Qualification Level by Source

Qualification Level by Source Variation

All of the above by key visitor segments

It’s vitally important to understand the inherent level of variability in these measures. If you don’t, then you may fool yourself into thinking the perceived changes are actually significant.

Once you have this baseline, however, you have a powerful tool for measuring the differential effectiveness of TV (or any other mass media) campaign. Simply track the baseline periods for the control groups vs. the target media markets. You can measure comparable traffic impact, comparable qualification impact and comparable impact by visitor type.

This information would be a great way to use the web to help optimize significant mass media expenditures. And while this tracking may not replace many traditional forms of mass media analysis (such as survey or focus group work) – it provides a number of benefits. After the initial setup, it is very inexpensive and scales well. It is quick – providing something close to real-time tracking of campaigns. It measures real behavior change – not just "memory impact." All of these are significant.

With the amount of money that some organizations spend on mass media campaigns, the web site might be more important as a cross-channel optimization tool than as a conversion tool in its own right!

Web Analytics is about so much more than the tactical improvement of the web site. These channel-focused analytics help decision-makers make large-scale decisions about the direction of the business. From deciding on which new marketing channels to exploit, to knowing when a particular channel has scaled sufficiently to measuring the impact of major expenditures in mass media, these analyses transcend much of what we think of as the traditional domain of web analytics.

I used to advise beginning web analysts to "know their website." And this is vitally important. Now, I’d be more inclined to say "know your business." That includes the web site, of course. But only a good knowledge of all the aspects of the business can insure that an analyst is focusing on the truly important problems and is able to use web measurement in unorthodox ways to make a difference!

Part 7 in a Series on Web Analytics and Search Engine Marketing Programs

My last post - some thoughts about Google Analytics inspired by Eric Peterson's thorough demolition of Brandt Dainow's post about GA - garnered me nice comments from both Eric and Google's Analytics Evangelist Avinash Kaushik. Which, frankly, seems wrong. At least ONE of those two really should have hated it.

I've been posting quite a bit recently (just seems like there's a lot to talk about) so if you are used to my "Sunday" cycle make sure you check out the recent posts. I actually have a bit of a back-log of topics to cover - very unusual! For now, I'm returning to my series on SEM Analytics - which is a lot less controversial than some recent topics but is, I hope, correspondingly more useful!

The first six parts of this series dealt largely with setting up and understanding the framework for doing Search Engine Marketing analytics. In this and upcoming posts, I’ll tackle some real-world SEM analytic studies. A previous post already covered analyzing overall SEM traffic (since that’s important for understanding the role Search plays on a site). I'm going to take something of a top-down approach; starting with some of the key studies for understanding Search as a channel. Next, I’ll cover time-based studies – looking at the longer term impact of Search sourcing. Following that, I’ll cover optimization analysis; these studies focus primarily on measuring the quality and nature of SEM visitor interactions with a site. After that I’ll cover some types of analysis that can help support SEO engagements. I’ll try to finish up with a nice summary of SEM Analytics – and maybe some further thoughts on the role of web analytics in Search Engine Marketing.

At Semphonic, we first started thinking seriously about Search as channel quite awhile back when one of our clients asked us to measure the impact of a "dark" period in their PPC program. For various reasons, their PPC program was shut-down for an otherwise typical two-week period. It was frustrating, but as it turned out, some good came of it. Because this two-week window gave us a very nice, unplanned view of what happened with traffic and conversions when the PPC program wasn’t running. Pay-Per-Click advertising has become such a significant driver of traffic for many sites that marketing people are afraid to stop doing it. That makes it difficult to measure its impact unless you assume that all of the traffic you get from Search is unrelated to other sourcing mechanisms.

When we looked at the traffic and conversions for the site in question, we found that their volumes had indeed dipped during the dark period. However, the drop was only about half what we would have expected. We checked direct sourcing and other campaigns to see if there was other activity compensating for the loss of Pay-Per-Click. What we found was that Organic Search Traffic had gone up; and it was up significantly – in some cases by more than 50%.

When the PPC program came back online, the Organic traffic went right back down to historical levels.

This was our first experience with organic cannibalization. And it opened our eyes to the extent to which PPC programs CAN interact with other channels. It also led us to question the conventional wisdom that PPC campaigns SUPPORT organic traffic. What we’ve found in the intervening years is this: adding a PPC campaign will always drive at least some INCREMENTAL volume. We’ve never measured a case where this wasn’t true. This fact is the basis from which many of the Search Engines claim that PPC traffic supports organic traffic. However, the actual effect of a PPC program on an organic program varies widely. We’ve measured everything from significant cannibalization of organic clicks to virtually no interaction to actual, real support for organic traffic.

The most common case we’ve measured seems to be mild to moderate cannibalization. Meaning that in most, but not all cases, you might expect your organic traffic on shared-terms to decline modestly when you add a PPC program.

What variables affect this outcome? We’re not really sure. Some that definitely matter are the positions of your organic terms (if you don’t have high-ranking then you should expect a very low level of interaction – positive or negative); degree to which you are buying Brand Terms; clarity of your organic listings, extent to which your organic and paid listings are "coordinated" (I got this from Bill Hunt at Searchnomics), and the strength of your brand. Piling all of these factors (and probably ten other ones) together makes it pretty much impossible to know how you’re going to come out – which is why it’s important to measure.

How do you measure organic cannibalization? It’s pretty trivial. The easiest way is to have a fairly extended dark period. Ideally, the dark period should be somewhat longer than the sweet spot for prospect conversion. You need to make sure you aren’t changing other campaigns during the dark period and you should also pick an historically flat time of year. You don’t want to pick March if you’re a tax preparation firm.

Measure organic traffic and conversion by engine for a period similar to the dark period. Then repeat the exercise during the dark period. It’s a good idea to also measure direct traffic and other online campaigns. Paid Search doesn’t just interact with organic – it can effect direct traffic and even banner traffic. You’ll also want to look at changes in response rate for these channels where possible. This will guard against hidden changes in total impressions that can bias the results.

By comparing the rate of Organic and other source traffic before and during the dark period, you can get a solid measure of potential cannibalization/support. If organic traffic increases significantly during the dark period, you’re likely cannibalizing organic traffic with paid. If organic traffic declines, you’re supporting organic traffic with paid. Understanding which is your real situation can make a pretty big difference in evaluating the effectiveness of your Paid program and in making allocation buying decisions within your Paid Keyword portfolio. Be sure to track conversion rates for channels as well. Some studies suggest that combining programs can boost conversion rates.

You don’t have to have a full dark period to test for cannibalization / support. You can do the study on a single PPC Campaign, Ad Group or significant Keyword. Just make sure you are setup to track at the appropriate level.

Of course, interactions between channels aren’t limited to PPC and Organic sourcing. I hinted at that above – any online channel may be either cannibalizing or supporting (or even some of both) others.

Of great interest in the last year or so has been the interaction within a single channel. Specifically, how users use Search as a process. A number of studies have documented a general tendency for searches to shift from sourcing on non-brand terms to brand terms as they move through the sales cycle. This can have a dramatic impact on your understanding of Ad Group and Search Effectiveness if you are using visit-based or "last-source" based campaign tracking.

Tracking search as a process is a non-trivial exercise and often requires some special setup. If you’re not concerned about the order of events, it is fairly easy in most tools to create a visitor segment based on one or more Search sourcing events. Apply this segment and you can see the number of times visitors sourced from other channels as well as Search.

This same segment will allow you to calculate the number of Search Sourced visits per visitor. These two metrics (non-search visits per Search Source visitor and search sourced visits per Search Source visitor) will give you a much better understanding of search as a process.

But to really understand what’s happening, you need to attach a time component to either your conversion or response measurement. There are many different ways to do this, some tool dependent. If you can create campaigns in your tool, you can setup a campaign for a specific time period. You can then use Response to that campaign as the criteria for a VISITOR segment that includes significant chunks of time before and after the Response. Now, you can compare the types of Ad Groups, Search Terms and Sourcing mechanisms before a particular campaign response (or conversion) AND after.

If you can’t create ad hoc campaigns in your tool, we suggest tagging key events (like campaign sourcing) with a date in a custom variable. Adding a date in a custom variable allows you to use that date or a date range to build a visitor segment identical to that discussed above. This is a very powerful, general tagging strategy that we almost always recommend to our clients.

This is an important technique and, from past experience, I know it isn't always clear to people. So here's a step-by-step recap of what you do:

Create a custom variable that contains the date YYYYDDMM when a conversion occurs (for example).

Now create a VISITOR-based segment where the criteria is that they have a value for that Custom Metric between two specific dates (say April 1-7th).

The timeframe for the segment should include a much wider range of data (say Feb. 1st to May 1st).

Because of the custom variable criteria, you know that all the visitors converted in a specific week (1st week of April). By applying the segment and studying other time periods, you can see how they sourced in the week of conversion, the week before conversion, the month before conversion the month after conversion, etc.

This technique can be extended by combining criteria (PPC sourced during period and converted during period) and using different dates.

This simple trick lets you generate all of the following: # of visits and # of visitors who sourced multiple times from PPC prior to a conversion or response; # of visits and clicks by Keyword and Ad Group from PPC prior to conversion or response; # of non-PPC source visits prior to a conversion or response; and, of course, all of these metrics for what happens after a conversion or response as well.

There is a boat-load of stuff you can do this with data. First, you can figure out whether your conversion optimization needs to be web analytic based not SEM tool-based. SEM tools will almost always use last source as the conversion optimization strategy. That may be fine but it may also be badly flawed. By looking at pre-conversion sourcing, you can also gauge the extent to which a channel is an ORIGINAL source. Let’s face it, it’s harder to get someone to your site the first time – and they are much less likely to use an alternative source. But subsequent visits have, at least, a better shot of being sourced via other channels. So understanding which channels drive the most original sourcing is vital for optimizing your channel balance.

These numbers can also be used to help understand what happens after a conversion or response. PPC Buyers tend, erroneously, to assume that everyone they source is a 1st Time Prospect. You may find, instead, that most of your PPC sourcing is existing customers looking for support or job-seekers looking for careers.

You might also find, for instance, that PPC prospects, once they convert, always visit your site via PPC. Depending on the circumstance, that may be a minor nuisance or a significant cost factor. If you have tight margins (as many publishers who arbitrage PPC do), then a visitor who always sources from Search is much less likely to be profitable than a visitor who only sources from Search initially.

If you’ve coded time as a variable and you have one of the more powerful current generation analytics tools, you can map out the "search process" with a fairly high degree of accuracy. But even if you’re limited to a basic "before" and "after" segmentation, you can get a much improved picture of the degree and manner in which a channel is used repeatedly by single visitors.

In Part 8, I’ll cover two more types of channel analysis – studying the over-time performance of channels to get clues about the potential opportunity and measuring mass media advertising using web sourcing and usage.

Part 6 in a Series on Web Analytics and Search Engine Marketing Programs

My co-founder Joel Hadary loves to quote "If you don't care where you are going any road will get you there." And it does pretty much sum up why finding and choosing the right optimization goal is the single most important part of doing any Search Analytics. In Part 5 of this series, I went over five different conversion optimization strategies for SEM when you HAVE traditional eCommerce like conversions. And that's the simple case. Because when you don't have easy, mappable conversions (or you don't have enough of them to measure well) things are quite a bit more complex.

It's a mistake to think that the world cleaves into two simple buckets: eCommerce and non-eCommerce sites. The different optimization strategies for eCommerce sites reflect basic differences in type of site (by sales cycle, product mix, margin mix and long-term customer value). For sites that aren't doing online sales, the mix is larger and has an even more dramatic effect on optimization strategy.

Because of this variation, I'm only going to work my way through some common cases and hope that the basic ideas for choosing optimization points come through.

Let's start with a "transitional" case - sites that have conversion but where the numbers are too low to optimize all or most of a SEM programs' variables. In this situation, you're ultimate goal is still an endpoint conversion. And what you're looking for on the site are actions that we call "conversion proxies." The idea behind a conversion proxy is simple. You correlate various actions on the site (downloads, video views, interactions, specific page views, number of visits, time on site, etc.) with conversion. You then use those highly-correlated points as proxies for conversion when optimizing. Keep in mind, you're looking for upstream actions that have significantly higher levels of occurrence than conversions. Otherwise, you aren't solving your problem.

With a conversion proxy, you're less concerned with navigational effects and self-selection than you might be in most correlation analysis. For instance, suppose you have an order process that takes 4 steps and has an 80% drop-out rate. It's quite likely that the first or second step of the order process could function as a conversion proxy. While it's obviously meaningless to say that people who reach step 1 are more likely to order than people who don't, for a proxy that isn't a big issue.

That doesn't mean that choosing a conversion proxy is without risk. You need to make sure your proxy is independent of the SEM programs you're analyzing. For instance, suppose you found that Page X on your site was correlated with conversion. You choose it as a conversion proxy. But, if some of your PPC programs land on that page or on another page that is immediately linked, these programs are going to "win" versus everyone else because they are navigationally close to the proxy.

This problem can even occur with examples like the first one - where the proxy is the first step of the order process. If one landing pages is nothing but a single giant drive to the order process and another drives to different places, then the first is going to perform better when page 1 of the order process is the chosen proxy. There is no 100% guaranteed way to screen off these effects. It's your job to think about the possible biases your proxy might introduce. So when you build a proxy, look very carefully at the first batch of results. If you see some SEM programs with extraordinarily high rates of "conversion" then there's a pretty good chance your proxy is biased.

Lead Generation sites form another, completely distinct tier. In a sense, lead generation sites often feel more like traditional eCommerce sites since the lead often has a specific site endpoint. But there are a few common twists you need to be aware of. One of the biggest mistakes I see with lead generation sites is that they often have multiple lead types on the site but don't weight them differently as conversion goals. This can be disastrous. It's precisely equivalent to an eCommerce site with vast disparities in average cart size choosing to optimize on sales not revenue or margin. But while eCommerce sites rarely make this mistake, lead gen sites do it all the time. If you do have multiple lead types on your site, and you don't weight them, then it's almost inevitable that my basic rule about SEM (bad traffic is cheaper than good traffic) will cause your program to emphasize crappy, low-value leads at the expense of high-value leads. Why? Because your competitors may have taken the trouble to actually do this - and they will bid up the words that send good leads and leave you with apparently less expensive bad leads.

Every bit as common a problem is what to do about phone leads - and it's much less tractable. Most lead gen sites rely heavily on phone lead generation. Typically, that means every page and every call to action has an 800 number. Ideally, that ought to be a set of unique 800 numbers. But even when it is, there's lots of fog in the analysis. If you take lots of phone leads, you have to make some tough decisions. First, you may decide that your online lead gen is a good proxy for your total lead gen. If that's true, then your online leads should be considered by your sales staff to be about as good as phone leads. If that's not the case, you've got a problem. If you don't put phone numbers everywhere on your site (and you probably should), you may be able to identify good conversion proxies based on tracked, unique phone volumes. More likely, you'll have to build some model of engagement on your site and test it as best you can with tools like survey research.

Media and Ad-Based sites form the next tier worth considering. For sites like this, there actually is a clear path to success. It just isn't a success with a traditional single-page endpoint. For most ad-based sites, success is measured by a combination of page views generated and value per impression. There's often a pretty dramatic difference in the impression value for ads placed on different pages. That complicates the optimization model in web analytics - which already does a pretty poor job of letting an analyst use measures like "Total Lifetime Pages" or "Pages in Period X After Event" when doing either optimization or analysis.

We generally use two strategies for ad-based optimization. The first is to optimize outside of real-time by analyzing chunks of data. We typically use campaigns or time-based variables to create a population of visitors who entered from the SEM program during a specific period. Then we analyze their total behavior from that period to date. This allows us to calculate value per visitor and appropriately optimize the SEM program.

If a program needs real-time optimization, however, that approach won't work. Instead, we shoot for developing an "engagement" proxy. It's really the same idea as a conversion proxy. We try to find behaviors in the 1st visit that are reasonably predictive of the extended value. If we can do this (sometimes you just can't), then the "engagement" proxy can be used to optimize the SEM program in real-time.

The last type of site I'm going to consider is one where there is no clear success event at all. Sites that exist primarily for branding, information or relationship building may all fall into this category. For these sites, the most common error by FAR in SEM programs is to optimize to traffic. Optimizing to traffic is always a bad idea (unless, I suppose, you can charge more for impressions on your Landing Page than you are paying). Remember the basic rule about SEM? Bad traffic is cheaper than good traffic.

For sites that exist primarily to brand, you'll probably want to optimize with some form of engagement metric. That's a topic I've written extensively on before - and it's also yet another case where there is no one right engagement metric for every site. What I'd suggest first is trying a range of engagement metrics and seeing what percent of your population falls into each. I'd also look at which Search Terms and campaigns are doing well or poorly by each measure. You probably have a fairly decent intuitive sense of which search terms and campaigns are driving better qualified traffic. If your engagement proxies aren't close to that intuitive sense they're usually bad. In addition, you should look for an engagement metric that qualifies enough visitors to be statistically interesting and doesn't qualify too many visitors from almost every campaign. In general, you'll probably want to pick the most DEMANDING metric that still generates enough data to analyze.

What are some common engagement metrics? Simple metrics like 2+ page views can often improve optimization dramatically compared to "traffic." But such simple metrics are rarely good in-and-of-themselves. 2+ visits is a surprisingly useful optimization metric given its simplicity. Time on site is appropriate for some sites and can be pretty decent optimization metric. Total pages can also be useful. You can use Functional measures of engagement - tracking the percent of people who reach key pages on your site. This is especially valuable if you know that significant percentage of the traffic on your site isn't engaging in ways you care about (maybe it's career traffic for example). Ideally, you'd probably like to use a fairly rich combination of factors (like key pages, interactions, time on site, etc.) to really come up with a good index of Engagement.

That's no easy task in most tools (VS can do it - and check out the new WebTrends Scoring system - very cool btw - and something I'm going to write about shortly). You might well have to short-cut here and there to come as close as possible to a good answer.

How do you know you've done a good job picking an Engagement metric? The truth is, there isn't any behavioral analytic technique for absolutely proving this. Instead, you might want to think about using survey research to gauge interest and then tie those results back to your various engagement models and see which one is most predictive.

What does this all add up to? There are endless permutations of appropriate optimization goals. When you pick the one(s) you are going to use, these are the things to keep in mind:

Do the optimization points capture all the important aspects of site success?

Do they weight those important aspects appopriately?

Are the optimization points biased in ways that favor specific campaigns or campaign elements?

Can the optimization points be used for real-time optimization?

Are there simpler proxies that would work for the same optimization points or that would provide real-time optimization?

Part 5 in a Series on Web Analytics and Search Engine Marketing Programs

[Of all the many reactions to my last rather scorch and burn post, the two that pleased me the most were Marshall Sponder’s and Douglas Karr’s. There were two things that pleased me about Marshall’s post: first that he "laughed and laughed" as he read it. Second, that he manages in a more gentle and probably reasonable way to sum up pretty much what I was trying to say. If I "stewed a few Irish babies" to get there I guess that’s just me. It isn’t the first time Marshall has managed to get to the pith of what I was saying in rather fewer and more polite words. Speaking of which, you should check out Douglas Karr’s extraordinarily pithy comment below. It made me nearly spit my Diet Coke out with laughter. And I can only say – touché! BTW, I posted (and unlike my usual practice, will post) every official Typepad comment. The following is the continuing part of the series on SEM Analytics that got supplanted from its usual place.]

Part 3 of this Series covered high-level Traffic Analysis: comparing Search Traffic to Total Site Traffic, Organic to Paid Traffic, Traffic by Engine and Traffic over time. Collectively, these high-level reports provide the basic context for understanding the impact of SEM on your site. Part 4 delved into the next level of SEM reporting: looking at traffic by Ad Group, Entry Page and, of course, Search Term. These reports are essential in most of the analyses from here on out. But all of the Posts so far have concerned themselves with looking at "Traffic." There are a few types of analysis (such as SEO Holes) that just focus on traffic. But most reports – whether they focus on finding growth opportunities or improving efficiency – demand the addition of some form of success metric. For many sites, finding the right success metrics for optimization is the single most important task in SEM Analytics.

If you’re site is a traditional eCommerce site, then finding the right optimization metric may look fairly straightforward. But even in this apparently simple situation, there often lurks considerable complexity. Here are five quite reasonable optimization models for eCommerce sites: optimize to sales per visit, optimize to revenue per visit, optimize to sales per visitor, optimize to revenue per visitor, optimize to lifetime value per visitor.

I’ve listed these in increasing order of complexity. Optimizing to sales is the easiest – both in Bid Management Tools and in Web Analytics tools. And it’s a perfectly workable solution in SOME situations. If the bulk of your conversion is single session, if you don't sell many products over time to a single customer and you have a relatively narrow band of product pricing, then sales per visit will work well enough. If any of these conditions aren't true, then optimizing by number of sales will cause significant mis-optimizations in your program if used as the goal.

Optimizing revenue per visit fixes one of these potential mis-optimizations. By taking into account the value of products sold (or – in cases where it doesn’t track well – the Margin on Products sold), you insure that your PPC program doesn’t optimize to the lowest-value conversions. But if you have significant multi-session sales cycle or significantly different customer values over time, then you’ll still be mis-optimizing.

Using Sales per Visitor fixes the other half of the equation. By going to a visitor metric, it solves attribution by visit problems. But, of course, it does nothing about mis-optimization by driving to low-value carts or different customer value.

It would seem that Revenue (or Margin per Visitor), our fourth model, would be the ideal. And indeed, it does solve most optimization problems. However, in cases where you expect significant ongoing sales or other revenue from 1st Time Buyers, then you may significantly under-represent the value attributed to a marketing campaign if you only track the current measured effects. Since optimization has to take place within a specific period of time, optimizing to revenue can have two negative effects: it can make it look like your programs are less effective than they actually are, and it can optimize purchases without a long value tail vis-à-vis purchases with a long value tail.

There are two key points you should take away from this. First, no one optimization strategy necessarily fits every site. Using Customer Lifetime Value is the most comprehensive strategy – but it’s also by far the hardest to implement. If you have good reason to believe that most of your optimization IS reasonably captured by using sales per visit (or any of the other metrics) then the simpler metric is the better choice. The second key take away is that choosing an optimization tactic is a HUGE decision. Because if you choose the wrong tactic, then every optimization effort you make will further mis-optimize your program. So before you pick one of the short-cut methods, be sure you understand what you’re missing and are fairly confident that it IS NOT important.

Choosing an optimization point has been compared to picking a direction on a map. If you pick the wrong direction, then every step you take going forward will move you FURTHER away from where you want to be.

The classic case of this type of mis-optimization is when SEM programs optimize to CLICKS (traffic). Typically, you start out with a program where the keywords and bid points are set by human intuition and reflect a reasonable understanding of which keywords likely produce good traffic and which are less valuable. But as the program is optimized for Clicks, bad traffic drives out good traffic. And the program will focus on increasingly worthless (but cheaper per click) traffic with each optimization. Eventually, you’ll end up with a truly rotten program. What’s ironic is that the harder you work to optimize in this situation, the quicker your program will go bad.

If you’re a traditional eCommerce site would you ever not focus on Sales or Revenue or LTV for optimization? As a matter fact, you might. For some sites, the rate of Conversion to Traffic is quite small. That means that if you try to optimize your SEM campaigns on Sales or Revenue, you may have to wait a very long time before you have statistically significant numbers. And for many aspects of your program (such as creative rotations and individual keywords) you may NEVER have statistical significance. That’s a big problem! So if your eCommerce site is stuck in this boat, you’ll need to consider the techniques for building models of Engagement or, as we sometimes call them, Conversion Proxies. That will be the topic of my next post in the Series – and it will deal with the many, many situations where eCommerce transactions either don’t exist or are two infrequent to optimize against.

Part 4 in a Series on Web Analytics and Search Engine Marketing Programs

Traffic and conversion numbers at the channel (Search/Organic/Paid) and engine (Google, Yahoo, MSN, etc.) level are the backdrop for almost all Search Analysis. But there are several deeper layers of Search data in web analytics and these deeper layers are where most of the optimization analysis actually takes place.

If you’ve setup your Search program to pass Ad Group as a campaign code (strongly recommended – see Part I of this series), then you’ll be able to break down PPC campaigns into much more detailed "concept" groupings. The words in an Ad Group should all revolve around a single concept – and they share both Landing Pages and Creatives. Because Search Term is often too granular for analysis, Ad Group is probably the single most common level of analysis in good Search Analytics.

Set up in this way, you’ll automatically get your basic tool reports about campaigns. Typically, these include at least "responses," "repeat responses," "conversion by type," and, quite often, information about the average "time to convert." For eCommerce sites, this will also typically include DETAILED information about actual purchases sourced by this campaign.

There is already some pretty useful information here. Repeat responses helps you understand how frequently visitors are coming in on PPC campaigns. Time to convert provides good insight into the length of the sales-cycle on your site. And, of course, having detailed eCommerce data is invaluable for really good optimization.

With many tools, you can also create roll-ups to different campaign levels. With SiteCatalyst, for example, you use SAINT to attach additional campaign attributes to a specific campaign. With these roll-up levels, you’ll get the same kind of statistics (and analytic opportunities) at every level. That means you can evaluate data like repeat responses for an Ad Group, a Campaign, an Engine and for Search in general. That’s good information. Information that’s impossible to get from your Search Engine reports.

For organic search programs, there is nothing quite like Ad Group. That makes it extraordinarily difficult to impose much order at the Search Term level. In some cases, though, you can use Entry Page as a form of grouping SEO. Since SEO search terms are strongly related to pages and pages to concepts, you can often analyze SEO terms by Entry Page or Entry Area.

Entry Page can also be very useful for analyzing Paid Search. You’d think you already have this information if you’re coding by Ad Group. But web analytic tools are often limited in the cuts they can give you by any one piece of information. So there are many cases where using the Entry Page will give you additional reports (such as path, next page and link analysis) beyond what you’ll get from campaigns.

When a Landing Page is specific to a PPC campaign, you’re set to go. You can use it without additional segmentation and be confident that you are only seeing Search visitor behavior.

For SEO and for cases where PPC visitors are landing on multi-use pages, an additional level of segmentation is required. The easiest segmentation is generally at the visit level; with the criteria being that the visitor was sourced by Organic Search or PPC (or a particular PPC campaign if your campaigns share Landing Pages).

Applying this segment and then looking at the Entry Page reports will tell you exactly what happened when search visitors landed on the page. Note that it is much more efficient to apply a single segment (PPC source) and then look at Entry Page behavior than it is to build a segment for each Entry Page and then look at PPC sourcing!

If you use Broad Match and you’ve setup your PPC campaigns to pass the "purchased term," this becomes another level of analysis. Sometimes, this may just be an Ad Group. However, some sites prefer to pass this as additional attribute on the URL. When done in this fashion, it will have to captured as either a campaign attribute or custom variable (depending on the tool).

At the lowest level, most web analytic tools provide you with at least some reporting about the actual keywords entered. This data will typically be associated with each of the higher levels already described. So you can look at all search traffic by keyword, engine traffic by keyword, Ad Group (campaign) traffic by keyword and Broad Match Term by ACTUAL keyword.

In some cases, data about search terms is broken out into two categories: clicks and referring clicks. What’s the distinction? People tend to think of Search Engine visitors as always "arriving" at the site. Most times that’s true. But it isn’t actually definitional. Suppose a visitor searches on Google, clicks on your Ad and Lands on your site. Then hits the back button and goes to Google. Now if that user clicks on your link again, searches again and clicks on another of your links, or clicks on your organic link instead of your paid link, then that visitor will land on your site again. But here’s the really interesting part. That second land DOES NOT start a session. It’s counted as a click, but not as a REFERRING click.

From a web analytics perspective, a session is the same machine landing on a web site within 30 minutes of a previous page request. What the visitor did in between those two requests is both unknown and immaterial. Situations like this are why you can have multiple EXIT LINKs and multiple Search Clicks for a single SESSION. Confused yet?

Incidentally, if you’re a publisher and you carry Google (or other Content Network Ads) on your site, one of the more interesting web analytic projects is to figure out how many of the visitors who click on those Ads come right back to your site. It can be quite a large number – good news for publishers but usually bad news for advertisers.

The upshot of all this is that Clicks (as opposed to Referring Clicks) don’t tie to other variables like Entries or Visits. But it’s often very interesting to see Clicks vs. Referring Clicks. We commonly observe 15-20% differences between these two numbers – a phenomenon we call PPC Self-Cannibalization. It’s important to keep this phenomenon in mind when you evaluate your PPC reach and cost per visitor.

Search Term level analysis is usually confined to the relatively small set of terms that generate lots of volume. For sites with long-tails in SEO or PPC, the majority of these terms simply won’t have enough volume to analyze. This is also an area where web analytic tools sometimes "clip" your data. That means you won’t even see all the words in the tail. If you think your data is getting severely clipped, it’s worth talking to your vendor and complaining. In many cases, these limits are adjustable on request.

So if you are set up correctly, you should be able to go from Total Search Traffic, to Organic and Paid Traffic, to traffic by Engine, to traffic by Ad Group and Broad Match Term and, finally, to traffic by Keyword. And you should be able to see Keyword, Ad Group and Engine cuts at every higher level of granularity. Finally, you should be able to use Entry Page to track some groupings of both organic and paid search terms and to get reports like next pages, path and link analysis for just the population of Search entries.

These reports form the bulk of what you’ll be looking at in most Search Analytic projects. And for all of these reports, you should be able to get traffic numbers out of the box and eCommerce and Conversion numbers if you’re site supports those.

For many sites, eCommerce and Conversion numbers aren’t really applicable. So in the next post, I’ll start to discuss optimization metrics, measures of engagement and conversion proxies. These are behavioral measures that can help both PPC and SEO optimizers better understand the quality of the traffic they drive and how it can be optimized.

[Don’t forget to register early for X Change – Like "Top Gun" for Web Analysts! The Early Registration Discount ends July 31st! Use Coupon Code XCForum at http://www.semphonic.com/conf to reserve a spot!]