Urchin WebAnalytics Software is discontinued and is no longer supported. All Urchin documentation applies only to the Urchin product as it was at the time of discontinuation, and does not apply to any Google Analytics products or services.

Urchin Traffic Monitor (UTM)

Overview

The patent-pending Urchin Traffic Monitor (UTM), originally
available in Urchin 4,
was specifically designed to provide the most accurate
measurements
of unique website visitors. For businesses looking to get a
deeper
understanding of their online visitor behavior, the UTM is an
extremely valuable technology that combines the best of
client and
server side information while letting you control the data.
Easy to
install, this technology allows business owners to exactly
identify
unique visitors, click paths, and return loyalty metrics
including:
first time visitors, returning visitors, and frequency of use.
The second version of UTM, UTM-2, released with Urchin 5,
expands
these capabilities, capturing additional browser parameters and
loyalty metrics. UTM-3, released with Urchin 5.5, adds a
powerful
campaign tracking capability. Subsequent versions of the UTM
released with Urchin
5.6, and Urchin 5.7 contain
a number of enhancements to the campaign tracking capability.

There are two components to the Urchin Traffic Monitor
System: the
UTM Sensor, which is a lightweight module installed into the
content
of the website; and the UTM Engine which is part of the log
processing Urchin Engine. The UTM Sensor enables
client-side data
collection, which is then funneled back through the web server
augmenting the normal logfile. The client-side information is
combined with the existing server-side data by the UTM
Engine to
provide a more accurate and complete picture of website
activity.

The UTM Sensor is a small amount of JavaScript code that
accomplishes
two important functions. First, the Sensor negates the
effects of
caching by forcing at least one hit to progress to the
original web
server for each pageview. The impact on the server is
minimal, and
the details about the additional hit are logged into the
normal web
logfiles resulting in a more complete data set. Secondly,
the UTM
Sensor uniquely identifies each visitor by using client-side
"1st party" cookies
to keep track of first time and returning visitors. This
cookie
identifier is a communication tag only viewable to your web
server in
the same nature as session ids. It is not a third party
cookie, which provides information outside your system,
violating
many privacy policies.

The above diagram illustrates the operation of the UTM
System. The
web server in the middle of the diagram provides two basic
functions:
content delivery and logging. The content of the website
includes
the UTM Sensor which is delivered to the userï¿½s browser,
shown on the
left. The UTM Sensor sets unique identifiers and sends an
additional
request back to the same web server. This additional
request is
logged into the normal log file along with all of the normal
traffic. The UTM Engine, which is part of the Urchin log
processing
engine, understands this additional data and merges the two
types of
data together providing an accurate and more complete
picture of
visitor behavior.

UTM Sensor

The UTM Sensor increases the accuracy and completeness of
logfile
data by negating the effects of caching and proxying. The
following
example illustrates how the UTM Sensor handles caching.
Shown in the
figure below, the user receives the content of a pageview
from the
cached memory of the browser. This typically occurs when
the user
goes back to a previously viewed page. The same model
applies if the
caching is provided by a service provider. In the example, the
content for page "X" is not delivered from the web server,
but from
the cached memory of the browser. At this point, there is no
knowledge of the pageview as it is not seen by the web server.
However, the UTM Sensor activates an additional unique hit that
forces at least one small record back to the original web
server.
This information is logged in the normal logfile, which now has
knowledge of the originating "X" pageview.

The second important function of the UTM Sensor is to uniquely
identify both sessions and unique visitors. Through a
patent-pending
combination of browser cookies, the Sensor detects and
initializes
the unique visitor and session identifiers allowing exact
monitoring
of new and returning visitors regardless of service provider
proxy
behavior. Most service providers take advantage of proxying by
recycling IP addresses and clustering users behind
firewalls. This
can cause problems with normal logfile tracking, which
typically
utilizes the IP address as an identifier of the user.

In the example shown in the figure below, the UTM Sensor is
able to
pierce the veil of the proxy by utilizing the cookie
identifiers
instead of the IP addresses. In the figure, a first time
unique
visitor accesses the website through a firewall with IP
address #1.
The delivered Pageview contains the UTM Sensor, which sets the
identifier on the visitorï¿½s browser.

On the return visit by the same visitor shown in the bottom
of the
figure, the unique id is passed to the web server along with
each
request. So even if the user is now assigned a second IP
address,
the UTM technology properly identifies the visitor with the
original
id. In addition to negating the effects of complex proxying
techniques, this also tracks visitors who travel and may use
their
laptops from several locations and through several providers.

Once the additional UTM data is recorded in the normal web
server
log , the UTM Engine will recognize and process these
additional hits
in order to create an exact analysis of each click of the
user.
During installation, it is important that the logging format is
checked for both referral and cookie logging to be present
so that
all of the appropriate data is stored.

Installation

There are four steps to installing the UTM system, which can be
accomplished in a very short amount of time. Complex sites
may be
able to take advantage of existing server-side includes or
centralized delivery
methods to shorten the installation. During installation,
you will
need access and permissions to modify the content of the
website.
You may also need to modify the logging of the web server,
which may
require a different set of permissions. The following four
steps do
not necessarily need to be performed in order.

Upgrade Note: UTM-2, which ships with Urchin 5, is
not recognized
by Urchin 4. Once UTM-2 is installed, you will no longer be
able to
run Urchin 4. All versions of Urchin 5 recognize both UTM-1
and UTM-2. As well,
as of Urchin 5.5 there is UTM-3. Only Urchin 5.5 and up can
process UTM-3
data. Therefore,
when upgrading, it is important to migrate to the
appropriate version of
Urchin before installing a more recent version of the UTM
sensor. UTM-4 data, however, can be processed by any Urchin
5.x version.

1. Install UTM Sensor into content: The first
step in
installing the UTM is to include the JavaScript and GIF
components of
the UTM Sensor in the content of the site. The two pieces
necessary for completing this step are included in the
util/utm/
folder within the Urchin distribution. It is important that
the
names of these two files are not changed and that they are
copied to
the document root directory of the website. Either drag and
drop,
upload, or copy the __utm.js and the __utm.gif files into
the main
directory of your website.

Once these files are in place, you will need to include the
__utm.js
file at the beginning of each webpage in the website. If
your site
utilizes server side includes and you use a header include
for each
file, it is possible to include the UTM in the beginning of
this
include file only. It will then automatically be a part of
each
webpage. For static HTML sites that do not use includes,
you will
need to modify and add the UTM entry to each page
individually. For
dynamic sites that use a content generation engine, the UTM
can be
included at the beginning of the template that is delivered
to the
customer.

In any case, the following line of code should be included
in the
beginning of each HTML page, but after any META tags, that
is delivered to the end
user. For
static sites, edit each webpage and add the line
below before
the rest of the HTML content (but after any META tags).

<script src="/__utm.js"
type="text/javascript"></script>

For sites that undergo regular maintenance or have multiple
authors,
be sure to build the addition of this line into the your
internal
website authoring procedures, guidelines, and QA processes.

If you are using a package like "HTML Tidy", you may want to
include
the Javascript line in the HEAD area of your page to make it
more
palatable, for instance:

2. Set UTM Domain (if necessary):
The UTM (beginning with UTM-2) has a
domain setting
that controls the scope of the cookies. For single
websites, the default
setting, "auto", can be left alone. If you have multiple
websites that
share a common root domain and you wish to process them
together, then
the domain should be set to the common root domain.

To set the domain setting, edit the __utm.js file that
was copied into your document root in step 1. Towards the
top, you
will see the line:

var _udn="auto"; /*-- ...

Change the word "auto" to the domain that the cookies should
apply
to. The domain must be part or all of the actual URL for
this site.
Example:

var _udn="urchin.com"; /*-- ...

3. Activate cookies in the logging: The third step to
installing the UTM System is to verify and potentially
modify the
logging format of your web server. For the UTM to function
properly,
it is required that both referral and cookie information is
logged.
You will need access to the configuration of the web server.
The
following general guidelines should work for most IIS and
Apache
users, however you should check with your system
administrator to
ensure proper formats.

For Apache Users:
Apache servers typically use a configuration file
called "httpd.conf." Within this file, configuration
directives
determine the format and location of logfiles. By default,
most
Apache configurations will log in the NCSA Extended Combined
format,
which includes referrals and user-agents, but is missing the
cookie
information. Be sure that your logfiles contain the
"{Cookie}i"
field specification. To modify your logging format from the
default,
a "special" LogFormat directive can be added and then the
log files
can reference this format using the CustomLog directive.

The above example is provided as a reference and does not
apply to
all possible Apache settings. Please refer to the Apache
documentation and consult with your system administrator on the
actual directives needed to activate cookie logging. The
LogFormat
directive specifies the specific format of the log file.
The example
shows the addition of the cookie information to the end of
the log
file. This format is then named "special" so that it can be
identified in the virtual host configurations. The CustomLog
directive in the virtual host specification identifies the
location
of the log file and the format to use. The example uses
the "special" format as defined previously.

For Microsoft IIS users:
The Internet Services Manager provides a point-and-click
interface
for adjusting the web server configuration. To access this
manager,
you will need to login to the web server with the appropriate
administrator privileges.

To access the Internet Services Manager, click on the
"Start" menu --
>
Settings --> Control Panel, and then double click on the
Administrative
Tools Folder and then on the Internet Services Manager icon
to open
the manager.

Modifications to each website can either be made
individually or the
entire server can be modified. In the left window either
right-click
on the server name to modify the entire server, or
right-click on the
website name such as "mysite1.com." Select the "Properties"
option
to open the properties dialog box. For the entire server,
click on
the "Editï¿½E button with "WWW Services" selected in the menu
to bring
up the Properties dialog box shown on the left below.

Shown in the above figure, be sure
that logging is enabled and set to the "W3C Extended Log File
Format." Then select the "Propertiesï¿½E button to configure
the log
file format specifics.

The window shown above will appear.
Click on the "Extended Properties" tab, scroll down and make
sure
both the "Cookie" and "Referrer" boxes are checked. If not,
check
these boxes and "Apply" the changes to the site.

Whether you use IIS, Apache or another web server, please
refer to
your server documentation for more information on
configuring logfile
formats. All major web servers support the logging of
cookies and
are easily modified to activate this feature.

4. Set Urchin configuration to UTM: The final step in
configuring the UTM for your site is to enable the UTM
tracking in
the Urchin Configuration. This is either done at the time
the Profile
was created or after by editing the Profile.
Open the Urchin Configuration either
directly on the machine or by logging in remotely as the
"admin"
user. Your installation instructions will provide more
details on
how to access the configuration.

Once open, Click on the "Configuration" icon to the left to
provide a
list of the existing Profiles in the configuration. To
enable UTM
tracking for a particular Profile, click the "Edit" to the
right of
the profile name. (Note: if you have not already added the
profile,
do so now using the "Add" button). After clicking on the
"Edit"
button, click on the "Reporting" tab to bring up the
Reporting Settings
Window.

Under the "Visitor Tracking Options" section, use the menu to
select "Urchin Traffic Monitor (UTM)" for the Visitor
Tracking Method.
If you explicitly set the UTM Domain in step 2, then set the
UTM Domain
setting in the above figure to the same value as in step 2.
If you did
not specify the domain in step 2, then set the UTM Domain to
the address
of your website without the "www.". If your website domain
does not start
with "www.", then use the whole thing. Click the "Update"
button to
save your settings. Thatï¿½s it. The installation is
complete, and
future traffic will contain and benefit from the the UTM System.