9 Monitoring and Maintaining the System

This chapter explains the tasks performed by the Administrator. These include monitoring the status of the system, performing backups and upgrades, working with the log file, and issuing messages to system users.

9.1 Monitoring the Status of the System

The Administrator can check the system's condition, and receive automatic status monitoring messages on the Status page. To reach this page, select System, and then Status. An example is shown in Figure 9-1:

Through the Status page, you can the status of the attached Collectors and the log file process, the current level of processing within the system, and the error log. You can also configure which users are notified (and how) about a system status error.

9.1.1 Temporary Delays and Alerts

Be aware that the system status indicator shown in Figure 9-1 is only updated when the browser screen is refreshed. If one or more of the system processes are found to be failing, a system alert can be generated (as described in Section 9.4, "Configuring System Failure Alerts"). Therefore, the situation can arise that a process is shown temporarily as failing (with a red cross), but no alert is generated. This is because the system status indicator has returned to normal by the time the system processes are checked.

Due to this design, when an alert is triggered, it is recommended that you regard it as a warning that the system is starting to fail. A failure can be the result of a system delay that is larger than the boundaries set the default (such as the latency between a hit on the monitored line, and the moment the information based on that hit is available in the Reporter, may not be long enough). This latency may be out of boundary within a high-traffic environment. A failure may also be the result of a temporary peak in traffic. However, if this condition persists, it is recommended that you review the monitored traffic level.

9.2 Viewing the Status of the Collectors

You can view the status of each Collector attached to the system by selecting System, then Status, and then Collector status. It opens the Network data Collectors window. An example is shown in Figure 9-2.

The System (localhost) refers to the Collector instance on the Reporter system. Other Collectors within the network are represented by their IP address. For each Collector, the following menu options are available:

View statistics: displays a detailed report of the traffic monitored by the Collector. An example is shown in Figure 9-3. This is described in more detail in the following section.

Restart: restarts the selected Collector. You are prompted to confirm the restart.

Disable: stops data monitoring by the selected Collector. In order to restart the Collector, you will need to register the Collector again. The procedure to do this is fully described in Section 9.2.2, "Attaching New Collectors".

9.2.1 Working With the Collector Statistics Window

The information shown in this window (Figure 9-3) refers to the traffic monitored since midnight for the selected Collector, or the counters were reset. The Uptime field in the bottom left-hand corner of the window shows the time the Collector has been running. The uptime is reset when the Collector is restarted to update its configuration. You can reset all HTTP request counters shown in the window by selecting Reset counters from the View menu. Note that the counters will be reset the next time a network packet is detected. Hence, on an installation with no network traffic, the counters will never be reset. The display is automatically refreshed every two seconds.

The tabs available in the top-left part of the part of the window provide a detailed breakdown of the traffic monitored by the selected Collector. They are explained in Table 9-1:

Table 9-1 Collector Statistics Report Tabs

Tab

Description

Interfaces

Provides information on the available network interfaces for data collection. The number of interfaces and their status depends on the system configuration. Note that you will not see any "normally" configured interfaces. For each available interface, the name (in the form ethx), utilization (that is, current bandwidth), and state are displayed. For each interface, the state can be indicated as "OK", "Down", "Not configured", "Not active", or "Not promiscous" (the network adapter is only able to see traffic sent to its MAC address).

Ethernet

Provides a breakdown of the raw packet data transmitted over the monitored ports in terms of its protocols (such as IPv4 and ARP), and the number of measured frames. The "Truncated" listing indicates corrupted or dropped frames.

TCP

Provides an analysis of the TCP stream. The following counters are reported:

In progress: the number of currently active TCP sessions. These are sessions for which there is currently data transfer, or which are still in the connection establishment stage, or sessions for which the disconnect procedure has been initiated, but has not yet completed. This counter is a direct indication of the network load.

Max simultaneous: the maximum number ever attained by the In progress counter since the Collector was started.

Connection reset: the number of sessions that were terminated with a TCP RESET segment. Such sessions are immediately dropped by both parties: no further data (including a disconnect procedure) can be sent on such a session.

Connection refused: the number of sessions that could not be established because the requested service was missing. This happens if a peer tries to establish a connection on a system to a port on which no one is listening.

Total: the total number of sessions that have taken place since the Collector was start.

The following network error meters are also shown:

Out of sequence: indicates the segments which are received out of sequence. A high level of errors could indicate a problem in the quality of the underlying network between peers, which is usually the Internet between a client PC and a server.

Bad checksum: indicates corrupted segments en route. A high number of issues can indicate either a hardware, wiring, or network problem.

Bad offset and/or length: indicates the number of packets that had an incorrect length compared to their advertised length, and indicates a corrupt packet.

Dropped segments: indicates the total value of segments dropped for any unexpected reason, such as bad checksum, length, and so on. Check your hardware and network architecture when this value becomes high.

HTTP

Provides an analysis of the monitored HTTP stream. In particular, the type of requests (such as GET or POST) they contain.

SSL connections

Reports the encryption method used for packets of encrypted data. In particular:

SSLv2: number of SSL version 2 connections (the Collector has no support for tracking these connections).

SSLv23: number of mixed mode SSL connections (that is, sessions that start as SSL version 2, but are scaled up to version 3 during the connection establishment phase).

SSLv3: number of SSL version 3 connections.

TLSv1: number of TLS version 1 connections.

Other: number of other connections (those connections that do not fit into one of above categories).

Errors related to SSL key management are reported. In particular:

No server key: the private SSL key for the requested server connection has not been made available to the Collector.

No master key: number of connections dropped because the master key for a connection could not be computed.

No session key: number of connections dropped because the session key for a connection is missing.

Information about (currently) unsupported encryption:

Pure SSLv2: client is using pure SSL version 2 protocol. This is not supported by the Collector.

Ephemeral: session relies on ephemeral keys for encryption. Such keys cannot be made known to the Collector and, as a result, such sessions cannot be tracked.

Anonymous DH: Session relies on anonymous Diffie-Hellman key negotiation. Such keys are unknown to the Collector and, as a result, such sessions cannot be tracked.

The Decrypt errors gauge indicates the connections which could not be decrypted. This can be caused by several reasons, such as the master key could not be decrypted, session keys were incorrectly computed, or a segment could not be decrypted.

SSL encryption

Provides a breakdown of the monitored encrypted data in terms of the employed encryption algorithm. The Used column indicates the amount (percentage) of total monitored SSL encrypted traffic that used an encryption algorithm, and the Errors column indicates the percentage of measured SSL encryption which failed (that is, could not be read).

Performance

Reports on the impact to the Collector. Note that if the peak load nears 100%, immediate action should be taken to prevent data being dropped by the Collector. See Section 8.2.2, "Limiting Overall Traffic" about traffic sampling. If this does not provide a solution, it is also recommended that you contact Customer Support. The Collector's memory usage is also indicated. The maximum memory threshold is 30% for Reporter/Collector systems, and 70% for Collector only systems).

9.2.2 Attaching New Collectors

To attach a new Collector to the system, select Register remoteCollector from the Configuration menu. The Register Collector dialog shown in Figure 9-4 appears.

Specify the IP address of the new system and, optionally, a brief description. When ready, click Register. See the Oracle User Experience Insight Installation Guide for more information about the configuration requirements for Collector systems.

Note:

This facility is also available by selecting System, then Status, and then Collector status. Note that users who are not authorized as the Administrator will receive a read-only version of this interface.

9.3 Specifying the Fallback Collector Encoding

The Collectors can monitor network traffic containing data in a wide variety of character encoding standards. Table G-1 provides a complete list of the encoding standards supported by RUEI.

In order for RUEI to correctly report on monitored network traffic, it must understand the encoding used within that traffic. Generally speaking, RUEI first attempts to use the encoding detected for the HTML document. If this fails to produce a satisfactory result, the fallback encoding (if one is specified) is used to decode URL and posted form arguments.

When using this facility, it is important to understand the following points:

At the HTTP level, there is no relationship between content encoding and URL encoding. Moreover, the HTTP protocol does not specify any standard for URL argument encoding.

The fallback encoding is only applicable to URL and POST arguments. Content-based reporting (for example, functional errors) is not affected by this setting. In addition, the selected fallback encoding applies across all applications, pages, and domains monitored by the selected Collector.

The fallback encoding is not a manual override to the auto-detected encoding. Rather, it specifies the encoding that RUEI should attempt to use once the auto-detected document encoding has failed to satisfactorily decode the URL and POST arguments. If the fallback encoding also fails to produce a satisfactory result, the arguments are reported in their original (non-decoded) format.

Important:

If you are using international characters sets within your Web sites, it is strongly recommended you carefully review your Web site content, and the encodings used for it. In addition, you should regularly review the reporting of full URL and POST arguments to ensure they are correct.

To specify the fallback encoding, do the following:

Select Configuration, then General, then Advanced, and then Fallback Collector encoding. A panel similar to the one shown in Figure 9-5 appears.

Use the Fallback encoding menu to specify the fallback Collector encoding. The list of available encodings is equivalent to that shown in Table G-1.

When ready, click Save.

Any change you make to this setting takes effect almost immediately.

9.4 Configuring System Failure Alerts

In addition to being notified about KPI and SLA violations, you can also configure alerts for system failure. It is strongly recommended that you do so to ensure prompt action in the case of system problems. To do so, select System, then Status, and then Status notification. The dialog that appears is similar to that described in Section 5.5.1, "Alert Profiles".

Note:

The system status alerting does not consider any alerting schedules or escalation levels. When configuring alerts, ensure all user information (such as e-mail addresses and telephone numbers) is correctly specified for the people who should be notified in case of system status failures. Note also that the system status check is run every 10 minutes. Hence, if a system failure is indicated in Figure 9-1, you may not immediately receive an alert about it, but when the scheduled system check is run.

9.5 Configuring Database and Disk Space Limits and Alerts

In order to ensure the uninterrupted operation of your system, limits are set to the maximum level of available database and disk space utilization. When the maximum database utilization level is reached, no further data is written to it until an administration mechanism has brought the database's size back to within its permitted boundary. Similarly, when the maximum disk space utilization is reached, no further data (in the form of log and enriched data exchange files) is written to the file system until an administrator process has deleted existing files. In addition, you can also configure alerts to be generated when either of these problems may be about to arise.

Important:

It is strongly recommended you only use this facility if you have a sound knowledge of RUEI, and clearly understand the use and effect of these settings.

In the case of an alert threshold, use the dialog to specify the maximum database or disk space utilization before an alert is generated. The generated alert is sent to the same recipients, and uses the same notification mechanism, as that defined for system failure alerts (described in Section 9.4, "Configuring System Failure Alerts"). In the case of a stop threshold, specify the maximum database or disk space utilization before database processing or data collection is stopped. When ready, click Save. Any changes you specify take effect immediately.

Defining Threshold Values

When defining threshold values, be aware of the following:

The maximum permitted setting for stopping the database or disk space utilization is 95%. This is because if the available disk space becomes completely (100%) full, other components on the system may no longer work. In addition, remote logging on to the system may no longer be possible. Similarly, if the database is allowed to become completely full, the administrative mechanism used to reduce its size will no longer work.

The specified thresholds refer to all partitions used for RUEI. That is, /var/opt/ruei, and any mounted partitions under it. The alert and stop mechanisms will be triggered if at least one partition reaches its specified threshold.

Checking of the defined thresholds is not performed continuously, but every 10 minutes. Hence, it is possible that by the time a check is performed, and an alert is issued, the database or disk space utilization is already higher than the specified threshold. For this reason, it is recommended that you set threshold values slightly lower than their intended target. For example, instead of setting the disk space stop threshold at 95%, set it to 93% or 94%.

An alert notification threshold cannot be higher than its associated stop threshold. For example, if the database stop threshold is 95%, the alert threshold cannot be higher than this.

By default, alert thresholds are 85%, and stop thresholds are 95%.

There is also a Linux operating system limit of 95% on disk space usage. If this limit is reached, only the root user can write to disk. Because RUEI does not have this privilege, further utilization of disk space is prevented.

9.6 Specifying Data Retention Policies

The availability of specific data within the Data Browser, as well as reports based on that data, depends on the amount of available disk space on the Collector and Reporter systems, as well as the amount of database space available on the Reporter system. This is illustrated in Figure 9-9.

Data gathered during monitoring is first written to log files, stored on the Collector system. These files are temporarily copied to, and processed by, the Reporter to populate the database that holds the multidimensional data structure viewable through the Data Browser and reports. These temporary log files are automatically removed from the Collector system after seven days. If the replay viewer facility is enabled on the Collector system, all hit-based information is held in a series of short-term data files. These files are regularly filtered into long-term data files that contain only information about failed events (that is, failed pages, objects, and function calls). While this information is viewable within the Session diagnostics replay facility on the Reporter system, the data is stored on the Collector system.

The size of the database user quota for the Report system is configurable during installation. By default, it is set to 200 GB. It is important to understand that data is consolidated when it is no longer required by the Reporter's defined retention policy. For example, by default, daily information about the last 32 days is retained. Daily information older than this is consolidated into the monthly information. Similarly, monthly information is consolidated into yearly information. Finally, if the enhanced data exchange facility has been enabled (see Section 9.18, "Exporting Enriched Data"), an export file is created every five minutes. The XML-based export files are, by default, retained for seven days.

By default, RUEI keeps information on a daily, monthly, and yearly levels for 32 days, 13 months, and five years, respectively. Hence, for example, the oldest daily information will be dropped after 32 days. In addition, temporary log files are kept on the file system for approximately seven days. Be aware that a new RUEI installation will grow quickest during the first 32 days. After that time, the growth rate will slow. Of course, the growth rate depends on monitored traffic levels.

As can be seen in Figure 9-15, every setting that has an impact on database utilization has a corresponding DB usage entry, while those that have an impact on disk space utilization has a disk space entry. For example, the Session diagnostics retention (days) setting has both a database usage and disk space usage entry.

The following options are available:

Maximum database size: specifies (in gigabytes) the maximum amount of data allowed to be stored in the database.

High-level data retention: specifies the period for which daily information is available. The default is the last 32 days. The maximum period for which daily data is kept depends on the monthly setting.

Medium-detail data retention: specifies the period for which monthly information is available. The default is the last 13 months. The maximum number depends on the yearly setting.

Low-level data retention: specifies the period for which yearly information is available. The default is the last five years. The minimum setting depends on the daily setting, while the minimum number depends on the monthly setting.

Failed event data retention: specifies the period for which information about failed URLs, pages, and service calls is available. The default is for the last 15 days. If information is not available in the Session diagnostics replay, you may need to review this setting. Note this setting is linked to the Collector long-term storage setting (described later in this section). If you intend to increase the Failed event data retention setting, it is recommended you also increase the Collector long-term storage setting in order to facilitate this. Note also this setting has a high impact on disk space usage, and any change to it should be carefully considered in terms of anticipated network traffic.

Session diagnostics retention: specifies the maximum number of days for which session diagnostics information is available. This facility is fully described in Section 3.9, "Working With the Session Diagnostics Facility". The default is the last seven days, and the minimum is the last two days. This setting has an impact on database and disk space usage. The reported database usage is not included in the reported disk space usage.

XML enriched data exchange retention: specifies the maximum number of days for which XML enhanced data exchange is available. This facility is fully described in Section 9.18, "Exporting Enriched Data". The default is the last seven days, and the minimum is the last 24 hours. Be aware that, if set to one day, the previous day's data is deleted at around midnight, and only a limited amount of information is available for the current day. In order for you to be able to download the previous day's data after midnight, it is recommended that a maximum of at least days is specified. The maximum depends on the available database and disk space. The location of the files is the directory /var/opt/ruei/processor/xml-events/wg/xml-sespage. Each file name has the format yyyymmdd.

For each option, the DB usage column shown in Figure 9-10 indicates the total database space (in gigabytes) currently used for the item, and the proportion this represents of the database's maximum permitted size. Because the session diagnostics and XML enriched data exchange facilities require disk storage, their total disk space utilization, and the proportion its represents of the maximum available disk space are reported.

Select the appropriate option. A dialog similar to the one shown in Figure 9-11 appears.

Use the dialog's control to specify the retention policy for the selected option.

For most settings, you can click Calculate to see the effect of your selection on database or disk space usage, as applicable.

When ready, click Save. Any changes you specify take effect immediately.

Note:

It is recommended that if you want to increase the amount of data kept, you start with the low-level data retention setting and work towards the high-level data retention setting. If you want to decrease the amount of data kept, start with the high-level data retention setting, and work towards the low-level data retention setting.

Calculating Required Days, Months, and Years

When specifying the high, medium, and low-level data retention settings, it is important to understand the dependency between stored days, months, and years. Use the following rules to calculate the required settings:

A month is assumed to have 30 days. The number of months that must be stored for a specified period of days is the number of days divided by 30 (rounded up to the next hole integer), plus one. For example, 33 days would require 33/30 (1.1 rounded up to 2), plus 1. Hence, three months.

The number of required years for a specified period of months is the number of months divided by 12 (rounded up to the next whole integer), plus one. For example, 11 months would require one year, while 13 months would require two years.

For example: 900 days, 31 months, and 3 years.

9.6.2 Defining Collector Data Retention Policies

To specify the data retention policy used by a Collector system, do the following:

Select Configuration, then General, then Advanced settings, and then Collector data retention policy. The panel shown in Figure 9-12 appears.

Use the View menu to select the required Collector. The System (localhost) represents the Collector running on the Reporter system. The Oldest data entry column indicates the age (in seconds, hours, minutes, or days) of the oldest entry in the corresponding store. Normally, if the oldest entry in the short-term replay viewer store is reported as less than 15 minutes, this indicates a possible problem. For example, there is not enough time to process the short-term replay files into the long-term replay files. Similarly, if the oldest entry in the long-term replay viewer store is less than the number of days for which failed data is available, then replay viewer data will not be available for those days. Conversely, if the oldest entry is much larger than the number of days for which failed data is available, this may indicate that disk space is being unnecessarily wasted.

If content information is not available in the replay viewer, you may need to review this setting. You should also review the Failed event data retention setting.

If the long-term replay viewer store holds more days of data than the failed event data setting, then the extra amount of data is not accessible via the GUI. Conversely, if the long-term replay viewer store size setting is lower than the number of days of failed event data, then replay viewer data will not be available for the extra period. However, the other views on the data will be available as usual, through the other Data Browser groups.

Note in the case of a local Collector, the size of log files stored on Collector item will always be zero.

9.7 Viewing a Traffic Summary

You can open an overview of the monitored network traffic by selecting System, then Status, and then Dataprocessing. This provides you with immediate information about hits, pages, and session processing, as well as the system load. An example is shown in Figure 9-14:

Note the Available resource usage (%) item on the Performance tab indicates the current processing level. If this approaches 100%, it means a lag in the processing of data is starting to occur, and it is no longer possible to process data in real time.

Be aware that because this facility is based on application logic, non-application traffic (such as suites, services, and SSOs), are not represented in the displayed reports.

Important:

In order for RUEI to correctly report on monitored traffic, it is strongly recommended that you regularly review this traffic summary. If necessary, review the RUEI configuration accordingly. For example, add additional cookie technologies. In addition, if the system is unable to track sessions, proper tracking of transactions will also not be available because transaction reporting requires session tracking.

9.8 Creating and Restoring Configuration Backups

You can create backups of your system's current configuration, and restore it if necessary. It is recommended that you regularly make backups. Note that backups only contain the system settings. For security reasons, SSL keys and collected data are not included.

To create or restore a backup, do the following:

Select System, then Maintenance, and then Backup and restore. The Backup and restore dialog shown in Figure 9-15 appears.

Use the radio buttons to selected the required operation. When ready, click Next.

You are prompted to specify the location for the created or restored file. When ready, click Next.

Important:

The generated backup file contains large amounts of information intended for Customer Support use only. Do not try to modify the file's contents. When performing a restore, be aware that all current settings are overwriten by the restored ones.

9.9 Issuing Messages to System Users

You can issue messages to system users to keep them informed about important system events or operational issues. For example, scheduled maintenance periods, installation of service packs, or reported problems. The messages you post are displayed in the Messages area of the user's dashboard (see Figure 1-2). You can create new messages, or re-configure existing messages.

9.9.1 Creating Messages

To create a system message, do the following:

Select System, then Messaging, and then Newmessage. The dialog shown in Figure 9-16 appears:

Specify the content of the message. It is recommended that you try to keep this as brief as possible.

Use the Date fields to specify when the message should appear on users' message areas. Note the last three messages in the user's message stack are displayed. Hence, the message will remain on users' screens until either three new messages have been displayed, or you explicitly remove the message.

Use the Recipients field to specify the user roles that will receive the message. By default, messages are sent to all system users.

When ready, click Save to create the message, or Cancel to discard the message.

The message is displayed in the Messages section of the appropriate users' dashboard (see Figure 1-2).

9.9.2 Modifying Messages

To change an existing message (for example, to modify its text or recipients), right click the message, and select Edit from the menu. You can then modify the message's properties using the dialog shown in Figure 9-16.

9.9.3 Removing Messages

To remove a displayed message from the users' message area, right click the required message, and select Remove from the menu. You are prompted to confirm the removal.

9.10 Working with the Error Log

In addition to the status information described in Section 9.1, "Monitoring the Status of the System", RUEI maintains an error log. This file contains a record of all system events. Normally, it should be empty. If any error is reported in the file, you should contact Customer Support.

To view the error log, select System, then Status, and then Errorlog. A listing of the file's current contents appears. Within the error log, you can select the following options from the File menu:

Reload: refreshes the displayed file with any event information that occurred since you opened the file.

Mark as read: all events reported in the error file are also reported in the message area (see Figure 1-2). Use this option to clear the Status indicator. That is, return it to status OK.

Download: saves the error log as an ASCII text file. It is recommended that you save the error log and have it ready when contacting Customer Support.

Close: closes the error log file.

9.11 Configuring Text Message Providers

RUEI supports the use of text message notifications. In order to make use of this facility, all text message providers that you are planning to use must be configured and known to the system. To manage your provider information, select System, then Maintenance, and then Text messageproviders. The dialog shown in Figure 9-18 appears.

Select the required text message provider from the list. It contains a number of predefined supported services. Each of these require an account with the associated provider. When ready, click Next. A dialog similar to the one shown in Figure 9-20 appears.

Important:

If you specify a local GSM modem, a GSM modem must be installed on the system. The installed local modem must be a USB or serial GSM ETSI 07.05-compliant modem.

The exact fields available within the dialog depend on the provider selected in Figure 9-19. For example, if you selected a local GSM modem, you are required to specify the local port and baud rate for the modem. If not known, automatic detection is available. Optionally, you can also specify a SIM PIN (if one is required).

If you selected the predefined Mollie or Clickatell services, you are required to specify the user name, password, originator, API ID, and protocol sending method used for the account. These should have been given to you by your account provider. When ready, click Save. You returned to the dialog box shown in Figure 9-18.

Right click the providers in the list and use the Move up and Move down options to control a provider's position in the list. Providers are tried in the order they appear in the list. Hence, the first account is tried and, on failure, the second one, and so on.

When ready, click Close to leave the dialog.

Unicode Support

While unicode is supported in text messages, there are a number of restrictions of which you should be aware. In the case of locally installed modems, messages are sent to the modem using the 7-bit GSM 3.38 alphabet. Any unsupported characters in the original message are replaced by a question mark (?) character. In the case of an external service provider, it is recommended that you consult your service provider for information about multi-byte character set support. In the case of both locally installed modems and external service providers, text messages are limited to 160 characters.

9.12 Creating Helpdesk Reports

If you experience problems with the use or operation of RUEI, you can contact Customer Support. However, before doing so, it is strongly recommended that you create a Helpdesk report file of your system. To do so, select System, then Configuration, and then Helpdeskreport. You are then prompted to specify a location to which the file should be downloaded.

This file contains extended system information that is extremely useful to Customer Support when handling any issues that you report.

Please note that this file contains software proprietary information. Do not try to modify its content.

9.14 Resetting the System

If you experience unexplained problems, you can restart processing to ensure that it is operating properly and synchronized. Note that selection of this option will result in a temporary delay in data availability and monitoring.

In the last resort, you can remove all collected data from the system. Alternatively, you can reset all parameters (such as created users and environment parameters) to their out-of-the-box default values.

To reset the system, do the following:

Select System, then Maintenance, and then Systemreset. The System reset wizard shown in Figure 9-21 appears.

Restart system processing to reactivate system processing. This is the default.

Purge collected data to remove all collected data from the system.

Reset to factory defaults to remove all collected data and SSL keys, and resets all system parameters to their default values.

When ready, click Next.

Caution:

The Purge collected data and Reset to factory defaults options are irreversible. All collected data will be erased. In the case of Reset to factory defaults, all system settings will also be returned to their original state. Therefore, a complete initial configuration (and the definition of an Administrator password using the set-admin-password script) will be required before you have access to the Reporter interface. If you have previously created a backup (described in Section 9.8, "Creating and Restoring Configuration Backups"), you can restore this backup after initial configuration. This initial configuration is described in the Oracle Real User Experience Insight Installation Guide.

9.15 Managing the E-Mail Configuration

As explained in Section 2.2, "Using the Mailing Facility", RUEI can send automatic e-mails of requested reports. This facility uses the information specified during the initial configuration phase (described in the Oracle Real User Experience Insight Installation Guide). However, this configuration can be changed by selecting System, then Maintenance, and then Mailsetup. The Mail setup dialog shown in Figure 9-22 appears.

Return address: specifies the e-mail address to which failed or problem e-mails are reported. It is strongly recommended that this an address that is regularly checked.

From address: specifies the address the recipient sees in their mail client.

Reply-to address: specifies the address that users can click within an e-mail to reply to an e-mail. If this is not specified, the From address is used.

Mail size limit: specifies the maximum message size (in kilobytes) allowed for e-mails. Note that if an e-mail contains reports that exceed this limit, the system will try to split up the reports into individuals e-mails to overcome this limitation. Reports that are too large to be sent individually are not sent, and the user is informed of the problem. The default mail size limit is 5000.

Reporter URL: specifies the exact URL required for e-mail recipients to connect to the Reporter system. Typically, this is the same URL used by users to access the Reporter system.

9.16 Setting System-Wide Preferences

As explained in Section 1.6, "Customizing Your Environment", users can customize the formatting settings used in their sessions. They can specify the characters used for the decimal point indicator and the thousand separator, and the date format that should be used. The administrator can also specify defaults for these settings on a system-wide basis by selecting System, then Maintenance, and then Formattingpreferences.

9.17 Managing Users and Permissions

To start working with user definitions, select System, and then User management. The screen shown in Figure 9-23 appears.

The creation and modification of user accounts, as well as the maintenance of their information, can either be managed locally by the RUEI installation, or by a configured LDAP server. The procedure to configure an LDAP server for user authentication is fully described in Section 9.17.5, "Configuring LDAP Server User Authentication".

Use the radio buttons shown in Figure 9-25 to specify whether the creation of the new user account, and it's associated user settings, should be authenticated against the settings held in the RUEI installation (this is the default), or against a configured LDAP server. When ready, click Next. If an LDAP server is configured, the dialog shown in Figure 9-26 appears. Otherwise, a dialog similar to the one shown in Figure 9-29 appears.

Use the dialog shown in Figure 9-26 to specify the following information for the new user:

The user name by which the user will be known within your RUEI installation. This must be a unique name.

The user's full name.

The user's e-mail address. This is the address to which reports and e-mail alerts will be sent. Ensure it is correct.

If the user will be authenticated against the settings held in the RUEI installation, you are required to specify and confirm a password for the new user. See Section 9.17.4, "Enforcing Password Security Policies" for information about password requirements. Note the new password must be changed within seven days or the user is locked out.

Optionally, use the Disabled check box to disable the user at this time. You are free to enable them later.

If you selected user authentication against a configured LDAP server in Figure 9-25, you can click the Get user data from LDAP button to retrieve the user' s settings from the configured LDAP server.

When ready, click Next to continue. The dialog shown in Figure 9-27 appears.

Use the check boxes and radio buttons to specify the permissions to be assigned to the new user. The Business and IT access rights are described in Table 1-3. Click Finish to create the user definition. You are returned to the user list shown in Figure 9-23.

Note:

In addition to the settings described above, there are a number of additional settings (such as language, mailing type, and so on) that are set to their default values when a user is created. These additional settings can also be modified using the procedure described in Section 1.6, "Customizing Your Environment".

9.17.2 Modifying Existing Users

To modify a user definition, select System, and then User management. The User management panel shown in Figure 9-23 appears. Right click the appropriate user. The menu shown in Figure 9-28 appears:

Enable/Disable account: allows you to enable or disable the user account at this time.

Switch to: allows you to temporarily change to the selected user. This is useful if you want to view the modules and reports that they are authorized to see. Select Switch back from the View menu to return to your own role.

Remove: deletes the selected user from the system's user administration. Note that any private reports that the user created are also deleted. However, public reports created by the user remain available to other users.

Use the radio buttons to specify whether the user's settings should be authenticated against the settings held in the RUEI installation (this is the default), or against a configured LDAP server. When ready, click Next. If an LDAP server is configured, the dialog shown in Figure 9-26 appears. Otherwise, the dialog shown in Figure 9-29 appears.

Optionally, modify any of the displayed information. Note that the fields shown with a red asterisk indicate they are mandatory. That is, they can not be left blank. You can use the Disabled check box to prevent the user from using this account. You are free to enable them later. Because user accounts are automatically locked after a user has failed to correctly enter their password on five successive attempts, you can use the Locked check box to reset it. Password security is fully described in Section 9.17.4, "Enforcing Password Security Policies". You can use this check box to unlock the user's account. When ready, click Next. The dialog shown in Figure 9-30 appears.

Language: this is the language in which system messages and prompts appear. Currently, only English is available.

Mailingtype: specifies whether the reports the user receives are sent in multiple e-mails (one for each report) or bundled into a single e-mail. The default is multiple e-mails.

Startup module: specifies the module in which the user starts their session. (For example, Reports, System, or User management). The default is the dashboard (described in Section 1.5, "Working with the Dashboard").

When ready, click Next. A dialog similar to the one shown in Figure 9-27 appears.

In the event that you need to reset the admin user password, you can do so with the use of the set-admin-password script. This is described in the Oracle Real User Experience Insight Installation Guide.

9.17.4 Enforcing Password Security Policies

Each user must be defined and authorized to work with RUEI. The procedure to do this is fully explained in Section 9.17, "Managing Users and Permissions". In order to optimize the security of your installation, you can use the password settings facility to enforce your organization's security policies. Specifically, you can control the maximum length of user passwords, and how often users are required to change their passwords.

To control your installation's password enforcement, do the following:

Use the Password length field to specify the minimum number of characters that user passwords must contain. The minimum length is eight characters, and the maximum length is 255 characters.

Use the Expiration period field to specify how often users are required to change their passwords. The default is 60 days. If set to 0, passwords will never expire. The maximum expiration period is 999 days. When ready, click Save.

Password Enforcement

When creating and authorizing users, the following rules are automatically enforced:

Newly created users must change their passwords with seven days. Otherwise, their accounts are locked.

User accounts are locked after five failed attempts. The account must be unlocked before the user can logon again (described in Section 9.17.3, "Modifying a User's Settings"). However, locked users will continue to receive mailed reports and alerts.

If a password's expiration period is set to 0, and later re-set to a non-zero value (or vice versa), all existing user accounts will adapt to the newly specified password expiration period.

A user password must have a minimum of eight characters. It must contain at least one non-alphanumeric characters (such as $, @, &, and !).

A password cannot include the defined user name, or their first and last name. In addition, the user's last three passwords are also remembered, and cannot be re-used.

Passwords are case sensitive.

9.17.5 Configuring LDAP Server User Authentication

In order to provide enhanced security, RUEI can be configured to enable user authentication via an LDAP server, rather than through the settings held locally on your RUEI installation. If an LDAP server connection has been configured, you can specify the authentication method to be used for each defined user. Note because the Administrator user is predefined, and their password is set during initial configuration (see the Oracle Real User Experience Insight Installation Guide), only local authentication is available for this user.

If you plan to use LDAP authentication, it is recommended that you define your LDAP connection before the creation of user accounts. This is in order to prevent having to modify previously specified user settings.

Configuring the LDAP Server Connection

To enable LDAP server authentication, do the following:

Select System, then User Management, and then click Configure LDAP connection. Note that if an LDAP server connection has already been configured, the option is indicated as Modify LDAP connection. The dialog shown in Figure 9-32 appears.

Use the Allow LDAP authentication check box to specify whether an LDAP server is available for user authentication. The default is unchecked (disabled).

Use the Server name field to specify the host name or IP address of the LDAP server to be used. Note that protocol information (such as LDAP://) should be omitted from the server name.

Use the Connection type menu to specify the LDAP version and connection method. The default is V2 (non-secure).

Use the Port number field to specify the port to which the LDAP server is listening. If necessary, discuss this with your System Administrator. The default port is 389 or 636 (for SSL encryption).

Use the Search base field to specify the location in the directory structure within which the user ID needs to be unique. This must be a valid DN. For performance reasons, this should be as specific as possible. The default is the root of the directory tree.

Use the Anonymous check box to specify if the LDAP server lookup should be performed using an anonymous user. If unchecked, then a valid Distinguished Name (DN) must be specified, and the password for that user is requested when a new user is created. The default is to use an anonymous lookup.

Use the User ID, Email address, and Full name fields to specify the attributes that should be used to extract user settings from the LDAP server. The defaults are based on standard LDAP functionality. If necessary, you should discuss these attributes with your LDAP administrator.

Optionally, you can click Test to verify whether a working connection to the LDAP server can be made. This is discussed in the following section.

When ready, click Save.

Any changes you specify to the LDAP configuration settings take effect immediately.

Testing the LDAP Server

As mentioned earlier, you can test the connection to the LDAP server. Do the following:

Use the User ID to look up field to specify the user ID that should be searched for. This should be a valid user ID. When ready, click Test. Upon successfully finding the specified user's entry in the directory, their retrieved details are displayed. When ready, click Cancel. You are returned to the dialog shown in Figure 9-32.

9.18 Exporting Enriched Data

The Enriched data exchange facility enables the alternative analysis of the data collected by RUEI. In particular, it allows you to combine the data collected by RUEI with other data warehouse data. For example, a Customer Relationship Management (CRM) or Business Intelligence (BI) system. Using this facility, you can extract a rich set of collected data, such as product names, shopping basket values, and address information. The external tools should be aware the data is in Unicode (UTF-8) format.

While the facility described in Section 2.11, "Exporting Report Data" is limited to report data, the enriched data exchange facility allows the export of all page-based data. In addition, report data export is based on HTTPS transfer, and enriched data exchange is based on SFTP file transfer. As described later, you can also customize the content of the exported data to include header information not normally collected by RUEI. Because the exported data is page-based, the available data is restricted to applications, and does not include service-related data.

Example BI Implementation Using Enriched Data Exchange

This section presents an outline of a BI solution utilizing data from the Enriched data exchange facility. It makes use of Oracle Business Intelligence foundation (part of the Oracle Fusion Middleware product family). Its schematic structure is shown in Figure 9-34.

The framework is based on Oracle Warehouse Builder (OWB). The RUEI-captured data is uploaded to a load database. This, via a staging database, then populates the production database. Once in the production DWH, the RUEI data is available through a wide variety of reports and dashboards. An example of these reports is shown in Figure 9-35.

Use the Enabled check box to enable or disable the Enriched data exchange facility. By default, it is enabled.

Optionally, you can define additional data items to be included in the exported data. Typically, these are elements in the client request or server response headers that are not normally collected by RUEI, but which you want included in the exported data. To do so, click «Add new item». The dialog shown in Figure 9-37 appears.

Use the Search type menu to define how the required item should be identified within the data collected by RUEI, and the scope of the search. You can specify to search within the client request header or server response header, using either a literal search or an XPath expression, or to search within a custom page-tagging implementation for a specific tag. Further information about support for custom page-tagging schemes is available in Appendix A, "Tagging Conventions".

Use the Source value field to specify the specific argument or element from which the data item's value should be taken.

Use the Export name field to specify the name to be assigned to the data item. This becomes the item's element name. For example, if specify the name "product", any matched data will appear in the export file with the label <product>. Note this field is not available if you select a header-related option in the Source type menu.

When ready, click Save. The new data item, if found in the monitored traffic, will start to be reported in the export files within 5 to 10 minutes.

Existing data items can be modified by right-clicking them within Figure 9-36, and selecting Edit. You can also select Remove to delete it, or select Removeall to delete all currently defined items.

The exported data is based on pageviews, and is in XML format. This enables its immediate importation into a wide variety of systems. An XSD file defines the structure of the exported XML. The XML schema is shown in Figure 9-38:

When enabled, the enriched data export facility creates an export file every five minutes. The files are Unicode (UTF-8) encoded. The files are located in the directory /var/opt/ruei/processor/xml-events/wg/xml-sespage. Each file within this directory has the following name structure:

yyyymmdd-hhmmss-nnnn[L|M].xml.gz

Where:

nnnn represents the file sequence. Because an export file is created every five minutes, 288 files can be created per day. This can range from 0001 to 0289.

L indicates that it is the last file for that day. This always has the file sequence 289, and is used to gather up any open sessions after the 24 hour period.

M indicates that more files are still to follow this file.

By default, exports are retained for a period of seven days before they are automatically deleted. However, this can be configured, as fully explained in Section 9.5, "Configuring Database and Disk Space Limits and Alerts". In order to access these files, you will need a working FTP file transfer connection to the Reporter system. Consult your System Administrator for further information on this facility.

If required, you can use a symbolic link definition to change the location to which files are exported. Consult your System Administrator for further information on the use of this facility.

Security Considerations

While access to the data generated by the Enriched data exchange facility can be controlled in several different ways at the operating system level, it is recommended that you use SCP/SFTP and create a separate OS user with minimal access rights to the directory containing the exported data. You can then use an scp command to copy the data to a local system. For example: