The script content on this page is for navigation purposes only and does not alter the content in any way.

11 Configuring Instrumentation

The Instrumentation component of the WebLogic Diagnostics Framework (WLDF) provides a mechanism for adding diagnostic code to WebLogic Server instances and the applications running on them. The key features provided by WLDF Instrumentation are:

Diagnostic monitors. A diagnostic monitor is a dynamically manageable unit of diagnostic code that is inserted into server or application code at specific locations. You define monitors by scope (system or application) and type (standard, delegating, or custom).

Diagnostic actions. A diagnostic action is the action a monitor takes when it is triggered during program execution.

Diagnostic context. A diagnostic context is contextual information, such as unique request identifier and flags that indicate the presence of certain request properties, such as the originating IP address or user identity. The diagnostic context provides a means for tracking program execution and for controlling the moment when monitors trigger their diagnostic actions. See Chapter 12, "Configuring the DyeInjection Monitor to Manage Diagnostic Contexts."

WLDF provides a library of predefined diagnostic monitors and actions. You can also create application-scoped custom monitors in which you control the locations in the application where diagnostic code is inserted.

Instrumentation Scope

You can provide instrumentation services at the system level (servers and clusters) and at the application level. Many concepts, services, configuration options, and implementation features are the same for both levels. However, there are differences, which are discussed throughout this document. The term server-scoped instrumentation refers to instrumentation configuration and features specific to WebLogic Server instances and clusters. By contrast, application-scoped instrumentation refers to configuration and features specific to applications deployed on WebLogic servers. The scope is built in to each diagnostic monitor; you cannot modify a monitor's scope.

Configuration and Deployment

Server-scoped instrumentation for a server or cluster is configured and deployed as part of a diagnostic module, an XML configuration file located in the DOMAIN_NAME/config/diagnostics directory, and linked from config.xml.

Application-scoped instrumentation is also configured and deployed as a diagnostics module, in this case an XML configuration file named weblogic-diagnostics.xml, which is packaged with the application archive in the ARCHIVE_PATH/META-INF directory for the deployed application.

Joinpoints, Pointcuts, and Diagnostic Locations

Instrumentation code is inserted (or woven) into server and application code at precise locations. The following terms are used to describe these locations:

A joinpoint is a specific location in a class; for example, the entry point, or exit point, or both, of a method or a call site within a method.

A pointcut is an expression that specifies a set of joinpoints, for example all methods related to scheduling, starting, and executing work items. The XML element that specifies a pointcut is <pointcut>. Pointcuts are described in Defining Pointcuts for Custom Monitors.

A diagnostic location is the position relative to a joinpoint where the diagnostic activity will take place. Diagnostic locations are Before, After, and Around. The XML element that identifies a diagnostic location is <location-type>.

Diagnostic Monitor Types

A diagnostic monitor is categorized by its scope and its type. The scope is either server-scoped or application-scoped. The type is determined by the monitor's pointcut, diagnostic location, and actions. For example, Servlet_Around_Service is an application-scoped delegating monitor that can be used to trigger diagnostic actions at the entry to and exit from specific servlet and JSP methods.

There are three types of diagnostic monitors:

A standard monitor performs specific, predefined diagnostic actions at specific, predefined pointcuts and locations. These actions, pointcuts, and locations are hard-coded in the monitor. You can enable or disable the monitor, but you cannot modify its behavior.

The only standard application-scoped monitor is HttpSessionDebug, which you can use to inspect an HTTP Session object.

A delegating monitor has its scope, pointcuts, and locations hard-coded in the monitor, but you select the actions that the monitor performs. That is, the monitor delegates its actions to the ones you select. Delegating monitors are either server-scoped or application-scoped.

A delegating monitor by itself is incomplete. To have a delegating monitor perform useful work, you must assign at least one action to it.

Not all actions are compatible with all monitors. When you configure a delegating monitor from the Administration Console, you can choose only those actions that are appropriate for the selected monitor. If you configure a delegating monitor using WLST or by editing a descriptor file manually, you must make sure that the actions are compatible with that monitor. WLDF validates a delegating monitor when its XML configuration file is loaded at deployment time.

To configure a custom monitor, you assign it a name, define the pointcut and the diagnostics location that the monitor uses, and assign actions from the set of predefined diagnostic actions. The <pointcut> and <location type> elements are mandatory for a custom monitor.

You can restrict when a diagnostic action is triggered by setting a dye mask on a monitor. This mask determines the dye flags in the diagnostic context that trigger actions. See <wldf-instrumentation-monitor> XML Elements, for information about setting a dye mask for a monitor.

Diagnostic Actions

Diagnostic actions execute diagnostic code that is appropriate for the associated delegating or custom monitor (standard monitors have predefined actions). For a delegating or custom monitor to perform any useful work, you must configure at least one action for that monitor.

The WLDF diagnostics library provides the following actions, which you can attach to a monitor by including the action's name in an <action> element of the DIAG_MODULE.xml configuration file:

DisplayArgumentsAction

MethodInvocationStatisticsAction

MethodMemoryAllocationStatisticsAction

StackDumpAction

ThreadDumpAction

TraceAction

TraceElapsedTimeAction

TraceMemoryAllocationAction

Actions must be correctly matched with monitors. For example, the TraceElapsedTime action is compatible with a delegating or custom monitor whose diagnostic location type is Around. See Appendix B, "WLDF Instrumentation Library," for more information.

Instrumentation Configuration Files

Instrumentation is configured as part of a diagnostics descriptor, which is an XML configuration file whose name and location depend on whether you are implementing system-level (server-scoped) or application-level (application-scoped) instrumentation, as follows:

System-level instrumentation configuration is stored in one or more diagnostics descriptors in the following directory:

DOMAIN_NAME/config/diagnostics

This directory can contain multiple system-level diagnostic descriptor files. File names are arbitrary but must be terminated with .xml; for example, myDiag.xml. Each file can contain configuration information for one or more of the following deployable diagnostic components:

Harvester

Instrumentation

Watch and Notification

The configuration of one or more diagnostic monitors can be defined in an <instrumentation> section in the descriptor file. Server-scoped instrumentation can be enabled, disabled, and reconfigured without restarting the server.

Only one WLDF system resource (and hence one system-level diagnostics descriptor file) can be active for a server or cluster at any given time. The active descriptor is linked to and targeted from the following configuration file:

Application-level instrumentation configuration is packaged within an application's archive in the following location:

META-INF/weblogic-diagnostics.xml

Because instrumentation is the only diagnostics component that is deployable to applications, this descriptor can contain only instrumentation configuration information.

Note:

For instrumentation to be available for an application, instrumentation must be enabled on the server on which the application is deployed. (Server-scoped instrumentation is enabled and disabled in the <instrumentation> element of the diagnostics descriptor for the server.

You can enable and disable diagnostic monitors without redeploying an application. However, you may need to redeploy the application after modifying other instrumentation features; for example, defining pointcuts or adding or removing monitors. Whether you need to redeploy depends on how you configure the instrumentation and how you deploy the application. There are three options:

Define and change the instrumentation configuration for the application directly, without using a JSR-88 deployment plan.

Configure and deploy the application using a deployment plan that has placeholders for instrumentation settings.

Enable the HotSwap feature when starting the server, and deploy using a deployment plan that has placeholders for instrumentation settings.

If true, instrumentation is enabled. If false, no instrumented code is inserted in classes in this instrumentation scope, and all diagnostic monitors within this scope are disabled. The default value is false.

You must enable instrumentation at the server level to enable instrumentation for the server and for any applications deployed to it. You must further enable instrumentation at the application level to enable instrumentation for the application (that is, in addition to enabling the server-scoped instrumentation).

<include>

An optional element specifying the list of classes where instrumented code can be inserted. Wildcards (*) are supported. You can specify multiple <include> elements. If specified, a class must satisfy an <include> pattern for it to be instrumented.

Applies only to application-scoped instrumentation. Any specified <include> or <exclude> patterns are applied to the application scope as a whole.

Note: You can also specify <include> and <exclude> patterns for specific diagnostic monitors. See the entries for <include> and <exclude> in Table 11-1.

As classes are loaded, they must pass an include/exclude pattern check before any instrumentation code is inserted. Even if a class passes the include/exclude pattern checks, whether or not it is instrumented depends on the diagnostic monitors included in the configuration descriptor. An application-scoped delegating monitor from the library has its own predefined classes and pointcuts. A custom monitor specifies its own pointcut expression. Therefore, a class can pass the include/exclude checks and still not be instrumented.

Note: Instrumentation is inserted in applications at class load time. A large application that is loaded often may benefit from a judicious use of <include> elements, <exclude> elements, or both. You can probably ignore these elements for small applications or for medium-to-large applications that are loaded infrequently.

<exclude>

An optional element specifying the list of classes where instrumented code cannot be inserted. Wildcards (*) are supported. You can specify multiple <exclude> elements. If specified, classes satisfying an <exclude> pattern are not instrumented.

Applies only to application-scoped instrumentation. See the preceding description of the <include> element.

<wldf-instrumentation-monitor> XML Elements

Diagnostic monitors are defined in <wldf-instrumentation-monitor> elements, which are children of the <instrumentation> element in the following descriptor:

The DIAG_MODULE.xml descriptor for server-scoped instrumentation

The META-INF/weblogic-diagnostics.xml descriptor for application-scoped instrumentation

The following fragment shows the configuration for a delegating monitor and a custom monitor in an application. (You could modify this fragment for server-scoped instrumentation by replacing the application-scoped monitors with server-scoped monitors.)

If true, the monitor is enabled. If false, the monitor is disabled. You enable or disable each monitor separately. The default value is true.

<name>

The name of the monitor. For standard and delegating monitors, use the names of the predefined monitors in Appendix B, "WLDF Instrumentation Library," For custom monitors, an arbitrary string that identifies the monitor. The name for a custom monitor must be unique; that is, it cannot duplicate the name of any monitor in the library.

<description>

An optional element describing the monitor.

<action>

An optional element, which applies to delegating and custom monitors. If you do not specify at least one action, the monitor will not generate any information. You can specify multiple <action> elements. An action must be compatible with the monitor type. For the list of predefined actions for use by delegating and custom monitors, see Appendix B, "WLDF Instrumentation Library."

<dye-filtering-enabled>

An optional element. If true, dye filtering is enabled for the monitor. If false, dye-filtering is disabled. The default value is false.

In order to use dye filtering, the DyeInjection monitor must be configured appropriately at the server level.

An optional element, whose value is one of before, after, or around. The location type determines when an action is triggered at a pointcut: before the pointcut, after the pointcut, or both before and after the pointcut.

Applies only to custom monitors; standard and delegating monitors have predefined location types. A custom monitor must define a location type and a pointcut.

<pointcut>

An optional element. A pointcut element contains an expression that defines joinpoints where diagnostic code will be inserted.

Applies only to custom monitors; standard and delegating monitors have predefined pointcuts. A custom monitor must define a location type and a pointcut.

An optional element specifying the list of classes where instrumented code can be inserted. Wildcards (*) are supported. You can specify multiple <include> elements. If specified, a class must satisfy an <include> pattern for it to be instrumented.

Applies only to application-scoped instrumentation. Any specified <include> or <exclude> patterns are applied only to the monitor defined in the parent <wldf-instrumentation-monitor> element.

Note: You can also specify <include> and <exclude> patterns for an entire instrumented application scope. See the entries for <include> and <exclude> in Table 11-1.

As classes are loaded, they must pass an include/exclude pattern check before any instrumentation code is inserted. Even if a class passes the include/exclude pattern checks, whether or not it is instrumented depends on the diagnostic monitors included in the configuration descriptor. An application-scoped delegating monitor from the library has its own predefined classes and pointcuts. A custom monitor specifies its own pointcut expression. Therefore a class can pass the include/exclude checks and still not be instrumented.

Note: Instrumentation is inserted in applications at class load time. A large application that is loaded often may benefit from a judicious use of <include> and/or <exclude> elements. You can probably ignore these elements for small applications or for medium-to-large applications that are loaded infrequently.

<exclude>

An optional element specifying the list of classes where instrumented code cannot be inserted. Wildcards (*) are supported. You can specify multiple <exclude> elements. If specified, classes satisfying an <exclude> pattern are not instrumented.

Applies only to diagnostic monitors in application-scoped instrumentation. See the <include> description, above.

Note the following additional information about the <dye-filtering-enabled> and <dye-mask> elements:

When a DyeInjection monitor is enabled and configured for a server or a cluster, you can use dye filtering in downstream delegating and custom monitors to inspect the dyes injected into a request's diagnostic context by that DyeInjection monitor.

The configuration of the DyeInjection monitor determines which bits are set in the 64-bit dye vector associated with a diagnostic context. When the <dye-filtering-enabled> attribute is enabled for a monitor, its diagnostic activity is suppressed if the dye vector in a request's diagnostic context does not match the monitor's configured dye mask. If the dye vector matches the dye mask (a bitwise AND), the application can execute its diagnostic actions:

Footnote 1 Currently used only by the DyeInjection monitor to set name=value pairs for dye flags.

Configuring Server-Scoped Instrumentation

To enable instrumentation at the server level, and to configure server-scoped monitors, perform the following steps:

Decide how many WLDF system resources you want to create.

You can have multiple DIAG_MODULE.xml diagnostic descriptor files in a domain. But for each server or cluster in a domain, you can deploy only one diagnostic descriptor file at any given time. One reason for creating more than one file is for flexibility. For example, you could have five diagnostic descriptor files in the DOMAIN_NAME/config/diagnostics directory. Each file contains a different instrumentation (and perhaps Harvester and Watch and Notification) configuration. You then deploy the descriptor file that corresponds to the particular monitors you want active.

Decide which server-scoped monitors you want to include in a configuration:

If you plan to use dye filtering on a server, or on any applications deployed on that server, configure the DyeInjection monitor.

If you plan to use one or more of the server-scoped delegating monitors, decide which monitors to use and which actions to associate with each monitor.

Create and configure the configuration file(s).

If you use the Administration Console to create the DIAG_MODULE.xml file (recommended), for delegating monitors the console displays only the actions that are compatible with the monitor. If you create a configuration file with an editor or with the WebLogic Scripting Tool (WLST), you must correctly match actions to monitors.

See the "Domain Configuration Files" in Understanding Domain Configuration for Oracle WebLogic Server for information about configuring config.xml.

Validate and deploy the descriptor file. For server-scoped instrumentation, you can add and remove monitors and enable or disable monitors while the server is running.

Example 11-1 contains a sample server-scoped instrumentation configuration file that enables instrumentation and configures the DyeInjection standard monitor and the Connector_Before_Work delegating monitor. A single <instrumentation> element contains all instrumentation configuration for the module. Each diagnostic monitor is defined in a separate <wldf-instrumentation-monitor> element.

Comparing System-Scoped to Application-Scoped Instrumentation

Instrumenting an application is similar to instrumenting at the system level, but with the following differences:

Applications can use standard, delegating, and custom monitors.

The only server-scoped standard monitor is DyeInjection. The only application-scoped standard monitor is HttpSessionDebug. For more information, see the entry for HttpSessionDebug in Diagnostic Monitor Library.

Delegating monitors are either server-scoped or application-scoped. Applications must use the application-scoped delegating monitors.

All custom monitors are application-scoped.

The server's instrumentation settings affect the application. In order to enable instrumentation for an application, instrumentation must be enabled for the server on which the application is deployed. If server instrumentation is enabled at the time of deployment, instrumentation will be available for the application. If instrumentation is not enabled on the server at the time of deployment, enabling instrumentation in an application will have no effect.

Application instrumentation is configured with a weblogic-diagnostics.xml descriptor file. You create a META-INF/weblogic-diagnostics.xml file, configure the instrumentation, and put the file in the application's archive. When the archive is deployed, the instrumentation is automatically inserted when the application is loaded.

The XML descriptors for application-scoped instrumentation are defined in the same way as for server-scoped instrumentation. You can configure instrumentation for an application solely by using the delegating monitors and diagnostic actions available in the WLDF Instrumentation Library. You can also create your own custom monitors; however, the diagnostic actions that you attach to these monitors must be taken from the WLDF Instrumentation Library.

Table 11-5 compares the function and scope of system and application diagnostic modules.

Table 11-5 Comparing System and Application Modules

Module Type

Add or Remove Objects Dynamically

Add or Remove Objects with Console

Modify with JMX Remotely

Modify with JSR-88 (non-remote)

Modify with Console

Enable/Disable Dye Filtering and Dye Mask Dynamically

System Module

Yes

Yes

Yes

No

Yes

(via JMX)

Yes

Application Module

Yes, when HotSwap is enabled

No, when HotSwap is not enabled: module must be redeployed

Yes

No

Yes

Yes

(via plan)

Yes

Overview of the Steps Required to Instrument an Application

Note:

In WLS 10.3, you are not required to create a weblogic-diagnostics.xml file in the application's META-INF directory, as was the case in previous WLS releases. However, you can still use this method to initially configure diagnostic monitors for your application.

To implement a diagnostic monitor for an application, perform the following steps:

Create a well formed META-INF/weblogic-diagnostics.xml descriptor file for the application. If you want to add any monitors that will be automatically enabled each time the application is deployed:

Enable the <instrumentation> element: <enabled>true</enabled>.

Add and enable at least one diagnostic monitor, with appropriate actions attached to it. (A monitor will generate diagnostic events only if the monitor is enabled and actions that generate events are attached to it.).

The diagnostic monitors defined in weblogic-diagnostics.xml will be listed on the Deployments: <server_name>: Configuration: Instrumentation page of the Administration Console.

If the META-INF/weblogic-diagnostics.xml descriptor in the application archive defines a monitor, it can't be removed using the Administration Console. However, it can be disabled or enabled using the Administration Console.

You can add additional monitors from the Administration Console. Any monitors you add from the Administration Console will not be persisted to weblogic-diagnostics.xml; they will be saved in the application's deployment plan. Any monitors that were added in this way can be deleted using the Administration Console.

Creating a Descriptor File for a Delegating Monitor

The following example shows a well-formed META-INF/weblogic-diagnostics.xml descriptor file for an application-scoped delegating monitor. At a minimum, this file must contain the lines shown in bold. In this example, there is only one monitor defined (Servlet_Before_Service). However, you can define multiple monitors in the descriptor file.

The Servlet_Before_Service monitor is an application-scoped monitor selected from the WLDF monitor library. It is hard coded with a pointcut that sets joinpoints at method entry for several servlet or JSP methods. Because the application enables dye filtering and sets the USER1 flag in its dye mask, the TraceAction action will be invoked only when the dye vector in the diagnostic context passed to the application also has its USER1 flag set.

The dye vector is set at the system level via the DyeInjection monitor as per the DyeInjection monitor configuration when the request enters the server. For example, if the DyeInjection monitor is configured with property USER1=weblogic and the request was originated by user weblogic, the USER1 dye flag in the dye vector will be set.

Therefore, the Servlet_Before_Service monitor in this application is essentially quiescent until it inspects a dye vector and finds the USER1 flag set. This filtering reduces the amount of diagnostic data generated, and ensures that the generated data is of interest to the administrator.

Creating a Descriptor File for a Custom Monitor

The following is an example of a well-formed META-INF/weblogic-diagnostics.xml file for a custom monitor. At a minimum, the file must contain the lines shown in bold.

The <name> for a custom monitor is an arbitrary string chosen by the developer. Because this monitor is custom, it has no predefined locations when actions should be invoked; the descriptor file must define the location type and pointcut expression. In this example, the TraceAction action will be invoked before (<location-type>before</location-type) any methods defined by the pointcut expression is invoked. Table 11-1 shows how the pointcut expression from Example 11-2 is parsed. (Note the use of wildcards.)

Table 11-6 Description of a Sample Pointcut Expression

Pointcut Expression

Description

call( * com.foo.bar.* get* (...))

call( ): Trigger any defined actions when the methods whose joinpoints are defined by the remainder of this pointcut expression are invoked.

call( * com.foo.bar.* get* (...))

*: Return value. The wildcard indicates that the methods can have any type of return value.

call( * com.foo.bar.* get* (...))

com.foo.bar.*: Methods from class com.foo.bar and its sub-packages are eligible.

call( * com.foo.bar.* get* (...))

get*: Any methods whose name starts with the string get is eligible.

call( * com.foo.bar.* get* (...))

(...): The ellipsis indicates that the methods can have any number of arguments.

This pointcut expression matches all get*() methods in all classes in package com.foo.bar and its sub-packages. The methods can return values of any type, including void, and can have any number of arguments of any type. Instrumentation code will be inserted before these methods are called, and, just before those methods are called, the TraceAction action will be invoked.

Defining Pointcuts for Custom Monitors

Custom monitors provide more flexibility than delegating monitors because you create pointcut expressions to control where diagnostics actions are invoked. As with delegating monitors, you must select actions from the action library.

A joinpoint is a specific, well-defined location in a program. A pointcut is an expression that specifies a set of joinpoints. This section describes how you define expressions for pointcuts using the following pointcut syntax.

An ellipsis (...) in the argument list signifies a variable number of arguments of any types beyond the argument.

A percent character (%) prefix designates the value of a non-static class instantiation, parameter, or return specification as not containing or exposing sensitive information. The use of this operator is particularly useful with the DisplayArgumentsAction action, which captures method arguments or return values. If this prefix character is not explicitly used, an asterisk string is substituted for the value that is returned; this behavior ensures that sensitive data in your application is not inadvertently transmitted when an instrumentation event captures input arguments to, or return values from, a joinpoint.

Note:

The % operator cannot be applied to an ellipsis or to a wildcarded type within a pointcut expression.

A pointcut expression specifies a pattern to identify matching joinpoints. An attempt to match a joinpoint against it will return a boolean, indicating a valid match (or not).

Pointcut expressions can be combined with AND, OR and NOT boolean operators to build complex pointcut expression trees.

For example, the following pointcut matches method executions of all public initialize methods in all classes in package com.foo.bar and its sub-packages. The initialize methods may return values of any type, including void, and may have any number of arguments of any types.

execution(public * com.foo.bar.* initialize(...))

The following pointcut matches the method calls (call sites) on all classes that directly or indirectly implement the com.foo.bar.MyInterface interface (or a subclass, if it happens to be a class). The method names must start with get, be public, and return an int value. The method must accept exactly one argument of type java.lang.String:

call(int +com.foo.bar.MyInterface get*(java.lang.String))

The following example shows how to use boolean operators to build a pointcut expression tree:

The preceding pointcut also matches the getter methods in class Foo$Bar.

Creating Request Performance Data

If you have configured server-scoped or application-scoped instrumentation, you can display request performance data in the WebLogic Server Administration Console. The Request Performance page displays information about the real-time and historical views of method performance that has been captured by means of the WebLogic Diagnostics Framework instrumentation capabilities.

To create request performance data, the following criteria must be met:

A WLDF system resource must be created and targeted to the server. Create the system resource as described in Instrumentation Configuration Files. You can do this using the WebLogic Server Administration Console or the WebLogic Scripting Tool (WLST).

Instrumentation in the targeted WLDF system resource must be enabled.

Application instrumentation must be enabled with a weblogic-diagnostics.xml descriptor, which you create in the application's META-INF directory, as described in Instrumentation Configuration Files.

Application instrumentation descriptors must use TraceElapsedTimeAction diagnostic actions attached to Around diagnostic monitor types. For example, a descriptor could contain the following:

WebLogic Server does not require the weblogic-diagnostics.xml descriptor to be pre-bundled in your application's archive in order to make instrumentation changes to a deployed application.

If your application uses a deployment plan, and you enable Java HotSwap before deploying your application, you can make instrumentation changes at run time without redeploying your application.

If your deployed application does not have a deployment plan and you modify the application's instrumentation configuration, the Administration Console automatically creates a deployment plan for you and prompts you for the location in which to save it.

If HotSwap is not enabled in your deployment plan, or if you do not use a deployment plan, changes to some instrumentation settings require redeployment.