WebSphere Application Server V7: Monitoring the Runtime

Transcription

1 Chapter 11 of WebSphere Application Server V7 Administration and Configuration Guide, SG WebSphere Application Server V7: Monitoring the Runtime Being able to measure and monitor system interactions helps IT in providing business continuity. Monitoring capabilities play a key role in successfully managing enterprise systems. In WebSphere Application Server, there are a number of tools that can contribute to an organizations monitoring strategy and provide insights into the performance of the application server. In this chapter, we provide an introduction to these toolsets. We cover the following topics: Overview on page 2 Enabling monitoring infrastructures on page 6 Viewing the monitoring data on page 22 Monitoring scenarios on page 33 ITCAM for WebSphere on page 47 Monitoring considerations summary on page 58 Copyright IBM Corp All rights reserved. 1

2 Overview IT environments are complex, involving many different servers working together to deliver the electronic functions of business. In a single user interaction, it is typical that information can be retrieved from many systems. Consider the very simple distributed WebSphere Application Server environment in Figure 1. The stars in the figure highlight that even a simple Web application request can pass through a whole series of dependent servers in order to successfully complete a request. JEE is a component based architecture, requiring a request to interact and use n number of these components to complete. Monitoring system components and their performance can become complex, yet is critical to understanding the overall performance of an application. Application Server Web Container EJB Container Web Services Engine Messaging Engine Security JCA Client Browser Firewall HTTP Server JMX Name Services Database Request Metrics PMI EIS LDAP Directory Figure 1 Simple Web system topology Monitoring the systems contributes to overall systems management by: Establishing an understanding of the performance baseline and of what runtime behaviors constitute normal operations 2 WebSphere Application Server V7: Monitoring the Runtime

3 Measuring performance and identifying poorly performing systems and components Identifying service failures, and can assist in root cause identification Monitoring scenarios WebSphere Application Server monitoring tools rely primarily on information gathered from two core data infrastructures: Performance Monitoring Infrastructure (PMI), which is a collection of statistical agents scattered through out the application server that gather statistical data on the performance of the application server components. websphere.nd.multiplatform.doc/info/ae/ae/cprf_pmidata.html Request metrics, which are primarily a set of timing agents that track a request as it navigates the components of the application server. A key differentiation of request metrics is that they are measured at the request level. The focus of a request metric is to record the time spent by individual requests in different components of the application and at the end of the request provide a record of where time was spent in the request. websphere.base.doc/info/aes/ae/tprf_requestmetrics.html This chapter demonstrates the use of the system monitoring infrastructures by using different monitoring scenarios, which are summarized in Table 1. Note: These scenarios cover several common uses of the monitoring tools, but it should be understood that many of the different types of data are not explicitly discussed. This chapter provides an introduction to the tool sets and a way for new administrators to get started, and serves as a reminder for experienced administrators about some of the tools that they might not have utilized in some time. Table 1 Monitoring Scenario Summary Monitoring Scenario How is monitoring data activated and what are the monitoring choices? PMI data defaults Enabling request metrics Chapter Reference Enabling monitoring infrastructures on page 6 PMI defaults and monitoring settings on page 6 Enable request metrics on page 15 WebSphere Application Server V7: Monitoring the Runtime 3

4 Monitoring Scenario What are the tools that I can use for understanding the collected data? Tivoli Performance Viewer. Understanding how application(s) are interacting with a database. This scenario helps identify data that will help investigate: Is the database responding fast enough? Is there enough connections to the database? Are the connections being returned to the pool? Understanding JCA connection pool utilization. This scenario helps administrators understand: What is the response like for the JCA connection pools. Are there enough connections to support the system interactions? Are the connections being returned to the pool? What about threading resources. Is there sufficient JMS, EJB, and Web threads allocated in the server thread pools? This scenario helps administrator understand: How to monitor and improve system related throughput. Current limits on system concurrency. Monitoring memory allocation and garbage collection. JVM memory tuning is vital to application server performance, and this scenario introduces administrators to tools that assist in making memory tuning choices. Finding bottlenecks in response time, Services, EJB, Web and response times. Understanding of component response time helps in the identification of application bottleknecks and performance issues. Special features of ITCAM for WebSphere in WebSphere Application Server v7.0 Chapter Reference Viewing the monitoring data on page 22 Database interactions on page 34 Database interactions on page 34 JCA interactions on page 35 Threading resources on page 36 JVM memory usage on page 40 Request level details on page 41 ITCAM for WebSphere on page 47 4 WebSphere Application Server V7: Monitoring the Runtime

6 Note: No special tuning has been done in the test environment. The Trade performance application was simply installed into a base server configuration, with the exception of default memory was changed. Enabling monitoring infrastructures This section shows you how to enable the PMI monitoring infrastructure and the request metrics that provide the performance data. PMI defaults and monitoring settings The enabling of PMI data is managed on a server-by-server basis. In the administrative console, do the following steps: 1. Navigate to the Performance Monitoring Infrastructure (PMI) menu item in the Monitoring and Tuning navigation menu. 2. Select the link for server for that you want to manage the PMI controls for. In this example, sys4server3 is the server. Figure 3 shows the PMI configuration panel for the server. 6 WebSphere Application Server V7: Monitoring the Runtime

7 Figure 3 Default PMI settings On this panel it is worthy to note that: PMI is enabled by default. The default statistical set is the basic set. The PMI data can be changed at runtime using settings in the Runtime tab. Note: Disabling and enabling of PMI data requires a server restart. The enabling and disabling of PMI is not available on the Runtime tab. However, the monitoring level can be set to None in a server with PMI enabled using the Runtime tab. Understanding the sets of PMI statistics The PMI statistic sets represent a group of individual statistical agents. The types of statistics that PMI can collect are classified. WebSphere Application Server V7: Monitoring the Runtime 7

8 Information about these classes can be found in the WebSphere Information Center on the PMI data classification page at: sphere.nd.doc/info/ae/ae/rprf_dataclass.html Everyone working in system administration knows that every action executed in an environment has a cost. Monitoring is no different, and for PMI, the cost of monitoring is impacted primarily by two factors: 1. The amount of data that is monitored. 2. The overhead of individual performance metrics. Not all metrics have the same collection cost. With PMI, there are multiple sets of statistics that can be enabled as shown in Figure 3 on page 7. These sets of statistics are: None Basic Extended All Custom None and All are fairly self explanatory, so here we take a closer look at the options provided by Basic, Extended, and Custom. Basic statistic set The Basic statistic set is the default setting, The basic setting is configured with the intention of providing an overall understanding of application server health, including statistics as outlined in the JEE specification, as well as other common performance hotspots and key monitoring points for JEE applications. Later, we discuss how to determine the overhead and level of a statistic (see Getting more information about statistics sets on page 12). Figure 4 shows the list of PMI counters that are active for the basic PMI data level. For details on each counter, refer to Getting more information about statistics sets on page WebSphere Application Server V7: Monitoring the Runtime

11 For example, consider the counters activated for ServletSession if the extended data set is selected. The counters are: LiveCount LifeTime NoRoomForNewSessionCount ExternalReadTime ExteranlReadSize ExternalWriteTime ExteranlWriteSize The NoRoomForNewSessionCount counter only applies if the Allow overflow from the Web container session management was changed from its default value of true. This attribute is shown in Figure 6. Figure 6 Allow overflow default is true Similarly, the counters related to external session management only apply if session persistence is configured. Hence the activation of the extended session information does little for an application where the overflow option is not modified and session persistence is not configured. WebSphere Application Server V7: Monitoring the Runtime 11

12 With a custom metric approach, the administrator could choose to simply add LiveCount and LifeTime counters as well as perhaps choosing other metrics of interest, such as the TimeoutInvalidationCount, to measure how many sessions are being timed out rather than logged off. Tip for using custom PMI settings: The counters used for the basic set can be customized to form a baseline for the custom counter activation. They should be supplemented with additional counters that are relevant for the application types that are being deployed. Overhead of PMI The actual overhead of each statistic level varies depending on the particular applications on the server and load that is being executed by the server. In the WebSphere Information Center, each counter has a documented qualitative overhead level to indicate the type of overhead it will incur (see Getting more information about statistics sets on page 12). This is not intended to prevent administrators from using counters with high overhead. It is important to remember that the overhead is a relative measurement, and the administrator needs to balance the need for the data versus the overhead incurred to enable a particular counter temporarily or for the long term. Note: If running in a stand alone server configuration, it is important to remember that the data collection and statistical counters are all working in the same JVM as the applications. In this circumstance, the administrator should anticipate that the activation of additional PMI data can incur additional CPU and memory usage by the JVM. The approximate overhead of the PMI statistic sets are as follows: Basic overhead up to 2% Extended overhead up to 3% All overhead of up to 6% Custom will depend on the counters enabled but it is reasonable to expect somewhere between 2%-6% Getting more information about statistics sets The WebSphere Information Center has extensive information to assist the administrator in understanding exactly which metrics are set for a particular level, and to appreciate the potential overheads of using the statistics. If we consider the number of components that make up an application server as shown in Figure 7 on page 13, it should be no surprise that there are many PMI counters available to help monitor the application server. 12 WebSphere Application Server V7: Monitoring the Runtime

14 Figure 8 shows the links found at the bottom of this article, taking you to a page with more information about each counter. Figure 8 PMI counter types From the links to the counter classifications, it is possible to navigate and view the individual counters that each counter classification contains. The following topics in the information center can provide more information: General PMI data organization: websphere.express.doc/info/exp/ae/rprf_dataorg.html WebSphere Application Server supports the eclipse framework for extensible applications. A key part of this framework is the implementation of the Extension registry. These counters are only relevant when referring to extensible applications. For more information, see: websphere.base.iseries.doc/info/iseries/ae/cweb_extensions.html 14 WebSphere Application Server V7: Monitoring the Runtime

15 Service integration bus counters: websphere.express.doc/info/exp/ae/rprf_sibcounter.html Enable request metrics The enabling of request metrics is a cell wide configuration and when activated, it is activated for all servers in the cell. In the administrative console: 1. Navigate to the Request Metrics menu item in the Monitoring and Tuning navigation menu (Figure 9). WebSphere Application Server V7: Monitoring the Runtime 15

17 When configured, the servers must be restarted for request metrics to be enabled. The servers must also be stopped when disabling request metrics. Understanding component instrumentation and trace levels Trace levels and component instrumentation work together to determine if the request is instrumented. The component instrumentation levels are shown in Figure 10. Figure 10 Components to be instrumented If you select All, all components will be monitored based on trace level settings. If you select Custom, you can select the components to be monitored. Data will be collected from the components if the trace level also calls for the capturing of data from this component. Note: When a component is defined as an edge component, meaning the request enters or exits the application server through the component, then this component is instrumented even is it is not selected as part of the custom component listing. Working in conjunction with the component instrumentation levels is the trace level (Figure 11). WebSphere Application Server V7: Monitoring the Runtime 17

18 Figure 11 Request metric trace levels The following trace levels are possible: None: No instrumentation is generated. Hops: Generates instrumentation information about process boundaries only. When this setting is selected, you see the data at the application server level, not the level of individual components such as enterprise beans or servlets. Performance_debug: Generates the data at Hops level and the first level of the intra-process servlet and Enterprise JavaBeans (EJB) call (for example, when an inbound servlet forwards to a servlet and an inbound EJB calls another EJB). Other intra-process calls like naming and service integration bus (SIB) are not enabled at this level. Debug: Provides detailed instrumentation data, including response times for all intra-process calls. Note that requests to servlet filters will only be instrumented at this level. Note: Working with instrumentation and trace levels are further explored later in the chapter (see Request level details on page 41). Important: Request metrics are checked starting with the HTTP plug-in for some Web related settings. The HTTP-plug-in configuration must be regenerated and propagated after enabling request metrics. Using request metric filters One final way that can be used to control the request metric instrumentation is to use request metric filters. Filters provide a way to specifically target flows and components to reduce the overhead of broad monitoring and to also make it easier to analyze the captured data by reducing the amount data that is captured. It is important to understand, however, that filters are implemented as edge component filtering, not as intra-component processing, so an EJB filter will not be effective if the EJB is always invoked from a servlet. In this case it is the URI that needs filtering, not the EJB. Filters should be applied on edge components. 18 WebSphere Application Server V7: Monitoring the Runtime

19 Filters are configured by selecting the filters link from the Additional properties section of the Request metrics panel. This navigates the administrator to the Request metric filter panel shown in Figure 12. Figure 12 Request Metric Filter panel Using filters, fine grained controls can be applied to the different edge types of EJB, JMS, IP address, URI, and Web services. The first step is to specify the filter. An example of each type can be seen by navigating the administration console for that type of filter. For example, selecting the URI link in Figure 12 takes you to the URI panel shown in Figure 13. Figure 13 Uri panel The enable check box must be checked for the filters to be enabled. Then the filters will be used along with the component and trace level settings to determine which components are instrumented. WebSphere Application Server V7: Monitoring the Runtime 19

20 Note: Enabling filters requires an application server restart. Select the Filter Values link in the filter panel to add or edit filters. This is also where the default example filters are displayed (Figure 14). Figure 14 Default filter values displayed in filter value panel Each filter type has its own syntax that is appropriate for the type. For example, the EJB filter specifies a method class or package that sets the scope of the filtering. Figure 15 shows the example URI filer value supplied for EJB filters. Figure 15 EJB default filter values 20 WebSphere Application Server V7: Monitoring the Runtime

21 It is possible to use wildcard settings in the filters if desired, as shown in the second entry in Figure 15 on page 20. Tip: To enable/disable a filter it requires the restarting of a server. It is important to plan the component levels and filters that an application might require to minimize the need to stop and restart servers. Destination type considerations The final consideration when configuring the request metrics is where the metrics will be gathered. There are two types of supported destinations, Data can be logged with standard logs. In this configuration the instrumented components are logged to the SystemOut.log file. Data can also be collated in an Application Response Measurement (ARM) data collector. In this case, the data is normally then moved to a monitoring system for analysis and display (for example, using IBM Tivoli Monitoring Transaction Performance). Tip: When configuring ARM agents for use with the application server, follow the installation instructions provided with the specific agent. For more information about ARM agents, see: websphere.nd.doc/info/ae/ae/cprf_arm.html Both logging types can be activated at once. Writing to standard logs is not recommended as a long term monitoring strategy because the overhead can be higher than is desirable. Overhead of request metrics The overhead of request metrics can vary significantly based on the components being monitored and the complexity of the request execution within the monitored components. There are no specific metrics on what the overhead is, but it is reasonable to assume that request metrics on every request and component might incur more overhead than is desired. We recommend that an organization consider and plan carefully the interactions that it wants to monitor, then measure the specific overhead associated with configuring request metrics for these components. WebSphere Application Server V7: Monitoring the Runtime 21

22 Viewing the monitoring data WebSphere Application Server provides an interface for viewing the monitored data. The interface is the Tivoli Performance Viewer (TPV) found in the administrative console. Starting TPV monitoring and configuring settings To work with the TPV from the administrative console, follow these steps: 1. Navigate to the Tivoli Performance Viewer panel, by selecting Monitoring and Tuning Performance Viewer Current Activity. 2. Select the check-box of server(s) that are to be monitored and click the Start Monitoring button (Figure 16). Tip: If you are only starting monitoring on a single server, monitoring can be started by simply clicking the server link and navigating into the TPV viewer. Figure 16 Start Server monitoring After monitoring is started, a message will be returned in the messages section of the panel, and the Status column of the server will be updated to Monitored. 22 WebSphere Application Server V7: Monitoring the Runtime

23 3. The PMI data can only be observed one server at a time when using a single user session. Select the server name link to navigate to the Tivoli Performance Viewer panel (Figure 17). Figure 17 Tivoli Performance Viewer The default view for this panel shows the Servlet Summary Report panel indicating recent servlet activity, as well as the TPV tree navigation panel. Note: The different ways that data can be viewed is discussed in Exploring TPV data views on page 26. But before exploring the data, let us first take a look at the settings menu to examine the User and Logging settings. 4. The user settings are reached by expanding the Settings category and selecting User. This panel helps you control how much data is retained and how often data samples are taken in the live system (Figure 18). WebSphere Application Server V7: Monitoring the Runtime 23

24 Figure 18 User settings The user settings are very significant and can have a direct impact on the performance of the server. The two key configuration settings are in the Data Collection section of the panel: The refresh rate indicates the interval between data sampling. Higher frequency rates mean that the server will gather and report on statistics more frequently, adding load to the servers that are collecting data. The buffer size indicates the number of data points that are kept. More data points simply mean that the PMI data will require more memory. For a standalone server this means that PMI data at high frequency and high buffer re-initiation will need more processing time and more memory. In a distributed server model the load is shared, but some settings might still need to be tuned. The data is collected at the node level and stored in memory on the node agent. Thus if a node has many servers, the memory requirements of the node agent will need to be adjusted. Also in a distributed server configuration, the data is viewed from the deployment manager. Thus, to process the data, the deployment manager should have adequate memory and CPU resources also. Consider also that more than one administrator might be observing data at once. 24 WebSphere Application Server V7: Monitoring the Runtime

25 5. A powerful feature of TPV is the ability to record PMI data and then replay the data later in a different deployment manager as if it were in real time or with options to fast forward and rewind. Logging is started by simply clicking the Start Logging button shown in the TPV viewing panel (Figure 17 on page 23). However, before clicking this button, the Log settings must be configured. The Log settings are found by selecting Settings Log as shown in Figure 19. Figure 19 Log Settings Examining the log settings that are available, the administrator is faced with several configuration choices: Duration: The logging of PMI data has with it a certain amount of overhead (resulting from logging to a file, the buffering of data in memory, and disk usage for log storage...). It is not intended to be used as a long term production monitoring strategy. Thus when logging is enabled, it is configured to be disabled after a period of time. WebSphere Application Server V7: Monitoring the Runtime 25

26 PMI data logging used in short durations can be used to capture runtime characteristics that might need further investigation of for sharing with development and troubleshooting specialists. Maximum file size and Maximum number of historical files: The settings for maximum file size and the number kept that are appropriate for your environment will depend on two conditions: How much PMI data is enabled The data sampling frequency (as configured in the user settings). File name: The server name and the time at which the log is started is appended to the file name specified to help users identify a log file Log output format: The other configuration item of consequence is the format type. The default is XML but the binary format requires a smaller footprint on the disk. If logging larger amounts of data, the binary logging format might be more suitable. Exploring TPV data views Tivoli Performance Viewer has three primary types of data that can be viewed Summary Reports Performance Modules Advisors Summary reports The summary reports provide a general overview in a tabulated format of the current system performance. The reports that are available include: Servlets EJBs EJB Methods Connection Pool Thread Pool Servlet and EJB summary reports can be useful for identifying the application resources that are most busy, and to the extent that averages can be used, to identify candidates that might warrant further investigation as to their performance. Together with request metrics, results from the summary reports can help identify possible candidates for request metric filters. The information for Connection Pool and Thread Pool utilization, while useful, can also be easily determined by monitoring the metrics in the performance modules. 26 WebSphere Application Server V7: Monitoring the Runtime

WebSphere Server Administration Course Chapter 1. Java EE and WebSphere Overview Goals of Enterprise Applications What is Java? What is Java EE? The Java EE Specifications Role of Application Server What

E-mail Listeners 6 E-mail Formats You use the E-mail Listeners application to receive and process Service Requests and other types of tickets through e-mail in the form of e-mail messages. Using E- mail

Oracle WebLogic Server 11g Administration This course is designed to provide instruction and hands-on practice in installing and configuring Oracle WebLogic Server 11g. These tasks include starting and

Transaction Monitoring Version 8.1.3 for AIX, Linux, and Windows Reference IBM Note Before using this information and the product it supports, read the information in Notices. This edition applies to V8.1.3

Holistic Performance Analysis of J2EE Applications By Madhu Tanikella In order to identify and resolve performance problems of enterprise Java Applications and reduce the time-to-market, performance analysis

IBM WebSphere Application Server Network Deployment for Distributed Platforms, Version 8.5 Monitoring Note Before using this information, be sure to read the general information under Notices on page 291.

Managing SOA Course materials may not be reproduced in whole or in part without the prior written permission of IBM. 4.0.3 4.0.3 Unit objectives After completing this unit, you should be able to: Explain

White Paper IBM WEBSPHERE LOAD BALANCING SUPPORT FOR EMC DOCUMENTUM WDK/WEBTOP IN A CLUSTERED ENVIRONMENT Abstract This guide outlines the ideal way to successfully install and configure an IBM WebSphere

IBM WebSphere Application Server for z/os, Version 8.5 Monitoring SA32-1084-00 Note Before using this information, be sure to read the general information under Notices on page 285. Compilation date: June

Instrumentation Software Profiling Software Profiling Instrumentation of a program so that data related to runtime performance (e.g execution time, memory usage) is gathered for one or more pieces of the

Administering batch environments, Version 8.5 Administering batch environments SA32-1093-00 Note Before using this information, be sure to read the general information under Notices on page 261. Compilation

NetBeans Profiler Exploring the NetBeans Profiler From Installation to a Practical Profiling Example* Gregg Sporar* NetBeans Profiler is an optional feature of the NetBeans IDE. It is a powerful tool that

Key Benefits of Aqualogic Monitoring System Aqualogic Monitoring System Monitoring Experience Redefined Agent less monitoring saves time and ensures system availability Avoid additional time and cost on

This presentation will discuss the Scheduler Service available in IBM WebSphere Application Server V6. This service allows you to schedule time-dependent actions. WASv6_Scheduler.ppt Page 1 of 18 The goals

44 Working with WebSphere 4.0 This chapter is for developers who are familiar with WebSphere Application Enterprise Server, version 4.0 (WAS 4.0) and would like to deploy their applications using WAS 4.0.

Copyright IBM Corporation 2010 All rights reserved WebSphere Business Monitor V7.0 What this exercise is about... 2 Lab requirements... 2 What you should be able to do... 2 Introduction... 3 Part 1: Install

This course teaches system/application administrators to setup, configure and manage an Oracle WebLogic Application Server, its resources and environment and the Java EE Applications running on it. This

1358 CHAPTER 33 Logging and Debugging Customizing the Event Log The properties of an event log can be configured. In Event Viewer, the properties of a log are defined by general characteristics: log path,

Oracle WebLogic Foundation of Oracle Fusion Middleware Lawrence Manickam Toyork Systems Inc www.toyork.com http://ca.linkedin.com/in/lawrence143 History of WebLogic WebLogic Inc started in 1995 was a company

vcenter Operations Management Pack for SAP HANA Installation and Configuration Guide This document supports the version of each product listed and supports all subsequent versions until a new edition replaces

Code:1Z0-599 Titre: Oracle WebLogic Server 12c Essentials Version: Demo http://www.it-exams.fr/ QUESTION NO: 1 You deploy more than one application to the same WebLogic container. The security is set on

IBM WebSphere Application Server Network Deployment for Distributed Platforms, Version 8.5 Establishing highly available services for applications Note Before using this information, be sure to read the

WebLogic Server Course Following is the list of topics that will be covered during the course: Introduction to WebLogic What is Java? What is Java EE? The Java EE Architecture Enterprise JavaBeans Application

NetIQ AppManager for IBM WebSphere Application Server UNIX Management Guide March 2015 www.netiq.com/documentation Legal Notice THIS DOCUMENT AND THE SOFTWARE DESCRIBED IN THIS DOCUMENT ARE FURNISHED UNDER

Rational Application Developer Performance Tips Introduction This article contains a series of hints and tips that you can use to improve the performance of the Rational Application Developer. This article

SW5706 JVM Tools This presentation will act as an introduction to. 4.0 Page 1 of 15 for tuning and problem detection After completing this topic, you should be able to: Describe the main tools used for

Vendor: IBM Exam Code: 000-420 Exam Name: IBM InfoSphere MDM Server v9.0 Version: DEMO 1. As part of a maintenance team for an InfoSphere MDM Server implementation, you are investigating the "EndDate must