Tags

Symptoms: I saw so many transaction timeout stacks in my trace (below). In the meanwhile, I got thread hung and operation unavailable on the WebSphere Application Server (WAS) admin console: [10/8/15 15:45:04:969 IST] 0000003e ThreadMonitor W WSVR0605W: Thread "Default : 9" (000000a2) has been active for 629861 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. at java.net.SocketInputStream.socketRead0(Native... [More]

I've been working on some PMRs (Problem Records) lately for threads either completing slowly or being reported as "may be hung" in WebSphere Application Server and many of the products that run on it. A very common scenario is seeing threads in socketRead: For example: [9/15/14 9:32:13:492 PDT] 00000023 ThreadMonitor W WSVR0605W: Thread "WebContainer : 0" (00000021) has been active for 692481 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. at... [More]

A common cause of hung threads in WebSphere Application Server is when a thread sends a request to a backend server, such as a database, and waits an unexpectedly long time for a response. When this occurs, you will see socketRead() at the top of the thread's stack trace. For example: [9/8/15 19:19:27:702 EDT] 00000062 ThreadMonitor W WSVR0605W: Thread "WebContainer : 2301" (00001785) has been active for 1425013 milliseconds and may be hung. There is/are 21 thread(s) in total in the server that may be hung. at... [More]

This is the process of investigating cpu utilization for a JAVA™ application server in response to a need to resolve a performance problem or to determine a base line for the application. Specifically, this investigation applies to AIX, linux, Solaris, HPUX, UNIX® like systems, and Windows. This will uncover the amount of cpu resources reported in use, the amount of available cpu used by each process, and the amount of cpu used by threads in processes. We use this process very often in WebSphere Application Server Support, but... [More]

Introduction: During the course of WebSphere Application Server operation, a JVM may produce important diagnostic information in a variety of files and formats. In this blog entry, we will briefly examine each of these files and their purpose. A wealth of valuable Java™ virtual machine diagnostic data can be gathered by doing the following tasks: 1) JVM process standard error stream In WebSphere, JVM writes some errors messages on the standard error stream for the JVM process and the file name is... [More]

A wealth of valuable JVM diagnostic data can be gathered by doing the following tasks: Generate a Java™ thread dump. Enable logging of verbose garbage collection data for Server. Generate a Java heap dump. Generate a system core dump & snap trc file. In the overall process of problem determination, diagnostic data must be collected or generated, and the data must be analyzed. Various tools are available to help you analyze diagnostic data for solving problems. Below are the links for the analysis of javacore, verbosegc,... [More]

We've probably all see a hung JVM at one time or another and chances are you've figured this out in one of two was if you're dealing with WebSphere Application Server: 1. the users are complaining that the browser just “spins” and never returns a web page, or 2. you've noticed output in the WebSphere logs (SystemOut.log) that indicate potentially hung threads. For the purposes of this discussion, we'll focus on the latter method. WebSphere Application Server provides a feature that detects hung... [More]

If you're having some trouble with your Java™ application, be it JSE or JEE, you've probably been looking at javacores (also called thread dumps or java dumps). If not, you should be and that is a topic for another day. When looking at the javacores with all those threads in there, have you ever wanted to know things like: Which user is causing these threads to spawn? What were they doing in the application? When did that thread get created? Well want no longer! With a few lines of some very simple Java code... [More]