A threshold is a boundary of acceptable or normal business transaction performance. AppDynamics provides default thresholds against which it compares the performance of every business transaction. Each transaction is classified into what are called a user experience: normal, slow, very slow, stall, or error transaction.

The user experience reflects the performance of a transaction instance relative to the usual performance of the business transaction. AppDynamics makes some intelligent guesses and calculations based on existing traffic flow to establish the default user experience profiles.

You can refine the criteria by which the classification is applied for your own requirements, as described here.

How is a transaction marked slow, very slow, stall, or error?

As soon as a transaction starts, the particular thread representing that transaction is monitored for a stall. Every 5 seconds, the in-flight transactions are evaluated to determine if they have met or exceeded the stall threshold.

If a transaction execution finishes before the stall threshold value, the execution time is first compared against the very slow threshold, then the slow threshold threshold, and so on, and marked accordingly.

If a transaction hits the stall threshold (takes more than 300 deviations above the average for the last 2 hours or the set stall threshold), a stall transaction event is generated. Whether the transaction eventually finishes successfully or times out, it is considered a stall for performance monitoring purposes.

Static and Dynamic Thresholds

Thresholds can be based on a static value or dynamic value. A dynamic threshold is based on performance for a most recent period of time, the previous 2 hours, by default. A dynamic threshold can be specified using either a percentage deviation or a standard deviation measure.

To understand how a transaction is compared to this moving window, consider a two-hour moving average window from 11:00 AM to 1:00 PM. All transactions that come in during 1:00 PM and 1:01 PM are measured against the response time and standard deviation of this window. For the next minute, between 1:01 and 1:02 PM, the window moves to 11:01 AM to 1:01 PM. The moving average itself is calculated based on the exponential moving average formula.

Note that:

Data for minutes where there is no load is not counted.

The data is maintained locally on each node for the starting tier for a business transaction. The Controller does not hold any of this data.

Percentage Deviation Threshold

Percentage deviation defines a threshold based on the moving average of the requests over a certain interval. The default time interval is 2 hours. If the average response time for the last two hours is X milliseconds, and if the request takes the percentage deviation over X ms, the transaction violates threshold.

For example, a transaction occurs at 10:00:01 AM. The average of the requests between 8:00:00 and 10:00:00 is 100 ms and the slow threshold is defined as 20 % over threshold. If the transaction takes 120 ms, it will be considered a slow transaction.

Standard Deviation Threshold

Standard deviation defines a threshold based on the moving average over a certain interval. Default time interval is 2 hours. This means if the average response time for the last two hours is X milliseconds, and if the request takes the standard deviation over X ms, it violates threshold.

To understand how a standard deviation threshold works, consider a moving average that is 1500 ms with a standard deviation of 100 ms. If you set the threshold to 3, as in the following example, it means that the threshold will be three times the standard deviation. In other words, a transaction that takes more than 1500 + (3*100) or 1800 ms will violate the threshold.

Alternatively, you can set the threshold to a static value or use a percentage of the average.

You can also disable stall detection. If you disable stall detection, a transaction that might qualify as a stall is reported as very slow if it completes. If it does not complete, nothing is reported for the incomplete transaction.

You can configure thresholds for slow, very slow, and stalled transactions. When a transaction exceeds a threshold, AppDynamics starts capturing snapshots of it. Because snapshots are not normally captured while performance is within normal range, the snapshots may not contain the full call graph for the transaction.