Oracle Application Testing Suite Review

Load testing is a valuable feature however the tool runs out of memory when the recording flows are long

Valuable Features:

Recording and playback

Load testing

Improvements to My Organization:

Our benchmark load tests for a major release involving various scenarios with a combination of different virtual users relied on the recording, playback and eventual load testing capabilities of OATS.

Room for Improvement:

The tool runs out of memory when the recording flows are long and involve many screens despite increasing the memory settings for OpenScript

We found the tool impossible to use for client side performance testing without filtering out JS, CSS, PNG etc as the method would error due to a Java method size limitation.

Parameterization of variables using regular expressions was not a foolproof way of running tests as it would mostly work for a single user but fail during load testing.

Parameterization involving SQL queries was not straightforward

Except a YouTube video/PDF which covers some basics and is probably outdated now, there are no tutorials available for other functionalities that the recording tool provides.There is no in-depth tutorial of the load testing interface and troubleshooting steps.

Collection of system metrics - although there are multiple parameters defined, it did provide a holistic picture of the state of the system.We had to implement our own way of obtaining this.

Load testing web interface did not provide details about errors. The only way to correlate was to inspect server logs.

Use of Solution:

One year.

Deployment Issues:

Yes, with 12.2.x

Stability Issues:

Yes, 12.2 was not stable. Upgrading to 12.4.x resolved a lot of previously known issues

Scalability Issues:

No issues encountered.

Previous Solutions:

This tool did a much better job recording the http requests during web load testing than other open source tools. Cookie management and other url parameters were handled efficiently and were easier to identify in the tree view.

Initial Setup:

It was straightforward for the most part except that at times the services wouldn't run and un-installation messed things up.

Implementation Team:

In-house.

ROI:

I believe we made a wise choice despite some shortcomings and hiccups.

Other Solutions Considered:

We evaluated other options.

Other Advice:

Read the tutorial and try out a few scenarios before going full throttle.

Disclosure: I am a real user, and this review is based on my own experience and opinions.