6 Answers
6

A year or two ago, my employer's existing automated testing system was nearing the end of its licence period, and I was tasked with evaluating EggPlant as a possible replacement. I spent about a month recording tests, writing test scripts, and generally trying to pick at EggPlant's flaws.

For the most part, EggPlant could do whatever we needed it to. As others have said, it was image-recognition based, so tests tended to break when the AUT's interface changed, but because it allowed image masking and fuzzy image recognition and (more importantly) could read text if given the font, it was possible to write tests that had non-zero tolerence to change, and which could be updated with relatively little effort when compared to some other image recognition-based automated testing systems out there.

That said, Eggplant did have two significant flaws when I was evaluating it. The first is that it was a very mac-based piece of software. That's not a problem on its own, of course, but it was the Windows version of the tool that I was examining, and most of its features and interface were designed for a Macintosh operating system. I'm not just talking about on the machine running the AUT, either: The EggPlant user interface, itself, used Mac UI conventions, which are markedly different to those of Windows. It tended to be a little confusing.

EggPlant's second significant flaw was that it was full of crippling bugs. To be fair, when we mentioned these bugs to TestPlant support they were fixed quite rapidly, but the sheer frequency with which showstopper bugs were appearing made it impossible to use the tool without being constantly distracted.

(I should stress that these two flaws may well have been fixed by now. The Windows port of EggPlant was brand new when I was looking at it, and many of the bugs and quirks of the system were probably teething problems.)

In the end, we ended up not using EggPlant, but that was because of the mostly windows-specific flaws I've described above. I imagine that if I'd been running the tool on a Mac, I'd have had a much more positive impression of its capabilities.

The major drawback to this tool is that it is driven using image recognition, this in my opinion is a major drawback. I can guarantee that your application will change and that you will need to go back and recapture most of your images to get your test suite to run again. This might be acceptable when you have 300 tests, but imagine you have 10000+ tests, you suite will take weeks to fix, and before long the effort to keep up with the changing UI will not be worth it.

A better approach would be to pick a tool that best fits the need of the application you are trying to test and not apply a one solution fits all applications.

If it is anything like Tevron's Citratest (also image-based, cross platform), it could be interesting but maintenance is a pain sometimes. Image change issues can be mitigating by building a good activity-based library (e.g. create order function) which will mean changing one image updates multiple test scripts.
–
StevenJun 30 '11 at 13:41

@Steven we are having problems finding a tool that will test our system on Mac & PC. Our software has a Java client, DTP plugins to Indesign and Quark and a web client. No tool we've looked at had good support for drag and drop which is used extensively in the Java and DTP side.
–
AlastairJun 30 '11 at 14:05

Any AGT system based on image-comparison is inherently brittle. If your AUT is browser-delivered, I highly recommend you adopt Selenium and/or Watir. It's got the best community support, cross-browser compatibility, and is implemented in most major languages.

With text recognition working fine ..i feel this tools seems to be more powerful compared to image recognition tools.
This tool is not suitable for drastically changing platforms but if UI or menus are freezed then for mid to long term projects eggplant should satisy most of the requirements of an automation tool

All in all I would say that for image-heavy application Eggplant is a great tool - just make sure it is the right tool for what you want to use it for - I think you can get a free trial from their website.