Resources for UI Automation and simulating user input

An important part of my job is to be able to interface our product with third party UI’s. Typically, I know virtually nothing about the UI before I begin the project, only that I will need to enter data into different fields.

Over the years, I’ve accumulated a list of my “goto tools” to help me assess each application. Behold:

Tools:

UI Spy: This tool used to come with older versions of the Windows SDK. I personally think this is the most important tool. It allows you to see properties of every Windows, Pane, and Control in an Win32 application. I had a real hard time finding this tool a long time ago. Fortunately I came across Andrew Tokeley’s blog and he has it zipped up and available. You can download it from there.Andrew’s Blog

UIA Verify / Inspect.exe / AccScope.exe / AccEvent.exe: These tools are extremely helpful for analyzing user interfaces and will help determine what automation and input functionality is supported. These are all included with the Windows 8.1 SDK.Get the Windows SDK | UIA Verify Codeplex Site

Snoop (WPF): Snoop is one of my favorite tools for analyzing WPF applications. It’s open source and can be downloaded from their GitHub page.Snoop GitHub

Window Detective: A great open source tool similar to Spy++ but it has some unique features and in some ways is more intuitive. I highly recommend checking this one out.Window Detective Sourceforge

Spy++: Good ol’ Spy++. This tool has been around for nearly a million years, and it’s still distributed with Visual Studio. You can find out a lot of information about a Window, Thread, or Process by monitoring the Win32 Messages and viewing some elements of the control and window hierarchy.

XAML Spy: I haven’t done much with automating WPF/Silverlight UIs, but as Keith mentioned (in the comments below), XAML Spy seems to be a nice tool for handling debugging WPF/XAML-based UIs. There is a free Express edition and the the full featured edition which costs money.XAML Spy Homepage

F12 Dev Tools (IE, Firefox, Chrome): When automating a Web UI through a browser, the Inspector in the F12 Developer Tools can be the difference between a one-week implementation and a one-day implementation.

Libraries:

Selenium: I have recently replaced WatiN with Selenium for all my web browser automation needs. It supports all the latest browsers on several operating systems. In order to trigger the automation routine from a Windows application, you should use the Selenium WebDriver libraries. Despite what their supported browser list may say, they are almost always on top of any new browser versions (their documentation guy must not be as on top of things).Selenium HQ | Selenium on Google Code

WatiN: WatiN (pronounced What-In) is an amazing open source .NET library for automating UI’s in a web browser. You have full access to the DOM from your WinForms application. If you ever need to have a .NET project automate the UI for a web form through IE or Firefox you will love the power of this library. Note, there is some limitations in browser versions, it seems to support IE 6 – 9, and FireFox 2 – 3. Hopefully a newer version is in the works.WatiN Homepage

UI Automation Framework: Microsoft .NET includes a set namespaces for helping with UI Automation in WinForms apps (Win32, WPF, .NET WinForms) and it’s really quite powerful. I believe the first version to include this was .NET 3.0. You can read more about this at MSDN. This is great for most native Windows apps, but if you find yourself needing to automate something more complex (such as an application through Terminal Services or Citrix) see the next option below.UI Automation Overview

BitCollectors.UIAutomationLib: And finally, my own shameless plug. If you find yourself needing to simulate keystrokes and mouse clicks it really doesn’t get much easier than this. This is a library I developed and have been maintaining for a couple of years now internally and I just published it on GitHub. You can pass in an XML file which defines how your keystrokes and mouse clicks will get executed against a specific application. This will work with just about anything, including applications running through Terminal Services and Citrix.Blog Post | GitHub Page | ReadMe

Thanks Keith, those are some great suggestions. I forgot about XAML Spy – I had to use that a while back for a Silverlight project (I’ll update my post with information on that). As for the other two, I haven’t ever done anything with those. I’ll definitely check out that Coded UI Map in VS2012, that sounds really useful.