Tuesday, January 24, 2006

Useful tools for writing Selenium tests

I've been writing a lot of Selenium tests lately and I've been using some tools that I find extremely useful for composing table-style tests. Let me start by saying that writing GUI-based tests for Web apps is no fun, no matter what your testing tool is. You need to navigate through pages, fill and submit forms, and verify that certain elements are present on the pages. Doing all this manually can quickly become tedious and kill whatever joy you may find in testing.

If you're writing Selenium tests, these activities are, if not fun, at least tolerable due to the existence of the Selenium Recorder ("the Selenium Recorder -- can't imagine life as a Web app tester without it" seems like a good line for a commercial :-)

The Selenium Recorder (which I'll refer to as SelRec from now on) is a Firefox extension that you launch via the Tools menu. It intercepts Web browsing actions such as opening a URL, clicking on a link, entering text in a field form, selecting a drop-down menu item, submitting a form, etc., and it records them in HTML format, ready to be copied and pasted into your Selenium test files. What I find most remarkable about this is that I don't need to worry about identifying HTML elements in order to find a suitable Selenium locator such as an XPath expression, or a DOM expression. For example, SelRec is smart enough to identify links using the simplest locator expression that uniquely identifies a link. If the name of the link is unique, SelRec will use it. If several links have the same name, SelRec will automatically use an XPath expression that uniquely identifies each link.

SelRec can also help with certain Selenium assertion commands. For example, if you highlight the text on a Web page and right-click, you'll see a menu item called "Append Selenium command" which allows you to add one of the following 2 commands to your test table:

verifyTextPresent|highlighted text|

verifyTitle|title of the current Web page|

You can also append "open|current URL|" to your test table via the "Append Selenium command" menu.

I would guess that most of the testing you're doing with Selenium consists in opening URLs, clicking on various page elements, submitting forms and asserting that some text is present on a Web page or that a Web page has a certain title. Since all these actions and verifications are supported by SelRec out of the box, your life as a tester suddenly becomes a bit easier.

One problem with the "verifyTextPresent" command is that it doesn't support regular expressions. Many times you do need to verify that a specific chunk of text on your page conforms to a regular expression. For this, you need the "verifyText" command, which takes as its first argument the locator of the HTML element whose text you need to verify. This is when things become a bit hairy: you somehow need to identify that element. You can always try to do it in your head by eye-balling the HTML source of the page and trying to come up with the right XPath expression. But fortunately there's an easier way.

At this point, I'm glad to be able to point you to another sanity-saving tool: XPath Checker, which is also a Firefox extension. After installing it, you highlight any text on a Web page, right-click and choose "View XPath". A Firefox window will pop up with an entry field containing an XPath expression, as identified by the tool, and with the text matching that expression. This window allows you to interactively change the XPath expression and see the text that it matches. It's great for playing with and learning XPath (BTW, here is a great XPath tutorial).

Together, the Selenium Recorder and the XPath Checker tools will allow you to easily write the vast majority of your Selenium tests. For more advanced functionality, I refer you to the many Selenium user extensions contributed by people active in the Selenium community. For some nifty Ajax testing (if I may say so myself), see also this blog post of mine.

Note that XPath expressions don't work that well in Internet Explorer. If browser compatibility is high on your list, then you probably need to identify HTML elements via their DOM locations. For that, you can use the DOM Locator extension which ships with Firefox, but you can also peek at the XPath expression generated by the XPath Checker and try to come up with the equivalent DOM expression. Camino, a Mac-specific browser, doesn't seem to have this problem, so you can very easily write your Selenium tests using Firefox with its extensions, then run them in Camino (Titus and I did just that when testing our PyCon tutorial app).

To wrap this thing up, here's an example of using the SelRec and the XPath Checker. Let's assume you want to test some of the search functionality of amazon.com. Note that you can't do this with table-mode Selenium unless you work for amazon.com, because you need to deploy your tests on the amazon.com Web app servers. But for the sake of this example, I just want to illustrate how to use the tools when testing a complex Web site.

Here's a test table that was generated with the Selenium Recorder and that I edited with Mozilla Composer in order to add my own verifyText commands, for which I used XPath expressions that were generated with the XPath Checker:

The Selenium Recorder can help even when adding your own verifyText commands. Here's how I did it:

I highlighted the text I wanted to verify on the Web page under test, then I right-clicked and I chose "Append Selenium command" -> verifyTextPresent "highlighted text"

I saved the file generated by SelRec, then I opened it in Mozilla Composer

The last row in the file was something like |verifyTextPresent|some text||

I edited the row and I replaced "verifyTextPresent" with "verifyText", then I did a cut and paste of "some text" from the second to the third cell

I highlighted again the text on the Web page under test, I right-clicked and I chose "View XPath", then I copied the XPath expression generated by the XPath Checker in the second cell of the table row

The row now looks something like |verifyText|//some/xpath[2]/expression|some text|

At this point, if you need to make sure that "some text" matches a regular expression, you can start the third cell with "regexp:" followed by the regular expression (for example, in the table above, I used the somewhat-contrived "regexp:Buy this book with .* by Alex Martelli today!")

Hope this all helps. I'll soon add post another entry on dealing with frames and pop-up windows in Selenium.

Updates

Patrick Lightbody, the maintainer of openqa.org, where Selenium is hosted, reminded me that the Selenium Recorder project is now called the Selenium IDE and is hosted at OpenQA. There is no release of the Selenium IDE yet, but you'll find out when it will be available, I promise :-)

Speaking of the upcoming release for the Selenium Recorder/IDE, one feature I'd like to see added is the capability of adding verifyText commands, with the XPath expression corresponding to the highlighted text -- an unification of the XPath Checker functionality with the current verifyText functionality in SelRec

Luke Closs points out that he wrote a tool that allows him to write Selenium tests in simple wiki-like tables, then it will convert them into HTML and auto-create a TestSuite.html for him; you can find more info about Luke's Perl-based tool at awesnob.com/selutils.

An anonymous commenter points out XPather as another Firefox extension that's superior in his/her opinion to the XPath Checker extension

54 comments:

When doing test driven development, I often write my selenium tests by hand. I don't like writing HTML tables, so I wrote a little tool to allow me to write my selenium tests in simple wiki-like tables, and it will convert them into HTML and auto-create a TestSuite.html for me.

The tool is called selenium-regen, and it is part of the perl package WWW-Selenium-Utils.

Personally, since I'm a developer, I try to make sure that each element that I want to test has a unique "id" attribute. But I've heard complaints from Selenium users saying that isn't a realistic possibility for many testers (especially when they have no access to edit or influence how the code is written)... So in those cases, XPath (or DOM) expressions to find elements to is the fail-safe option for those users. The downside to XPath is that it isn't always easy to write those XPath expressions... But you're right, with "SelRec" and XPath Checker, the situation is much better! Now we just need to find a tool that can write all those tests automatically!

Simon -- you need to choose "Custom mode" when installing Firefox; then you need to make sure you check the "Developer tools" box. I think the DOM Inspector doesn't get installed by default. Thanks for bringing this up.

I must say that I used to use Selenium Recorder extensively - however, the current version does not "seem" to work as well as the previous one (perhaps it is just the app I'm testing)...

However, I do second your recommendation of this tool, as it can save many hours of tedious work.

An anonymous user recommended XPather - I must second the recommendation. In actuality, I use both XPath Check and XPather. I find that the XPaths shown by XPather are much more cross-browser friendly. Many of the XPaths I "got" from XPath Checker simply would not work under MSie (again, this could just be for the application I'm testing), so I use XPather to get a more "specific" XPath, and then use XPath Checker to "test" a shortened version of the XPath...

My name is David, and I am doing testing for a web application where I am constantly creating new registrations. At the end of a registration, my application id is printed in text on the page, surrounded by "You have been assigned the following Registration Number: XXXXXXXXXX. Please print a copy of this for your records."

Is there a way to save that text using a selenium IDE command, and then print said text variable to a file?

David: you need to identify the element that contains the text you're interested in. You can identify it by HTML id, or by name, or by XPath expression. You can use a Firefox extension such as XPather or XPath Checker to identify it by XPath.

Then you do something like this in your table to store the text in a variable my_text:|storeText|your_element_id|my_text|

Then you need to write a simple CGI-type script that takes an argument and writes it to a file. You then open the URL corresponding to your CGI script -- something like this:

Hi,Thanks for the post. It's very informative and I've bookmarked it for repeated reference (Along with the other posts that you have published :) )

I noticed you mentioned that for text matching, Selenium does not allow regex.But on checking the Selenium reference, they do list out 3 different types of pattern matching capabilities for matching string values.The reference is availaible at http://www.openqa.org/selenium/seleniumReference.html

Am I mistaken or has this functionality been recently added to Selenium?

Hi,I work with dotnet and I use datalist and data grid in my page that auto generate tables and rows how can I test such this thing that I don't have their ID,I asumme its a posibility to use xpath could you please send me a sample code for that. I want to write my code for this test case.and not using the recorder.

I am facing a problem in Popup window.I want to verify the text present in window and then click on ok of popup window.I tried it by using

assertAlertPresentId should be Unique

But it gives result false in this case and unable to click on ok button of popup window.Also if I use asserAlertNotPresent then it returns true but in this case also it was unable to click on ok button of popup window(message)

I am so thankful for your blog entry. I had no idea we could use XPATH in Selenium. Using the XPath tool I was able to solve my problem: I needed to somehow manipulate a nonstandard slider that was not addressable by ID. I struggled for hours to write a user-extension but could not make it work. Whew and thanks!

When using verifyTextPresent of assertTextPresent, once it returns FALSE (highlights red in the table), it halts my test. Is there a way to tell selenium IDE to continue the test if a condition fails? I'd like to be able to test a bunch of pages then look through my table results and see which highlighted red/green, etc. Thanks!

Denise -- verify commands are supposed to not halt your test (as opposed to assert commands, which do halt it). I don't know why verify behaves the same way as assert in your case. Sounds like a bug in Selenium to me :-)

Hello Grig,if I do have a DOM locater and know where that element is, what is the format of the expression should I put in the selenium file (table cell).Because I need to test web apps that have dynamic generated element id.

Hi,Selenium is not identifying the popup.i used selenium.selectwindow("name=windowid");where windowid is the id of the popup.In this case the windowid of the popup is dynamically generating which is equal to the sessionid.i tried but its still not identifying the popup and i am not able to performs the actions on those popup controls.i really need it soon please provide me some solution.i will be thankfull to u.

I want to do parameterization in selenium is there any option or command or any code available to it?I have a application and I want to add multiple projects in it but every time I have to give different project name and Project Id.How would i do this in a singe script?Is there any option to do parameterization/looping in selenium??

Hi My name is Nteen:I want to do parameterization in selenium is there any option or command or any code available to it?I have a application and I want to add multiple projects in it but every time I have to give different project name and Project Id.How would i do this in a singe script?Is there any option to do parameterization/looping in selenium??

In Java (or whatever language you are coding your tests in if testing programatically) you can wait in a loop (perhaps calling waituntilElementExists) until the target XPath appears, then send your event at it. So you can wrap the selenium proxy (DefaultSelenium) with one that inserts this wait before calls to the wrapped methods. Cheers

Hi Grig!!Thanx alot for such a wonderful article...I am new to selenium and exploring it.I have a problem testing with selenium and I hope you could help me out in it.I am testing an application with selenium ide.This application dynamically generates ids and i dont want to write down xpaths manually each time.I get following id by clicking some button and it changes each time.e.g. - click gwt-uid-172

I want some kind of expression or may be some way by which i can capture these ids and wont need to change the xpaths each time.How should i go about this?I hope you could help me out.Rashmi.

Is anybody know, how it is possible to use Selenium with XC Engine (http://xcengine.googlecode.com/files/xcengine%28minotaur%29-v0.2.zip) test automation framework? I'm looking for some examples, how I can do this.

I am in confusion, i have implemented the selenium automation using perl script. But i come across that no body implemented the automation using perl script,they have used java lot, this is making me worry.I wanted to know which is better one.