June 27: Colombo Selenium MeetupSelenium’s Journey
By Jason Huggins, Co-Founder of Sauce Labs
Get an introduction to Selenium’s colorful history in this virtual meetup with Jason Huggins, original creator of Selenium.

July 9th: Boston Selenium MeetupJuly 11th: DC Selenium MeetupOptimizing Selenium for Executed Performance
By Santiago Suarez Ordoñez, Selenium Ninja at Sauce Labs
This talk will cover test writing tips gathered over the years from helping hundreds of Selenium users and engineers make their tests run faster and more reliably, independently of the technology stack used. We’ll be talking about things such data independence, making atomic tests and generating application state.

July 10th: Continuous Delivery NYC MeetupContinuous Delivery at New Relic
By: Bjorn Freeman-Benson, VP of Engineering at New Relic
Bjorn will talk about New Relic’s experiences in growing from a few developers supporting a few thousand users to 70+ developers delivering continuously to thousands of large and small commercial customers.

July 17th: San Jose Selenium MeetupHARdy Har Har: How to generate HTTP Archive Files
By Adam Goucher, Selenium Expert and Consultant at Element 34
This talk will show you how to generate HTTP Archive (HAR) files in two different ways; using a custom Firefox WebDriver profile and with the BrowserMob Proxy.

July 18th: San Diego Web Performance MeetupCapturing Performance Data from your Selenium Tests
By Patrick Lightbody, Director of Product Management at Neustar Enterprise Services
Learn from one of the original Selenium developers how you can use open source projects and new specifics, such as Navigation Timings, to augment your existing test suite and capture performance in your existing continuous integration environment.

July 18th: OC PHP MeetupSelenium: Less Testing and More Coding
By Jonathan Lipps, Front-End Engineer at Sauce Labs
Jonathan will give a demo of a basic PHP web application and go through the motivations for using Selenium as a way to consistently and reliably test the application. He’ll then describe how Sauce Labs’s cloud service enables testing to transparently scale to multiple browsers and platforms and reduce several frustrations with running tests locally.

PHP packages are distributed through Pear channels. If you want to download a PHP package, it’s as simple as downloading Pear and using it. The process of using pear is telling it the “channel” you want to download the package from, and then telling it to download the package. That’s what you do if you want to download somebody else’s PHP. If you want other people to be able to download yours, you have to make your own pear channel.

We had to do that recently, and it turns out there’s an easy way to do it thanks to GitHub and Fabien Potencier‘s Pirum. It’s pretty straightforward. Here come the lists. So many lists! Like 40. Still. Straightforward.

If you are in SaaS business, one of the best things you can do for yourself is to setup a continuous automated browser testing environment. It will save you a lot of headache, money, and customers down the road.

As a happy new Sauce OnDemand user, I’d like to share our story with automated browser tests and how we integrated a PHP based web app with Sauce OnDemand.

Our web application, JotForm, is a web based form builder. Using its drag and drop WYSIWYG form editor, you can create web forms quickly and easily. Since our core application is written in JavaScript, it is extremely important for us to do automated cross-browser tests. We accomplish this by using Selenium.

We use Hudson to automate our releases. We do continuous integration and deployment. We release our product ten times a day every day. When a developer makes a commit, Hudson executes hundreds of unit, integration, and cross browser tests. Our service has over 300,000 users and they can be very loud when we mess up. That’s why we care a lot about our tests. If we mess up once, we investigate how we missed it and make sure to add new tests so something like that never happens again.

Why Automated Selenium Tests?

If you are developing a web app, you must have already discovered that unit tests don’t work well. They can only really cover the server side and little bit on the JavaScript, but when it comes to testing your code on a multi-browser environment, they are pretty much useless. Browsers constantly change. Just this week, our developers spent a lot of time making our app work properly on IE 9 Beta.

Ideally, you would want to test your app on all browsers and all platforms. Selenium is the best tool for the job. You can easily record tests using Selenium IDE and integrate them with your continuous integration tools, such as Hudson or Cruise Control.

Why NOT Automated Selenium Tests In-house?

The biggest problem with running Selenium tests is creating and managing many testing environments. Usually if you are running Selenium tests constantly on a machine, it cannot be used by anyone. So in practice, they have to be on dedicated machines. And if your releases depend on the Selenium tests, you have to make sure they are always on and running smoothly. It is a lot of maintenance headache.

The other problem with Selenium is that it is slow. It takes Selenium a long time to launch up the browser to do the tests. Unless you have many dedicated machines, the tests have to run sequentially.

To speed up our Selenium tests, I looked for ways to run them in parallel. My first approach was to use Amazon EC2, since I am already pretty familiar with it. That turned out to not be a great solution. We still had to maintain many selenium environments. Keeping many EC2 instances always on can be quite expensive. Launching new instances on demand turned out to be too slow for our case.

Final Solution: Sauce OnDemand

Then, I found out about Sauce Labs’ Sauce OnDemand service. This is basically an on demand service that lets you run Selenium tests instantly on the cloud. It has many good code samples. So, setting it up was pretty straightforward. The great thing about Sauce OnDemand is that you can choose from many browser setups and run your tests on them instantly. No need to maintain dedicated machines.

Unfortunately, there was no documentation about running tests in parallel in PHP. So, we first looked into using PHPUnit’s new parallel processing features: @runTestsInSeparateProcesses and @runInSeparateProcess. They turned out to be too buggy to be usable at this time. So, we hacked together a simple script that runs the tests on the shell as background processes.

A Simple Example

I am a big Perl junkie, but since our existing unit and integration tests were written in PHPUnit, we wanted to keep our Selenium tests consistent and write them in PHP. Below, you can find a simple example showing how you can run automated cross-browser Sauce OnDemand tests on PHP. We hacked this together quickly so it is not documented or optimized. But it should give you a good jump-start:

1. BrowserTests.php: Includes actual Selenium tests and simplified for this example with a single test. On our production version, we have all the tests here, since we prefer to save time and run all tests at once on a specific browser. You can create these tests using Selenium IDE or Sauce IDE.

2. allBrowserTests.php: You can run this on the shell then add it to Hudson or any other Continuous Integration tool. Basically, this script runs Selenium tests on the background in parallel. It then checks their log files to see if they finished successfully or failed.

3. SeleniumTestHelper.php: This is the integration script with Sauce Labs. You should change the domain name, Sauce username and access key to your own.

Download all files files here. You must first have phpunit installed to use them.

Happy testing!

Aytekin Tank is a co-founder of Interlogy Internet Technologies. Their flagship product is JotForm online form builder.