Search

In particular, I’m most intrigued by them offering support/integration with WebDriver API access to SilkTest. That means automating with SilkTest using WebDriver APIs (via a tool called SilkAppDriver) in your language of choice, not running Selenium tests through SilkTest IDE/code, although I’m sure the latter is supported in some way.

WebDriver API is a W3C specification, and it’s great if we can get more applicable tools to use that protocol to drive (G)UI automation than to have different proprietary languages, frameworks to for each tooling vendor. Say for example, you can switch out your home access point or router and your network still works the same as they speak same protocol. Shouldn’t have to be a pain to switch out test automation tools either, the dreaded vendor lock-in.

And Microfocus also has a nice improvement over Selenium IDE as well: Silk WebDriver.

Now if only other (commercial) test automation tool vendors could follow suit.

NOTE: on a related note, I hope to see (more) variants of SilkAppDriver for the free and open source non-browser/mobile automation tools (since we have Selenium WebDriver and Appium/Selenoid for those already). We currently have these in the free/open source space: AutoItDriverServer, AutoPyDriverServer.

Update 01/26/2018: I just came across Ranorex Selocity, a Firebug/FirePath/Firefinder replacment, but for Chrome instead of Firefox. Cheers to Ranorex for offering such a tool. Now I hope they follow on with their equivalent to SilkAppDriver, and perhaps especially Silk WebDriver since Selocity is closer to the latter than the former, but I personally prefer more options of the former.

Unlike other public bug tracking systems, Apple’s one is closed to public viewing other than Apple and the reporter of the bug.

So bugs you report, others can’t see the status of. And as such, there can be likelihood of duplicate bugs being filed, etc. I assume Apple has good reasons to overlook potential duplicate bugs in favor of this closed access bug tracking system.

Amusingly, a public repository of Apple bugs is available at OpenRadar. It’s useful for the public community to look for existing bugs filed to Apple before filing their own, and check status of said bugs. There is one caveat though, OpenRadar does not pull data from Apple’s system. Therefore to get data into OpenRadar, one must file a “duplicate” bug to it for whatever one files to Apple.

As such, I would recommend those of us who deal with Apple bugs to:

Check OpenRadar first before filing bug to Apple. If one exists on OpenRadar, just subscribe to and monitor that. Although one is free to file a duplicate to Apple just in case.

If one doesn’t exist on OpenRadar, then file bug to Apple. And if the bug is not specific to your internal case (confidential/security/proprietary info, intellectual property, etc.), post a duplicate filing on OpenRadar. This allows the public to see the bug you filed. Also makes it easier for team members to view the bug for those teams that don’t use a shared/team Apple developer account or whose said accounts aren’t all under the same organization access.

This thought just came to mind as I read through forum posts recently regarding automation scenarios like the following:

a web app or site is delivered to the user via a browser presented within a browser. e.g. via Citrix virtualization solutions delivered over a browser for desktop & browser-based apps.

a web or desktop app delivered to the user remotely via virtualization but made to feel as if it was installed, running, and deployed locally on the local machine, delivered via tools like MS Application Virtualization Client, Citrix, etc.

How do we test for these apps/cases, particularly say if we wanted to test against the exact deployment setup of how they are used? It would seem from the current infrastructure, this is likely a manual testing activity. And for automated testing, we are relegated to simplifying the test setup and assuming the simplified setup will behave the same as actual deployment, and just cover any differences with manual testing.

I bring this up as I know not of any existing automation solutions, commercial or free/open source, but particularly the latter, where these kinds of special deployment setups are supported for automation testing. As these deployments are delivered through virtualization technology, unless the test automation tools are integrated in that virtualization delivery (e.g. hosted and running in the same context as the app that’s delivered over virtualization rather than locally on the client receiving the virtualized app), the test tools won’t be able to detect and automate the virtual app. That integration is likely not feasible or easy to do due to platform isolation and security restrictions and sandboxing. It’s not quite the same as just running everything (the app under test & the automation tools) on a remote host and accessing it via remote desktop/VNC to see it in action.

Perhaps this isn’t much a concern for the community/industry until more and more people use this type of app delivery infrastructure where it is ubiquitous like mobile devices/apps are today. Still, something to ponder about for the future of testing. Wonder what is to come in this space for test automation. Or have I just been ignorant and we already have some tooling for this today?

It just occurred to me today, from reading a Selenium forum post, that a past blog post of mine can also apply to the following cases:

you have both desktop and mobile views for a website or web app (e.g. responsive design, etc.)

you have a website and a native mobile app that offers similar service/functionality

and as such in both these cases there is shared logic (user/app/site workflow, element locators – even if the actual values differ, but the “logical” representation is the same such as login button on both web site and mobile app, etc.).

Using the techniques defined in that other post, you can share locator references/variables and share page object methods and have a single page object represent both mobile & web versions. For sharing of methods with branching logic, instead of checking which A/B flow to go through, you check what platform you’re on and go to appropriate branch logic based on that.

Also on some specifics, I haven’t tested this type of implementation before, but in theory should work out even for cases where you have website and a native mobile app:

You just have to use the right driver instance in the page object methods (or instantiate the page object with the right driver instance), etc. based on the platform (e.g. Appium/ios-driver/etc. vs Selenium WebDriver).

For shared locators, XPath may work best, using the multi-valued locator approach with pipes “|”. That is because I know Appium (at least) supports XPath based locators. Not sure about CSS selectors. So, I’m hoping/assuming Appium supports the multi-valued XPath functionality. Don’t know about the other tools lke ios-driver, etc. If this doesn’t work out, you would need separate locator references/variables then.

Some of you might not agree, but this is one way to do it with code reuse via some manageable complexity. The alternative is to have separate page objects with separate methods and locators. That keeps things simpler but give you extra files, extra code, and some redundancy when some of that code and locators are kind of similar. In the end of course, choose whatever works best for you.

I recently started working with Docker. And Docker is the rage these days, with people trying to deploy and run their systems/applications as Docker containers. And how to test these new deployments, etc.

As a matter of fact, even Selenium is made available in Docker containers for test infrastructure:

It got me thinking, should we as test automation specialists take things further and also dockerize our tests (and test framework, tools, etc.) as a package? Something like pull down a docker container that has all your test environment preconfigured (Java, maven, Eclipse, JUnit/TestNG, etc.) for you to easily run functional/black blox/regression/integration/API/UI tests, etc. rather than set up your localhost environment with the right software and pull down test code from source control then run, or use a fat bloated virtual machine with all this preconfigured.

I made a post to StackOverflow for this but have yet to receive any feedback. Some folks upvoted my post though. Do you as my blog reader have any thoughts you’d like to share here or in my SO post?

Was reading a slide presentation today from another blog post: http://www.slideshare.net/abagmar/automate-across-platform-os-technologies-with-taas. Brought back memories of earlier work I did of defining a test automation framework: http://yadiytaf.sourceforge.net. I still think my YADIYTAF specification is a nice read. Although I never did create a reference implementation of the spec. I ended up prototyping what I needed at the time against the existing RobotFramework (RF). From that effort, I ended up expanding the functionality/feature set of RF by expanding it’s interoperability with Java, .NET, Perl, PHP via their remote library server interface implementing servers in those languages. So RF is a lot more powerful now since I first evaluated it years back.

In the end, makes me think, should one re-invent the wheel or not at times? Or see what’s already available and build upon that. I guess it depends on what works for you or not.

These tips may be found elsewhere on the web already, I’m just archiving this info I put together a while back into a single repository in the form of a blog post.

Tips for both iOS devices and iOS simulator

Firebug for iOS

Set up a javascript bookmarklet to Firebug Lite, so that you can invoke the lite version of Firebug in mobile Safari on actual device or simulator. See Martin Kool’s article for details, if you don’t know how to set up javascript bookmarklets. NOTE: on simulator, you will lose the bookmark when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete bookmark or do a factory reset.

View Source of a page

iOS 8+: install an iOS 8 extension like the one called View Source. There are several extensions to choose from. The View Source one may be best as most popular, well known. Or you can also choose the other option for older iOS versions.

iOS 8+, 7.1, and earlier : use this Javascript bookmarklet trick/tip, or this version called Snoopy. NOTE: on simulator, you will lose the bookmark when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete bookmark or do a factory reset.

Tips for iOS simulator

Set up camera roll to be able to use photos and videos for testing

See: How to import photos into the iPhone Simulator — Aptogo. In general, drag & drop photo into simulator and then tap & hold to save image in simulator will add to camera roll for later use. See the linked article if you need to bulk import photos into simulator. NOTE: on simulator, you may lose the photos when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete photo or do a factory reset.

If you use an Apple/iCloud account across the devices (This might work on simulator iOS 7/7.1+ if you sign in to iCloud from simulator and then do a Debug > Trigger iCloud sync, although I haven’t tested it):

The photos on each device (if not existing photos, then new photos taken) will be accessible across devices via the Photo Stream section of Photos app. And deleting photo locally in camera roll of a device won’t delete it from the Photo Stream (for that device and all other shared iOS devices), so you have to specifically delete from Photo Stream to delete across devices, the camera roll is treated separately from Photo Stream on each device, so you will notice copies of what you have locally on camera roll also in the Photo Stream that’s available across devices. NOTE: that the Photo Stream (for iOS 6-7.x) is temporary storage, and to make it permanent on each device, you have to save it to local system (device/simulator) before it expires in 30 days (I forget, but the system may copy the photos you’ve previewed to your device after expiration rather than let it disappear from the stream). The iCloud Photo library in iOS 8 is different though with it being permanent storage.

Creating a bookmark on one device will automatically sync (if enabled, by default) them to the other devices using same account, so you don’t have to set up bookmarks/bookmarklets individually on each device.

Recording actions against or test session of iOS physical device into a video