Thursday, November 22, 2007

FunFX - How to write tests

The last post talked about howto get started and how to make the Flex application ready for testing. I am sorry about the time it took me to post this sequel.

FunFX is a framework that is composed of two parts, one adapter (the last post talked about this part) that enables the Flex application to be tested, and one Ruby framework that enables you to drive an Internet Explorer instance with the Flex application.

Some people ask me if there is an easy way to implement support for either FireFox or other browsers, and the answer to this is both yes and no. It depends, there are currently only IE that is implemented and the reason for this is IE's support for COM objects. I will explain this implementation in a bit more detail in another post.

This post will talk about the Ruby framework mentioned in the beginning. This Ruby framework is a Ruby gem. This framework creates an instance of a Internet Explorer window. Then with some initializing methods you will direct the IE window to the address of the Flex application.This is shown in the follwoing table. The first sentance creates an instance of FunFX. When executing the second line with the start method FunFX creates an instance of IE in the background with the help of Win32OLE. The argument tells FunFX if you want the IE to be visible or not.

The last sentance in the initialization is the method that directs the browser to the Flex application and hooks on to the actual Flex object with the help from Win32OLE, and the name of the swf file. If you have an html file named index.html that contains a swf file that is named flexApp.swf and is located at http://localhost/flexapplication/index.html. The

@ie.goto("http://localhost/flexapplication/index.html", "flexApp")

After this initialization, the @ie variable (which is an instance of FunFX) is ready to interact with the Flex application.

The FunFX in itself does not test Flex applications in the sence of asserts, it just enables you to drive an Flex application programmatically and get the state and text of all display objects. To make assertions and create test scripts you will need to use FunFX together with a testing framework of your choice. A couple of examples are RSpec, and Test::Unit. FunFX together with any of these tools make up an automated functional testing tool.

Before I will show you an example of FunFX together with these tools, I will explain in short how to use FunFX in the sence of driving the Flex application programmatically.

A Flex application is built as an hierarchy of different display objects, with the Application object as the root node. With FunFX you can think of @ie the root node of the Application object.

An example of interacting with a display object is shown in the box below.

@ie.button("buttonName").click

This line of code will click the button with id or label named "buttonName". This button can be located anywhere in the Flex hierarchy. In version 0.0.1 FunFX only supported single elements, and then it stopped when it found a object matching this signature. But this was a problem with repeaters, which creates children with the same id's.

So in 0.0.2 it finds all objects that matches that type and name, and delivers them as an array, as shown below:

@ie.button("buttonName")[0].click

To interact with a display obejct you must provide the label or id.

Some flex event needs arguments as when inserting text into a textbox. This is done as follows:

There are many more methods. Unfortunately there are no good overview of all the methods and their arguments. But you should look at the AutomationGenericEnv.xml, which describes all the display object types and their events and arguments.

When testing the functionality of an application it does not help being just able to drive the application. It must be possible to extract some information to assert with a testing tool such as RSpec or Test::Unit.

You can get information about alot of thing, such as visibility, text, rows in a datagrid, enabled etc. These are called properties and are also described in the AutomationGenericEnv.xml.

The following displays some examples of extracting data. It is possible to extract the selected index of an data grid, making it possible to extract information about that grid line with the tabular_data property. This creates a comma separated string containing the row information. If there are images in the row, the name of the images will be displayed.

This post is a bit messy, but i tries to explain some of the basic elements. The following post will try to explain other concepts in more detail.

I have put out an test application that set up for testing with a couple of simple actions to perform at TestApplication. At the following links I have also written a couple of tests that test this application. Please try them out and alter them to see if you get it going (the source code of the view files are available at view source, or this address source-code).

55 comments:

I'm testing your API and I'm finding a problem with a custom component. It's the fisheye component from Ely Greenfield: http://www.quietlyscheming.com/blog/components/ fisheye-component. It has a property "selectedIndex" that, when set, changes the selection to the item desired:

Very impressive! I'm a member of the Philips QA Team and we are going to use this to test the Flex functionality of consumer.philips.com.We will use this together with Watir for the non-flex parts. Do you know if there is documentation about the object syntax (like radio buttons, checkboxes etc.)? Because it is different from Watir, e.g. 'radio' in Watir and 'radio_button' in FunFX.

I tried the example from you video, the WorkGroupTest and it somehow is not working for me. I keep getting errors soon after creating a blank FLEX application. When I build it, I get the following error:Error #2044: Unhandled ioError:. text=Error #2032: Stream Error. URL: AutomationGenericEnv.xml

It is nice that people like it, even though it is still an early edition. But this makes it fun to improve it.

About the object syntax John, the entire class hierarchy in Ruby is created dynamically in memory from the AutomationGenericEnv.xml, and the names in this xml file for instance DataGrid kan be written both @ie.DataGrid("name") and @ie.data_grid("name"), when a method is called the framework renames the Ruby way of writing names to the Flex way of writing names. So just look at that file. But your are right I should create something better that people can look at. I will see if I can create something, because there are also some special method I have created in Ruby (these are located in the xml_parser.rb file)

The first error is because you have forgot to add the AutomationGenricEnv.xml to the application, or it has not been copied to the output catalog when the application was built. Try to copy the xml file into the bin catalog or where your swf file is compiled.

The other sandbox error I am not sure what is, but keep me posted about your progress. I am glad to help you get going!

OK, so I just have to look in the XML file, great. Our website has a lot of customized object, so I suppose I have to add them to the AutomationGenericEnv.xml as well? Anyway, keep up the good work and if you need any help or testing, just let me know.

I realized now that my answer ws not to good, but it is correct that you should look at the xml file and that alle the names in that file is possible to use. But to make it more ruby alike and more the same way as Watir i created the framework so it understands ruby syntax as DataGrid in Actionscript and Java would be data_grid in ruby. So all the names in the xml file kan be written instead of camelcase but with the same names divided by a underscore.

If you see something that needs improvements or want to improve it go right ahead and please post stuff about the framework on either the user or the developer mailing list at http://rubyforge.org/projects/funfx/(And if you want a developer access at Rubyforge just let me know :-)

I really would like to get some feedback on usage.

If the customized objects uses special methods and events you will have to add them to the xml file and then build a new gem, but if you custom objects only inherits for example datagrid and does not use any special methods and events you just use the inherited version.

But I have not tried to use custom components to much, but I will try to write a post about thet this weekend.

And if there are something you are stuck with please let me know. The fastest way to get in tuch is with the mailing list.

It seems that you do not have a datagird object in your Flex application that is called dgEmployees or the automation package does not find it. If the latter then I am not really sure. Then you must send me more of the Flex code to see what is going on.

Do you remember to write the dataprovider of the combobox as this: dataprovider="{locationModel.location}", if you do not use the brackets it will probably show the text.

Let me know if this helps or not, if not I will take a closer look at the code.

Few days ago I found FunFX and I think that is exactly what I was looking for. But during I was working with your tool I met problem which I can't manage.In my very simple flex application I'm using custom object, Container which extends UIComponent and button object. I added ClassInfo to AutomationGenericEnv.xml:

Firstly, I would like to thank you for having released your peerless 'FunFX' tool as 'open-source'. Though I am totally new to your tool (had worked on another tool named Selenium with Ruby before), when I executed your sample script, I was very pleased to see it worked fine for me. But i was wondering how you got the names or properties of the controls you had referred to in your sample script and whether I can get the properties of the controls in our flash-based application, which do not have the 'View Source' option, in the same way. If you could let me know how to get the properties of various controls in a flash-based application, it would be extremely helpful.

If I understand you correct, you need to be able to find the names or id's of the controls. Like @ie.button("nameofbutton"), you would like to know how to get the "nameofbutton" ? Am I right?.

If so, this name is either the id of the control, which you will find in the source code, or by using FlexSpy http://code.google.com/p/fxspy/, but in both cases you need to be able to reach the code. You can also use the viable text on components, but not all components do have visual text.

I'm trying to get up and running on MacOS with FunFX. Is there anything I need to do in addition to the windows instructions? Any other gems I need?

I'm running into the following: Although I have the gems installed I'm getting a load error with require 'funfx'. I added require 'rubygems' which resulted in a load error for rbosa. I tried to install the rbosa gem (since I figured AppleEvents might be needed for Safari?) but that led to a number of other issues. Just want to check if there somewhere documenting getting FunFX running on MacOS.

I have not used FunFX on Mac, thus not sure what you need. Sorry about that. I think you should post a message on the mailing list for FunFX, I think you will reach more people that use Mac and FunFX there.

The documentation is not very good, I am sorry about that. We are now rebuilding FunFX so it will be easier to use.

But there is a mechanisme that renames the methods from SelectText to select_text to make it more Ruby like, but it should work both ways. But the error you get noe might be because om an error in rexml. Try to add this patch: http://groups.google.com/group/ruby-talk-google/msg/29820f7a7e444265

Hi peter !Actually i don't want to compile the application with Automation libraries and FunFXAdapter.swc for automation.

I m trying to use runtime loading...

Is dr possibility to do that?

I edited and used the runtimeloading.mxml which is in (flex_builder_install_dir/sdks/3.0.0/templates/automation-runtimeloading) directory and compiled it with Automation libraries and FunFXAdapter.swc using the Batch file.Then i edited the runtimeloding.html.

Then i put this runtimeloading.html, runtimeloading.swf and AutomationGenericEnv.xml into the web server along with the compiled application.

And called the runtimeloading.html for accessing.

I just tried this using FlexRepeater and execute the test_product_one.rb

i got the error as >ruby test_product_one.rbLoaded suite test_product_one Started E Finished in 11.859 seconds.

Batch file i used to compile: SET OPTS=-include-libraries+=E:\libs\FunFXAdapter.swc;E:\libs\automation.swc;E:\libs\automation_agent.swc;

..\..\bin\mxmlc.exe %OPTS% runtimeloading.mxml

One more thing when i tried to compile by creating new flex project and placing the runtimeloading.mxml in src and include the libraries in the compiler option, I got an error when loading in browser as:

I have not been using the swfloader that much, and I am not sure if this is possible to test with FunFX. Can you use FlexSpy to inspect if it is possible to insepct the elements within the loaded swf file from the main application?

Hi Peter ! Thanks for ur reply ... Flex spy inspect only the SWF loader and its not inspecting component of the swf file that i want to load.Now i m automating only by compiling the application with the all swc files.

Now, i m trying to click the menubar. For controlling the menubar only show event is available in the XML and I found unable to click the menu or the sub menu using the show command. Can u give any idea to click the menu or sub menu in the menu bar.

Hey Peter, need a small help. As it turns out that FunFX cannot select a particular row in a table unless I provide each and every values of all the cells separated by '|'. I was thinking of placing some hidden values such as an id against each of the rows in the table. This id will help me click on that particular row. Is it actually possible? Happy to discuss further.

This might look a silly question at this stage, but I really need an answer. I have an application readily built with flex3 (not built by me, my client has it). My task is to automate a huge set of test cases. I have tried exploring open flex automation tools available and found FunFx interesting as all other tools need some or other swf/swc to be placed with source files or in app sever while buiding the application. I assumed Funfx would not need this approach and started exploring the framework. Could you please confirm if this the same case with FunFx and it requires swc file to be placed with app source files, in which case my client wouldnt allow me to do so and Funfx may not be useful for automation to me. Quick response would really help me alot as I have already spent last 3 nights on Funfx which are effecting my job hours.

FunFX along with all other tools that are based on the automation framework from Adobe must be compiled with an additional swc. I should be possible to compile the already compiled swf with the new swc I think, I am not sure.

I am newbie to Funfx I am using Eclipse Classic SDK with Ruby & flex plugin,i am trying to run ruby application but getting following error.

C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- fastercsv (LoadError) from C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from C:/Ruby/lib/ruby/site_ruby/1.8/funfx/decoder.rb:2 from C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from C:/Ruby/lib/ruby/site_ruby/1.8/funfx.rb:4 from C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require' from C:/Ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from test_case.rb:2