Cross Testing

From Gnash Project Wiki

When Gnash was a new project, it had few test cases beyond running SWF files and seeing what happened. Using the free Ming SWF compiler, it became possible to create unit tests for specific functionality and bugs. At that time it was decided to use [DejaGnu http://www.gnu.org/software/dejagnu/], as I was the primary author, and was designed to support cross testing on embedded hardware for the GNU toolchain. Although for quite time none of the tests were actually setup to be built cross, till I recently decided to do this for Android.

To duplicate any of this work, you'd need to use the toolchain available off this web site, as that's what I used instead of the NDK. The NDK from Google doesn't support C++ very well. These tests have been run using both real Android hardware, but I primarily used the Android emulator.

It got interesting, as Gnash's testsuite is very complex. It often builds an executable, which has to be executed to produce files for testing. To start, I had to introduce the same concepts used for the GNU toolchain, which is there are three systems involved. These are the build machine (where I'm sitting), one is the host (where the code gets executed), and the final one is the target. We don't use target, because in a sense it's SWF, and Gnash is the machine.

Once this support for configuring with multiple compilers was done, I then separated the test cases in the Makefiles between build and host executables. Most of the time this was pretty simple, as the library unit and API tests were nicely self contained executables that only ran on the host machine. The Ming and Haxe based tests were the most complex, as the initial test gets compiled for the build machine, then it gets executed to produce the SWF file. Then there is a "Test Runner" wrapper that gets compiled for the host machine with the cross compiler.

After all this compiling, the test case and it's SWF file (if it has one) get downloaded to the device, and then executed remotely using 'adb'. The existing test driver then just parses the output from the test case.