The pencils’ down date for Google Summer of Code 2010 is right now. Here’s a quick overview of what I feel I have contributed to scxml-js thus far, and what I feel should be done in the future.

Tests and Testing Framework

Critical to the development of scxml-js was the creation of a robust testing framework. scxml-js was written using a tests-first development style, which is to say that before adding any new feature, I would attempt to map out the implications of that feature, including all possible edge cases, and would then write tests for sucess, failure, and sanity. By automating these tests, it was possible to avoid regressions when new features were added, and thus maintain robustness as the codebase became more complex.

Testing scxml-js was an interesting challenges with respect to automated testing, as it was necessary to test both the generated target code (using ahead-of-time compilation), and the compiler itself (using just-in-time compilation), running in all the major web browsers, as well as on the JVM under Rhino. This represented many usage contexts, and so a great deal of complexity was bundled into the resulting build script.

The tests written usually conformed to a general format: a given SCXML input file would be compiled and instantiated, and a script would send events into the compiled statechart while asserting that the state had updated correctly. A custom build script, written in JavaScript, automated the process of compiling and running test cases, starting and stopping web browsers, and harvesting results. dojo.doh and Selenium RC were used in the testing framework.

Going Forward

It would be useful to phase out the custom JavaScript build script for a more standard build tool, such as maven or ant. This may be challenging, however, given the number of usage contexts of the scxml-js compiler, as well as the fact that the API it exposes is asynchronous.

Another task I’d like to perform is to take the tests written for Commons SCXML and port them so that they can be used in scxml-js

Finally, I have often noticed strange behaviour with Selenium. At this moment, when run under Selenium, tests are broken for in-browser compilation under Internet Explorer; however when run manually, they always pass. I’ve traced where the tests are failing, and it’s a strange and intermittent failure involving parsing an XML document. I it think may be caused by the way that Selenium instruments the code in the page. I feel it may be worthwhile to investigate alternatives to Selenium.

scxml-js Compiler

This page provides an overview of what features works right now, and what do not.

In general, I think scxml-js is probably stable enough to use in many contexts. Unfortunately, scxml-js has had only one user, and that has been me. I’m certain that when other developers do begin using it, they will break it and find lots of bugs.

I’m hoping to prepare a pre-alpha release to coincide with the SVG Open 2010 conference at the end of the month, and in preparation for this, I’m reaching out to people I know to ask them to attempt to use scxml-js in a non-trivial project. This will help me find bugs before I attempt to release scxml-js for general consumption.

Going Forward

There are still edge cases which I have in mind that need to be tested. For example, I haven’t done much testing of nested parallel states.

I also have further performance optimizations which I’d like to implement. For example, I’ve been using JavaScript 1.6 functional Array prototype extensions (e.g. map, filter, and forEach) in the generated code, and augmenting Array.prototype for compatibility with Internet Explorer. However, these methods are often slower than using a regular for loop, especially in IE, and so it would be good to swap them out for regular for loops in the target code.

Another performance enhancement would be to encode the statechart’s current configuration as a single scalar state variable, rather than encoding it as an array of basic state variables, for statecharts that do not contain parallel states. This would reduce the time required to dispatch events for these types of statecharts, as the statechart instance would no longer need to iterate through each state of the current configuration, thus removing the overhead of the for loop.

I’m sure that once outside developers begin to look at the code, they will have lots of ideas on how to improve performance as well.

There are other interesting parts of the project that still need to be investigated, including exploring the best way to integrate scxml-js with existing JavaScript toolkits, such as jQuery UI and Dojo.

Graph Layout, Visualization, and Listener API

As I stated in the initial project proposal, one of my goal for GSoC was to create a tool that would take an SCXML document, and generate a graphical representation of that document. By targeting SVG, this graphical representation could then be scripted. By attaching a listener to a statechart instance, the SVG document could then be animated in response to state changes.

I was able to accomplish this by porting several graph layout algorithms written by Denis Dube for his Master’s thesis at the McGill University Modelling, Simulation and Design Lab. Denis was kind enough to license his implementations for release in ASF projects under the Apache License. You can see a demo of some of this work here.

Going Forward

The intention behind this work was to create a tool that would facilitate graphical debugging of statecharts in the web browser. While this is currently possible, it still requires “glue code” to be manually written to generate a graphical representation from an SCXML document, and then hook up the listener. I would like to make this process easier and more automatic. I feel it should operate similarly to other compilers, in that the compiler should optionally include debugging symbols in the generated code which allow it to map to a “concrete syntax” (textual or graphical) representation.

Another issue that needs to be resolved is cross-browser compatibility. It’s currently possible to generate SVG in Firefox and Batik, but there are known issues in Chromium and Opera.

Also, there are several more graph layout algorithms implemented by Denis which I have not yet ported. I’d really like to see this happen.

Finally, my initial inquiries on the svg-developers mailing list indicated that this work would be useful for other projects. I therefore feel that these JavaScript graph layout implementations should be moved into a portable library. Also, rather than generating a graphical representation directly from SCXML, it should be possible to generate a graphical representation from a more neutral markup format for describing graphs, such as GraphML.

Demos

I have written some nice demos that illustrate the various aspects of scxml-js, including how it may be used in the development of rich, Web-based user interfaces. The most interesting and complex examples are the Drawing Tool Demos, which implement a subset of Inkscape’s UI behaviour. The first demo uses scxml-js with a just-in-time compilation technique; the second uses ahead-of-time compilation; and the third uses just-in-time compilation, and generates a graphical representation on the fly, which it then animates in response to UI events. This last demo only works well in Firefox right now, but shows what should be possible going forward.

I have several other ideas for demos, which I will attempt implement before the SVG Open conference.

Documentation

The main sources of documentation now are the User Guide, the source code for the demos, and Section 5 of my SVG Open paper submission on scxml-js.

Conclusion

This has been an exciting and engaging project to work on, and I’m extremely grateful to Google, the Apache Software Foundation, and my mentor Rahul for facilitating this experience.

I worked really hard today, and I have lots to report. I started at 6:30am and worked straight through on GSoC until about 7:30pm. I also spent some time investigating alternate tooling, and I’ll talk a bit about this in another post.

So, to recap, we have this stack: Demos/GEF/Draw2d/SWT/JRE. The goal of this week is to get Draw2d to compile under GWT. After that, I’ll have to try to get it to run, but for now, I’m just concerned with getting it to compile. This, of course, means figuring out what Draw2d’s dependencies are in SWT and the JRE, and where they are not satisfied, writing out stub code and small TODO comments. Also, I will atempt, where possible, to reuse the existing, non-functional code from the previous project that used GWT to compile SWT to js.

To summarize my experiences of today: JRE == SUCCESS, SWT == FAIL.

First some background: GWT already provides some JRE emulation, but it is not sufficient to compile the SWT codebase. To get SWT to compile, one must at least implement certain stub interfaces that GWT does not provide. There are additional issues, such as reflection, that may be more serious, and I’ll talk about those in a later post. The extension to the GWT emulation classes live in the project org.eclipse.swt.e4.jcl. Left over from the previous project were three GWT modules which I was never able to compile. Additionally, it was unclear to me how these modules were supposed to compile, as they used the GWT module “source” tag, rather than the “super-source” tag, which, according to all of the documentation I have read, is what one is supposed to use when implementing JRE classes.

Today, I successfully compiled the previous project’s JCL classes. In order to accomplish this, I applied Technique 1 from my previous post, which is to say, I added a prefix (“e4″) to the JCL packages, where before there was no prefix, and I used <super-source path=”e4″/> in my GWT modules. I had to futz a bit to discover in which order I should import the modules, but after that, they compiled without complaint, w00t

//TODO: insert complete project structure here

Note that, since yesterday, I have been developing this prototype outside of the Eclipse environment. My toolset has consisted of GNU Screen, Vim, a Vim plugin called VimExplorer, and Ant. This has allowed me to ignore all of the IDE-specific confusion that I have so far encountered. My next post will be about my experiences using some of these tools.

I was feeling very optimistic after succeded in getting JCL to compile, so I set it as my goal to get the necessary SWT libraries to compile as well. Unfortunately, I failed in this task, because, I believe, I took the wrong approach. The approach I took was to attempt to reuse directly the SWT emulation classes left over from the previous project, just as I reused the previous project’s JCL classes. Unfortunately, these SWT emulation classes had many missing dependencies. My strategy was to just keep hacking on it (adding in missing classes, stub methods, etc.), until the thing worked. I spent most of the afternoon working on this, probably about 4 hours. Ultimately, I ended up pulling in most of the code in “org.eclipse.swt/Eclipse SWT/common”. I didn’t get to the end of the dependency chain, but I think I got far enough along to recognize a trend that I wasn’t happy with, namely, pulling in dependencies that in turn rely on other dependencies, etc. It should be a major priority to limit dependencies, because the more deps I cut out, the less complex my project will be. Given that this project is still at the prototyping stage, reducing complexity is of critical importance.

Note that I also tried compiling “org.eclipse.swt/Eclipse SWT/common”, but this pulled in some native code which GWT was unhappy with. So, I really do need to be selective.

The alternative to today’s approach is to start at the top of the stack (work top-down) and discover what the dependencies are for the Draw2d demos; then, find out what the dependencies are for Draw2d; then, very carefully, attempt to pull in those deps from org.eclipse.swt, or stub them out. I intend to use Doxygen to figure out the complete dependency chain. This will be tomorrow’s work.