Compilation Struggles with Java 6 ScriptEngine, JavaScript, and Maven

One of the projects I am currently on is to create a Java based headless browser that I can easily control programmatically. I have wanted this for a couple of reasons over the years, one for navigating websites when trying to scrape information and secondly for use in automated functional testing. I love the Selenium IDE but I find that it is impossible to automate your Selenium scripts into an automated build because it requires an actual browser (e.g. Firefox, Internet Explorer, etc.) to be started and executed each time. So I want a headless browser that will do navigation, manage the DOM model of the page, and execute JavaScript to handle AJAX calls. (SIDENOTE: I am planning on open sourcing this and am waiting for approval from SourceForge to move the code out there).

To support JavaScript, I wanted to just use the ScriptEngineManager found in Java 6 to execute JavaScript. Well, one problem that I found is that it is a real pain to get at objects that have been created in an executed script. Consider the following JavaScript example:

OK, so far so good. Now let’s say you want to retrieve the value of foo from the ScriptEngine environment and iterate over the values. The ScriptEngine class has a get method, however, this returns an Object of type sun.org.mozilla.javascript.internal.NativeArray, not a Java Array. So to access values, you will need to cast the Object to a NativeArray. Then, as you iterate the array, you might have to cast each value to a NativeObject as the get method on NativeArray only returns an object.

Now your code is relying on a Sun internal class and not a base JDK class. The appropriate classes exist in the $JAVA_HOME/lib/rt.jar file. Unfortunately, this jar file is NOT used by default when running javac. So to get my code to compile, I had to use the rt.jar in my bootclasspath. Since I am using Maven 2.0 as my build system, I had to add the following to the pom.xml where I am defining my compiler

This put the rt.jar into the bootclasspath when building and allowed my code to compile. What I had hoped would be a simple task turned out to be a real ordeal.

Using the sun.org.mozilla.javascript.internal classes is very, very ugly and has the added distraction that it is hard to build and will be hard to maintain. I do not see a way around the problem and I have found other people having similar problems working with the scripting API. If anyone has a better approach, I am all ears.

The second technique I’ve used to avoid adding rt.jar to a build classpath is to use Class.forName trickery. Code the “sun.xxx.yyy.Zzz” class as a string in Class.forName and cast the new instance to a generic super type that isn’t in rt.jar. It sounds like this technique might not work for you, short of casting the instance to an Object, if the NativeArray class can’t be cast somehow to a java.lang.reflect.Array type or equivalent.

Trying out Cappuccino

Renoir and Arthritis

Went to the Late Renoir exhibit at the Philadelphia Museum of Art today. Was amazed at how arthritis crippled his hands and yet he was still able to persevere and still make beautiful things. Very motivating.

Rails / MySQL / Concurrency

Learned a lot about Rails, MySQL and concurrency. Yehuda wrote a great post on threads that was very helpful interesting. Wound up prototyping with MySqlPlus and MySQL2. I now understand why Merb was created and wished my application was based on it rather than Rails. I am not looking forward to the work that it will take to migrate to Rails 3.

if statements do not introduce scope in Ruby.

The Missing Rule of Productivity

In my browsing travels today, I stumbled upon the Rules of Productivity. Really interesting presentation and many good rules of thumb. But one thing I believe they missed.

One of the key conclusions of the presentation is that working > 40 hrs per week will result in a short performance boost followed by a productivity drag. So you should always try to have your teams work 40 hour weeks, right.

Well one thing that is not considered in the presentation is that not all days can be considered equal. There are periods where pushing the team harder in the short-term can result in huge wins, even if you have to pay for it during a refractory period after the exertion. Some examples of this:

Your company has an opportunity to present at once-a-year conference/trade show coming up. A burst of productivity ahead of this may be worth sacrificing productivity from the team after

There is a scheduled vacation coming up. Should we ramp up the week before Thanksgiving or Christmas knowing that the next week is going to result in sub-optimal performance anyhow?

The Great Derangement and what I learned about Congressional earmarks.

Last night/this morning read The Great Derangement by Matt Taibbi. Taibbi is one of my favorite writers, especially when he is revealing the relationship between corporate influence buyers and politicians. What I would recommend reading in the book are his interludes on how laws actually get made (hint, not like this, which of course inspired this fantastic Simpsons clip). Learning about how earmarks actually get put into a bill, about how the Rules Committee works, was fantastic and why you want to read a book like this.

As for the overall book, very disappointed. The parts on evangelical Christians were predictable, the parts about the Truthers just sad. Reminds me much more of Malcolm Gladwell’s stuff; Taibbi has taken what would have been a fascinating 10 page magazine articles and stretched it out into a book.