Introduction

Like I’ve mentioned before in this blog, I develop Flex applications using the Flex SDK and jEdit instead of Adobe’s Flex Builder IDE. This setup has worked very well for me but one annoying issue I’ve had to deal with because of this is the slow compilation speed: every time I make a small change into one of my projects and recompile it, the mxmlc compiler (that my build scripts are calling) has to load the JVM into memory and recompile my whole project from scratch, which obviously takes a while. Compiling things in Flex Builder is a lot faster, and the reason for that is the Flex Compiler Shell, which it uses for compilation instead of mxmlc.

Update (Version 0.5.1, Mar 17, 09):

Display a “compiling…” message before sending fcsh a compilation command in order to let the user know that something is going on and that they should wait instead of just killing the process.

If the log file exists when fcshctl runs, check if there are any other fcshctl instances running, and if not, assume that the log file has been orphaned, delete it and continue.

Exit with status code 0 (“ok”) if fcsh output ends in the “Nothing has changed since the last compile” message.

Update (Version 0.5, Dec 10, 08):

Made it possible to specify a filename as the argument to the “clear” command — fcshctl will then try to find the first compile target id matching the specified filename and clear the corresponding id (example: “fcshctl clear MyApp.mxml”)

Made fcshctl find compile target ids corresponding to specific filenames or paths by asking fcsh directly via the “info” command instead of retaining them on its own in a temporary file (this is in order to avoid problems that could arise if this temp file and fcsh’s own list of retained compile target ids would get out of sync.)

Added command “id” which can be used to find the first compile target id matching a specified string (example: “fcshctl id MyApp.mxml”)

Update (Version 0.4, Nov 4, 08):

Made the script use only one logfile (i.e. one with a static path) and check its existence (and if it does exist, wait for it to be deleted) before continuing, so as to make sure no two fcshctl instances can try to send commands to fcsh at the same time, or before an already running command has finished executing and returned control to the prompt.

The basic idea behind the Flex Compiler Shell is that it would stay in memory between compilations (thus avoiding the cost of loading up the JVM every time you want to compile something) and use a technique called incremental compilation that, as the name suggests, is based on the idea of only compiling the parts of your project that have changed since the last compilation (thus making the process a lot faster). When you run the Flex Compiler Shell executable (called fcsh), you’re thrown into an interactive shell where you can run commands in order to compile your projects and control the operation of fcsh itself.

The Problem

When I found out about fcsh, tried it out and saw how much faster it was than mxmlc, of course I wanted to immediately start using it instead in my build scripts. The biggest problem with doing this, though, is the fact that the basic operation of these two programs is fundamentally different: when you run mxmlc, it compiles the project, prints some messages to standard output and then exits (with the exit status 0 if the compilation was successful or 1 if it wasn’t), which is perfect for integrating it into build scripts. Fcsh, on the other hand, throws you into a shell of its own and expects you to essentially operate it from within itself (instead of simply calling it with some arguments and then having it return with some standard output and an exit status like mxmlc).

The other issue that makes fcsh more difficult to use in general is that in order to use the incremental compilation feature, you have to first compile your project like you normally would (using the “mxmlc” command), then make note of the “compile target id” number that fcsh then assigns to this project and specifies in its output, and for future compilations use the “compile N” (where N is the compile target id) command.

The Solution

I tried to search for a solution to this problem that somebody else might have come up with but didn’t find an adequate one (a list of the ones that I could find is further down), so I decided to give it a shot myself. My approach, called fcshctl (short for “fcsh controller”), is basically a simple bash script that:

starts the fcsh process if it isn’t already running

forwards its arguments to fcsh as commands

reads the output of that command from the fcsh interactive shell and prints it to the standard output

exits with status 0 (success) or >0 (failure)

In addition to this, I also made the script automatically and transparently invoke the incremental compilation feature whenever applicable so that I wouldn’t have to worry about keeping track of the compile target ids and using a different command for the compilation whenever one had already been assigned.

Below is a summarization of the pros and cons of this solution:

Pros:

It’s quite simple — just one shell script (a few others are included in the .zip that are only for convenience and not required)

The running fcsh instance is accessible to all user processes (i.e. not just within a single shell session, for example)

It is possible for the user to enter the running fcsh interactive shell at any time (by attaching to the screen session via the command screen -r -S fcsh) and manually interact with it

It automatically checks whether you’ve compiled a specific file before and keeps track of the compile target ids that fcsh assigns so that it can automatically and transparently invoke the incremental compilation feature and let you access the speed gains that are fcsh’s main offering without any additional work. This feature can also be turned off by setting an environment variable.

A proxy script (called fcshctl-mxmlc) that can be used in place of mxmlc is provided. It simply calls fcshctl with “mxmlc” as the first argument, followed by all the other arguments you specify, so that you can simply use this script as a stand-in for mxmlc in any build scripts that you may be using without having to modify them any further.

Cons:

It uses the GNU screen logging mechanism and polling for changes in the log file in order to catch the output of commands sent to fcsh, which is not very elegant and causes a slowdown of maybe about a couple hundred milliseconds before the output is printed to stdout.

The output of the executed fcsh command is not printed to the standard output as it comes, but instead all at once when the command finishes (this is not a problem for me personally, but others might want to fix this).

It’s implemented as a bash script, which means that it’s not the fastest monster out there (we’re talking about milliseconds here, though, so it doesn’t bother me at all).

Download

fcshctl has been tested with fcsh versions 3.0.0 build 477 and 3.1.0 build 2710 on Mac OS 10.5.5. It is licensed under the MIT License.

The script requires the following to be in your $PATH: rm, mv, cat, ps, sleep, tail, head, grep, awk, screen, xargs and basename. You also need to have the FLEX_HOME environment variable set to point to the Flex SDK root path.

Other similar solutions

Below is a list of other approaches to the problem of integrating fcsh to non-standard (i.e. non-Flex Builder) workflows (roughly in order from most promising to least promising), and short lists of reasons why I personally didn’t want to use them. If my solution doesn’t seem good to you, you should maybe check some of these out.

Comments

Thanks for this extremely useful utility! I have been using it for a bit and it has been pretty stable. I have written a follow-up article to yours which explains how to set up the Error List plugin to listen to fcshctl’s output.

I tried running your fcshctl utility. It worked fine on my Ubuntu 8.04.1 desktop machine (except I have to rm the log file as Chris Hill reported above). However, when I tried to run it on an old server running Debian Etch, it hangs up even when trying to run the help message

i.e.

./fcshctl help

This produces the following error messages:
./fcshctl: line 227: screen: command not found
./fcshctl: line 126: screen: command not found
./fcshctl: line 132: screen: command not found
./fcshctl: line 133: screen: command not found
./fcshctl: line 153: screen: command not found

If you can give me an idea as to what might be causing this, I will try to track down what might have happened. I am able to run the fcsh utility from the SDK, so the SDK itself runs okay.

Thanks for putting out this utility. I will probably try placing it on a better server when I get a chance (as the Debian Etch machine I tried it on is an old machine and it may also be missing some dependencies).

I made some changes to the script (see the changelog at the beginning of the post.) You should try the new version to see if it fixes the lock-ups you’ve been having due to the logfile not being removed correctly.

Hi ali,
I have installed the new version, I’ll tell you if I see any improvement. I hope I haven’t made it out to be a big deal, as it happens about once a week max. Its so infrequent it makes it hard to debug :)

Thanks for figuring out that I was missing GNU screen. After installing this package everything works. I have tried the new version (v0.5) and it seems to have cleared up the screen log issue. Great work!

Thanks for the suggestions. I uploaded a new version that checks for other running fcshctl processes using ps and if no other instances seem to be running, deletes the log file (assuming it has been orphaned for example due to a previous fcshctl process being killed before it had had a chance to remove the log file) and continues. It also displays a “compiling…” message like you suggested.

If you’re using Windows, I suggest you check out the other options I listed here (at the bottom of the post), or just search Google for some other solution that’s designed for Windows. You can of course install some sort of Unix environment (like Cygwin) onto your Windows box for running this, but it doesn’t seem like a smart way to go about it in my opinion.

Hi! I was quite inspired by your script, great job thanks. The only downside is the fact it’s *nix only. For this reason I created a cross-platform version of a similar wrapper around fcsh – http://efiquest.org/2010-09-28/50/