Thursday, April 15, 2010

Scripting with Lua

For about two weeks, I've been planning to put together a way to generate test cases for one of my current projects. The project involves adding new capabilities to the RTEMS scheduler, and to test it I need to create RTEMS applications that spawn various periodic and aperiodic tasks. I already had written two such applications by hand, but I found it tedious to generate the parameters.

At a high level, the parameters are task type (periodic or aperiodic), duration (execution time), period, relative deadline, and release time (a.k.a. phase). To spawn tasks, I had some arrays defined in my RTEMS code of length equal to the number of total tasks. These arrays hold the values of the period, priority (relative deadline), release time, and execution time. Then my code uses the arrays to generate tasks with the given parameters. This isn't too difficult to write up by hand, but my project also requires that I compute some runtime characteristics of the periodic tasks.

In particular, I am implementing a static slack stealer in RTEMS, which requires a pre-computed table of all the jobs released in a hyperperiod. A job is an instance of a periodic task, which is released at a specific time and has an absolute deadline by which it must complete. The absolute deadline of a job is equal to the release time plus the task's relative deadline. A hyperperiod is a length of time equal to the least common multiple of all the periods of periodic tasks, which is the time it takes for the system to "cycle" the periodic tasks. At the start of a hyperperiod, the periodic tasks execute identically the same as at time 0 (or the start of the first hyperperiod).

The pre-computed table involves computing for each job in a hyperperiod the slack of the job, which is the difference between the job's deadline and the sum of all the execution times of jobs with earlier deadlines. Then each w(i,j) entry of the pre-computed slack table w is the minimum of jobs with deadlines greater than or equal to i and less than or equal to j. The slack table is an N x N upper-triangular matrix, where N is the number of jobs in a hyperperiod. Calculating these values by hand is tedious, so I decided to implement a script to automatically generate the pre-computed slack table for a given set of periodic tasks.

I don't do very much scripting work, so I have no preference on language. Most of the time I can get away with shell scripts. But in this case, I wanted to use a language with more support for functions and string processing. I settled on either Lua and Python, and eventually went with Lua. Lua is a dynamically typed language, which is designed to be embedded in C programs. This was the tipping factor for me, I'm curious to see if there are any useful use cases for Lua in future projects. I've heard you can do the same with Python, but that it can be difficult.

For this project, I used Lua solely as a stand-alone interpreter, rather than embedding my Lua scripts in a C program. My first task was to learn Lua. I spent a morning reading the Lua reference manual, which was accessible and easy enough to get through. One of the interesting quirks of Lua is that it uses a primitive type table as the only way to implement arrays and dynamic structures!

Tables are relatively straightforward, and the language provides built-ins to manipulate the tables. I found two surprises while working with tables: first, tables are one-indexed by default (rather than zero-indexed like C arrays); and second, nil values cannot be added to tables. However, tables are extensible types, and one can use metatables to define how operations, like <, work on tables. This could be important, since there is no default way to compare two tables. Adding elements to tables is easy with the table.insert function, which by default appends to the table, but can also be used to insert relative to a table element. The table.sort operation takes a function as its second argument that defines the less than operation to use for the sort. table.concat was another useful function, which converts a table to a string, and adds an optional delimiter between each entry of the table. For example,

table.concat(t, ", ")))

converts t to a comma-delimited string.

The last function that was really helpful (and non-obvious) is unpack, which returns the elements of a table as a "list", which can be used as a varargs argument to a function. For example, I found some code to implement the map function:

Note that for x, y in ipairs(T) defines an iterator over T, where x is the key (index) and y is the value T[x]. In the lcm function, I convert the varargs argument to a table (which would discard nils!) and iterate over it. Most of the functions I wrote make use of the ipairs method to create an iterator to process a table.

My script starts by parsing its arguments, which take the form -T d,e,p or -A d,e,r; T indicates a periodic task, and A indicates an aperiodic task. d is for deadline, e is for execution time, and r/p is for release time/phase. The parameters are used identically, but I need to differentiate between periodic tasks, which are used to compute the slack table, and aperiodic tasks, which are included only because the script will generate the parameters needed to run the aperiodic task in the RTEMS test program.

tasks = parse_args(arg)

Next, the script sorts the tasks, so that periodic and aperiodic tasks can be differentiated:

table.sort(tasks, task_lt)

where task_lt is a function that defines how to compare the strings held in the table tasks, extracted by parse_args.

The script then splits the tasks table into aperiodic and periodic tasks:

All of the jobs of a hyperperiod are in J, which is a sorted table of tables, where the table at J[i], i = 1 to n, has entries J[i][1] = deadline of job i, J[i][2] = execution time of job i, and J[i][3] = release time of job i. The table is sorted by earliest deadline, with ties broken by earliest release. Now the script can compute the slack:

The write_header function makes a lot of use of the string.format function, which acts like sprintf (only safer!), and the table.concat function to generate comma-delimited array initializer lists from tables.

An example execution is:

lua gen-slack-header.lua -T 6,1,0 -A 100,5,4 -T 12,1,0

Which generates parameters for two periodic tasks, period (relative deadline) of 6 and 12, and with execution time of 1 and phase of 0. Parameters for an aperiodic task with deadline 100, execution time of 5, and release time of 4 are also generated. This is what the resulting C header file looks like:

/* slackparams.h * * This is a generated file. * Use the gen-slack-header.lua script to create this file. */