Lua, Lua, Oh, Oh

Compiled programming languages and scripting languages each have unique advantages, but what if you could use both to create rich applications? Lua is an embeddable scripting language that is small, fast, and very powerful. Before you create yet another configuration file or resource format (and yet another parser to accompany it), try Lua.

Compiled programming languages and scripting languages each have unique advantages, but what if you could use both to create rich applications? Lua is an embeddable scripting language that is small, fast, and very powerful. Before you create yet another configuration file or resource format (and yet another parser to accompany it), try Lua.

While interpreted programming languages such as Perl, Python, PHP, and Ruby are increasingly favored for Web applications and have long been preferred for automating system administration tasks, compiled programming languages such as C and C++ are still necessary. The performance of compiled programming languages remains unmatched (exceeded only by the performance of hand-tuned assembly), and certain software — including operating systems and device drivers — can only be implemented efficiently using compiled code. Indeed, whenever software and hardware need to mesh seamlessly, programmers instinctively reach for a C compiler: C is primitive enough to get “close to the bare metal” — that is, to capture the idiosyncracies of a piece of hardware — yet expressive enough to offer some high-level programming constructs, such as structures, loops, named variables, and scope.

However, scripting languages have distinct advantages, too. For example, after a language’s interpreter is successfully ported to a platform, the vast majority of scripts written in that language run on the new platform unchanged, free of dependencies such as system-specific function libraries. (Think of the many DLLs of the Windows operating system or the many libc’s of Unix and Linux.) Additionally, scripting languages typically offer higher-level programming constructs and convenience operations, which programmers claim boost productivity and agility. Moreover, programmers working in an interpreted language can work faster, because the compilation and link steps are unnecessary. The “Code, Build, Link, Run” cycle of C and its ilk is reduced to a hastened “Script, Run.”

Like all engineering pursuits, choosing between a compiled language and an interpreted language means measuring the pros and cons of each in context, weighing the trade-offs, and accepting compromises. There is no correct answer, except perhaps, “It depends… “

Mixing the Best of Both Worlds

But what if you could enjoy the best of both worlds: close to bare metal performance and high-level, powerful abstractions? Better yet, what if you could optimize algorithms and functions that are processor-intensive and system-dependent, as well as separate logic that’s system-independent and highly susceptible to changes in requirements?

Balancing the need for high-performance code and high-level programming is the purview of Lua, an embeddable scripting language. Applications that include Lua are a combination of compiled code and Lua scripts. Compiled code can drop to the metal when necessary, yet can call Lua scripts to manipulate complex data. And because the Lua scripts are separate from the compiled code, you readily and separately revise the scripts. With Lua, the development cycle is more akin to “Code, Build, Run, Script, Script, Script… “

For example, the Lua Web site “Uses” page lists several mass-market computer games, including World of Warcraft and (the home console version of arcade classic) Defender, that integrate Lua to run everything from the user interface (UI) to the artificial intelligence (AI) of foes. Other applications of Lua include an extension mechanism for the popular Linux software update tool apt-rpm and the control of the “Crazy Ivan” Robocup 2000 champion. Many of the testimonials on that page laud Lua’s small size and excellent performance.

Getting Started with Lua

As of August 2007, the most recent stable version of Lua is 5.1.2. You

If you use Debian Linux, you can install Lua 5.1.1 quickly and easily by running the following command:

# apt-get install lua5.1

All the examples shown here were run on Debian Linux “Etch” using Lua 5.1.1 and the 2.6.18-4-k7 Linux kernel.

After you’ve installed Lua on your system, give the stand-alone Lua interpreter a try. The entire Lua core, including the standard libraries and the Lua compiler, is less than 200 KB in size. (All Lua applications must be embedded in a host application. The interpreter is simply a special kind of host, useful for development and debugging.) Create a file called factorial.lua, and enter the lines:

Lua has many of the conveniences found in a modern scripting language: scope, control structures, iterators, and a host of standard libraries for processing strings, emitting and collecting data, and performing mathematical operations. You can find a complete description of the Lua language in the Lua 5.0 Reference Manual.

Like every scripting language, Lua has its own peculiarities:

•In Lua, values have types, but variables are dynamically typed. The nil, boolean, number, and string types work mostly as you might expect. Nil is the type of the special value nil and is used to represent the absence of a value. Boolean is the type of the constants true and false. Nil also represents false, and any non- nil value represents true.) All numbers in Lua are doubles (but you can easily build code to realize other numeric types). A string is an immutable array of characters. (Hence, to append to a string, you must make a copy of it.)

•Tables are the catch-all data structure in Lua. Indeed, tables are the only data structure in Lua. You can use a table as an array, a dictionary (also called a hash or an associative array), a tree, a record, and so on. Unlike other programming languages, the contents of a Lua table need not be heterogeneous: The table can include any combination of types and can contain a mix of array-like elements and dictionary-like elements. Moreover, any Lua value — including a function or another table — can serve as a dictionary element key. To explore tables, start the Lua interpreter and type the lines prefaced by the prompt in Listing One.

As you might expect, Lua also provides a fair number of iterator functions to process tables. The global variable table provides the functions (yes, Lua packages are just tables, too). Some functions, like table.foreachi(), expect a contiguous range of integer keys starting at 1 (the numeral one):

While some iterators are optimized for integer indices, all simply process (key, value) pairs. For fun, create a table, t, with elements {2, 4, 6, language="Lua",version="5",8, 10, 12, web="www.lua.org"}, and run table.foreach(t, print) and table.foreachi(t, print).

•The table, function, and thread types are all references. Each can be assigned to a variable, passed as an argument, or returned from a function. For instance, here’s an example of storing a function:

•A thread is a co-routine created by calling the built-in function coroutine.create(f), where f is a Lua function. Threads do not start when created; instead, a co-routine is started after the fact, using coroutine.resume(t), where t is a thread. Every co-routine must occassionally yield the processor to other co-routines using coroutine.yield().

•Lua allows multiple assignments, and expressions are evaluated first and are then assigned. For example, the statements…

… produce 4 7 5 nil nil. If the list of variables is larger than the list of values, excess variables are assigned nil; hence, b is nil. If there are more values than variables, extra values are simply discarded. In Lua, variable names are case-sensitive, explaining why I is nil.

•A chunk is any sequence of Lua statements. A chunk can be stored in a file or in a string in a Lua program. Each chunk is executed as the body of an anonymous function. Therefore, a chunk can define local variables and return values.

•Lua has a mark-and-sweep garbage collector. As of Lua 5.1, the garbage collector works incrementally. Lua has full lexical closures (like Scheme and unlike Python). And Lua has reliable tail call semantics (again like Scheme and unlike Python). You can find more examples of Lua code in the Programming in Lua book and in the Lua-users wiki.

Because Lua is intended to be embedded in a host application written in a language such as C or C++ and is intended to cooperate with the host application, data must be shared between the C environment and Lua. As the Lua 5.0 Reference Manual says, the userdata type allows “arbitrary C data to be stored in Lua variables.” You can think of userdata as an array of bytes — bytes that might represent a pointer, a structure, or a file in the host application.

The contents of userdata originates with C, so it cannot be modified in Lua. Of course, because the userdata originates in C, there are no pre-defined operations for userdata in Lua. However, you can create operations that operate on userdata using another Lua mechanism, called metatables.

Metatables

Because tables and userdata are so flexible, Lua permits you to overload operations for objects of either type. A metatable is a (normal) Lua table that maps standard operations to custom functions that you provide. The keys of the metatable are called events; the values (in other words, the functions) are called metamethods.

The functions setmetatable() and getmetatable() modify and query an object’s metatable, respectively. Each table and userdata object can have its own metatable.

For example, one event is __add, for addition. Can you deduce what this chunk does?

The function String() takes a string, string, wraps it in a table ({value=s or’’}), and assigns the metatable mt to the table. Function mt.__add() is a metamethod that appends the string b to the string found in a.valueb times. The line print((s+’There’+’World!’).value) invokes the metamethod twice.

__index is another event. The metamethod for __index is called whenever a key doesn’t exist in a table. Here’s an example that” memoizes,” or remembers, the value of a function:

Put the code into the Lua interpreter, and then type print(color[1],color[2],color[1]). You should see something like blue black blue.

This code, given a key, node, looks up the color for the node. If it doesn’t exist, the code permanently assigns node a new, randomly chosen color. Otherwise, the node’s assigned color is returned. In the former case, the __index metamethod is executed once to assign a color. The latter case is a simple and fast hash lookup.

The Lua language offers many other powerful features, and all of them are well documented. But in case you ever run into trouble or want to talk to a wizard, head over to the Lua Users Chat Room IRC Channel at irc://irc.freenode.net/lua for some very enthusiastic support.

Embed and Extend

Beyond Lua’s simple syntax and powerful table structure, Lua’s real power is evident when mixing it with a host language. As has been intimated, Lua scripts can extend the host language’s own capabilities. But the reciprocal is true as well: The host language can simultaneously extend Lua. C functions can call Lua functions and vice versa, for example.

At the heart of the symbiotic relationship between Lua and its host language is a virtual stack. The virtual stack — like a real stack — is a last in-first-out (LIFO) data structure that temporarily stores function arguments and function results. To make a call from Lua to its host language and vice versa, the caller pushes values onto the stack and calls the target function; the callee pops the arguments (verifying the type and value of each argument, of course), processes the data, and pushes the result on the stack. When control returns to the caller, the caller extracts the return values from the stack.

Virtually all the C API for Lua operations operate through the stack. The stack can hold any Lua value; however, the type of the value must be known to the caller and callee, and specific functions push and pop each type from the stack (such as lua_pushnil() and lua_pushnumber().

Listing Two shows a simple C program (taken from Chapter 24 of the Programming in Lua book) that implements a minimal but functional Lua intrepeter.

Lines 2-4 include the Lua standard functions, several convenience functions that are used in all Lua libraries, and functions to open libraries, respectively. Line 9 creates a Lua state. All states are initially empty; you add libraries of functions to the state using luaopen_…(), as shown in Lines 10-14.

Line 17 and luaL_loadbuffer() take the input from stdin as a chunk and compile it, placing the chunk on the virtual stack. Line 18 pops the chunk from the stack and executes it. If an error occurs during execution, a Lua string is pushed on the stack. Line 20 accesses the top of the stack (the top of the stack has an index of -1) as a Lua string, prints the message, and then removes the value from the stack.

Using the C API, your application can also “reach” into the Lua state to extract information. This snippet grabs two global variables from the Lua state:

Again, notice that the stack enables the transfer. Calling any Lua function from C is similar to this code: grab the function using lua_getglobal(), push arguments, make a lua_pcall(), and then process the results. If a Lua function returns n values, the first value is at location -n in the stack, and the last value is at position- 1.

The inverse — calling a C function from Lua — is similar. If your operating system supports dynamic loading, Lua can load and call functions on demand. (In operating systems where static loading is a necessity, extending the Lua engine to call a C function requires you to rebuild Lua.)

Lua is Out of This World

Lua is an incredibly easy language to pick up, but its simple syntax disguises its power: the language supports objects (which are similar to Perl’s), metatables make its table type quite malleable, and the C API allows great integration and extension between scripts and the host language. Lua has been hosted in the C, C++, C#, Java, and Python languages.

Before you create yet another configuration file or resource format (and yet another parser to accompany it), try Lua. The Lua language — and its community — is robust, inventive, and ready to help.

Comments on "Lua, Lua, Oh, Oh"

clowenstein

Programming examples in Listing One and Listing Two have been garbaged up by text reformatting that ignored newlines in the original. Really hard to decipher.

Squirrel (http://squirrel-lang.org/) is a language designed following Lua in its internal architecture but supporting a C/C++ style syntax with classes. It may be attractive to people who find Lua too minimalistic.

I was just searching for this info for some time. After 6 hours of continuous Googleing, at last I got it in your site. I wonder what’s the lack of Google strategy that do not rank this type of informative websites in top of the list. Generally the top sites are full of garbage.

Hey! This post couldn’t be written any better! Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this write-up to him. Fairly certain he will have a good read. Many thanks for sharing!

“Hi there would you mind letting me know which webhost you’re working with? I’ve loaded your blog in 3 different browsers and I must say this blog loads a lot faster then most. Can you suggest a good internet hosting provider at a honest price? Cheers, I appreciate it!”

Hello! This is my first comment here so I just wanted to give a quick shout
out and say I really enjoy reading your
posts. Can you suggest any other blogs/websites/forums that go over the same topics?
Many thanks!