Search

Testing Applications with Xnee

Xnee can record user actions during a session
and then replay those actions. By recording sessions when testing a
program, Xnee automatically can test the program for you later. These test
sessions can be replayed before every release, or every night, to ensure
the quality of your program. Is it as easy as it sounds? Almost.

Xnee doesn't test only GUIs. You also can use Xnee to
test command-line programs by making a few test scripts that test all the
options of a command-line program and analyze the results. Xnee also has
been used to test how much traffic is being sent over a large network with
numerous thin clients. Support for distributing events to multiple
displays has been added if you want to test the same cases on multiple
machines at the same time. Besides testing programs, Xnee also is used to
demonstrate programs. In this case, Xnee acts as a patient demonstrator,
doing the same job over and over again without complaining.

The History of Xnee

In 1997, Henric Johansson and I wrote our master's thesis
on recording and replaying X events. We implemented a nonfree recorder
and replayer for a Swedish company for its internal purposes. After
finding
a job, I often lacked a free test program for X11, so I decided to
implement one on my own, using the experience gained from the thesis. The Xnee
Project started in the summer of 1999 and was licensed under GPL from
the start. In November 2002, version 1.0 was released, and by the end of
February 2003, Xnee was dubbed a GNU package.

Introduction

Before we go on with Xnee, this short introduction to X explains a
lot of the terms used in this article. X is a window-based user interface
system for various platforms. The X server is a program that handles all
the hardware and actually does the drawing on the screen. On GNU/Linux
systems, XFree86 is the most-used X server. X programs are known as
clients; examples are xterm and Galeon. The clients communicate
with an X server using the X protocol.

In this article we concentrate on the packets used to send information
between the X server and its clients. These packets are called Event,
Request, Reply and Error and are referred to here as protocol data. The
following list shows the X11 protocol data description:

Request is sent by the client to the server. The
server is asked to perform some action or to send some information.

Reply is sent by the server to the client as
a response to some request from the client. Not all requests result in
a reply.

Event is sent by the server to the client to
inform the client of user input or that something has happened that the
client may want to do something about, for example, the client is out of
focus.

Error is sent by the server to the client if a
request wasn't valid.

The most interesting thing here is every time the user interacts
with the computer using the mouse or the keyboard the X server
sends the appropriate client one or more events. Some of these events
result directly from user input. These events are referred to as device
events. The device events are ButtonPress, ButtonRelease, MotionNotify,
KeyPress and KeyRelease. If we could record all of these events during a
session, we would have a complete description of all the actions the user
performed. If we had a robot that could read these events if they were
printed to a file or on paper, the robot then could interact with the system
as the user did when recording, and we would have ourselves a test robot.
Or,
even better, if we had support for faking those events, we would have
a test robot made of software. Fortunately, support
exists for both recording and replaying in X.

To record X protocol data we can use the extensions RECORD or XTrap. There
are other ways to accomplish recording, such as sniffing the X socket,
but we'll focus on RECORD as it's what Xnee uses. To
replay events, we can use both the XTest extension and the RECORD
extension. During replay, the RECORD extension is used to synchronize
what's happening when replaying with what happened when the session
was recorded.

The RECORD extension sends copies of the data sent between the clients
and the server to the client requesting it. Using the RECORD extension,
Xnee can record all protocol data the user wants and save it to a
file to replay later.

The XTest extension can reproduce or fake all device events. This
extension lets Xnee fake user actions, such as moving the pointer,
pressing and releasing a key or pressing and releasing a button. No other
data can be replayed.

Xnee Installation

Xnee is a GNU
package, and the sources can be found at the Xnee home page. Download the
latest source; as of this writing, the latest version is 1.0.6. Extract
the package, and then configure, build and install it:

RPM packages are available at the home page, and Xnee also
is available in the FreeBSD ports tree. Xnee comes with a user manual and
a developer manual in various formats. The TeX sources to the manuals are
distributed with Xnee and covered under GNU FDL. Instead of building
the documents yourself, you can download the format of your choice (PDF,
HTML, INFO and TXT) from the
Xnee home page. As of this writing, the Xnee documentation version is
1.0.4. Extract the documents:

tar zxvf xnee-doc-1.0.4.tar.gz

When running Xnee, make sure the RECORD extension
is enabled. On XFree86 make sure the RECORD module is loaded. Open the
XFree86 configuration file, typically /etc/X11/XF86Config-4, and look in
the Module section. The following line should be included:

Load "record"

See the Xnee FAQ for more information about this.

Simple Usage Examples

We don't go into any details about Xnee in this
section, but rather begin slowly with a simple example. Start
Xnee with the --all-events option. This sets up the recording of a few
events. The option is not useful when doing serious Xneeing. It is
intended to simplify your introduction to using Xnee:

xnee --all-events

When moving the mouse or pressing the buttons or keys, Xnee prints
information about the action. We move on to record a simple session
that we replay immediately. To record 20 mouse motions, start Xnee
like this:

The options on the command line mean use recording mode
(--record),
save output in a file called session1.xnr (--out session1.xnr), record
the device-event MotionNotify (--device-event-range
MotionNotify) and
record 20 events (--loops 20).

To replay this event, start Xnee like this:

xnee --replay --file session1.xnr

The options on the command line mean use replay mode
(--replay),
and read data to replay from file session1.xnr (--file
session1.xnr).

Setting Up Recording Ranges

Xnee uses ranges to explain what is to be recorded. Ranges have a start
value and a stop value. The following data can be recorded: core-requests,
device-event, delivered-event, error, reply, extension requests and
extension replies. We don't describe the above data in this article. If
you want to read more, see the RECORD extension documents. When, for
example, you want to record the device event MotionNotify, use:

--device-event-range MotionNotify

To record the events from KeyPress up to MotionNotify and
CreateNotify, use:

--device-event-range KeyPress-MotionNotify,\
CreateNotify

You can use the number corresponding to the event name instead of
the name itself if you want shorter command lines. To find the number for the
data you want to record, use Xnee's --print-data-name option:

xnee --print-data-name

Stopping Xnee

You can stop recording by setting the number of the data to record
(--loops option), or you can interrupt Xnee by sending a TERM signal
(pressing Ctrl-C in the terminal window where you started Xnee).
Alternatively,
you can dedicate a modifier and key combination that won't
be used to do anything else during recording. Setting the modifier and key
is done with the --stop-key option. To set up Xnee so it stops recording when
Ctrl-Alt-A are pressed, add the following to the command-line option:

--stop-key Control+Alt,a

Synchronize

But why even bother to record data other than device events
when you can't replay it? Xnee uses that other data to
synchronize, which is where things get complicated. Think of recording a session
when using Galeon or any other Web browser. When recording, everything goes
well and the network is up and running. But when replaying the Galeon
session, you can't reach the Internet. If not for synchronization, Xnee
might replay user events such as clicking on a link on the Web page. If
Galeon could not load the page, it is not useful to continue the replay
until the network is up and the page can be loaded.

When recording
other data, we can use it to synchronize the session. For example, if we record the
data sent when displaying the Web page in the Galeon window, we can wait
for the same data to be sent when replaying. This ensures that the Web page
is loaded before we go on and replay the coming events. In this example,
we skip a lot of the X protocol data sent when recording in order to
keep it simple (see Table 1). When replaying this simple session,
Xnee uses the same events (see Table 2).

Table 1. X Events at the Start of a Galeon Test

Protocol Data Name

User or Client Action

MotionNotify

The user moves the pointer to the Galeon launch icon.

ButtonPress

The user presses the button and Galeon starts.

CreateNotify

Galeon is started and the window is created.

VisibilityNotify

The start page is loaded and visible to the user.

MotionNotify

The user moves the pointer to a link on the loaded page.

ButtonPress

The user clicks on the link.

VisibilityNotify

The new page is loaded and visible to the user.

Table 2. How Xnee Replays a Test Session

Protocol Data Name

Xnee Action

MotionNotify

Xnee moves the pointer to the Galeon launch icon.

ButtonPress

Xnee presses the button and Galeon starts.

CreateNotify

Xnee waits for this event to be sent. When Xnee receives a
CreateNotify notice, it continues with the next event in the file.

VisibilityNotify

Xnee waits for this event to be sent. Because the network is down
and the page can't be loaded, this event isn't sent. Xnee
continues to wait. Eventually the event is sent and Xnee can
continue.

MotionNotify

Xnee moves the pointer to a link on the loaded page.

ButtonPress

Xnee clicks on this link.

VisibilityNotify

The new page is loaded and visible.

What to Synchronize

Although synchronization is needed, finding the right
data to use for synchronization may be difficult. Xnee solves this
with plugin files that specify what should be recorded for a range of
applications. These plugins are named after the applications they are
intended to test. If you want to test a browser you've written,
it would be a good idea to use the Galeon plugin.
Sometimes, though, no plugins are suitable for your program, and you need to find
the right protocol data to synchronize. The following example
hopefully makes it easier for you in the future. We chose gnumeric as a
program for which the right options need to be found. First, launch gnumeric. Then
start Xnee in a terminal emulator with the following options:

Type some stuff into the gnumeric spreadsheet and use the menus to insert
today's date or other input. When you're done, go to the terminal and
press Ctrl-C to stop recording. It is now time to replay your session. Set
gnumeric in the same state it was in when you recorded. Launch Xnee in
replay mode like this:

xnee --replay -f session1.xnr

Xnee sometimes pauses when replaying the session. This happens if the
protocol data isn't sent in the same order as it was recorded. Xnee
pauses execution for a while in order to wait for the expected data (as
read from file) to be sent by the server. Eventually, a timeout expires
and Xnee tries to continue. If Xnee can't synchronize between
the recorded data traffic and the data traffic as sent when replaying,
it bails out.

Xnee supports giving record options through plugins. When you have
found the settings for your applications, save them in a plugin file. The
syntax of a plugin file is similar to the command-line options. The
easiest way to crate a new plugin is to copy an old one, fill in your
settings and then rename it to some appropriate name. Xnee is distributed
with plugins for different clients. If you want to send a plugin file for
your application to Xnee, please do. The Xnee home page has instructions
for how to contribute.

If you have a program that creates windows for user feedback, you have
to make sure these windows pop up at the same location. Xnee
records all device events with coordinates referring to the root window, not the window that was created.

To ease recording, make scripts that start Xnee with the right
settings for a specific purpose. You can add a launcher to your panel
or add a menu item to your window manager menu.

Conclusion

Xnee has seen a lot of activity lately, mainly due to feedback from
Xnee users. We hope you consider Xnee for your project. Happy testing
and happy hacking.

Henrik Sandklef lives in Gothenburg (Sweden) with his wife and
daughters. He spends most of his time awake with his family, cooking,
hacking and evaluating GNU software, and occasionally, he tries to play
football. You can reach him at hesa@gnu.org.