Data Persistence with Waba

Last month we
took a look at some of the GUI components within Waba. There aren't
many useful Palm OS programs that don't have a GUI
interface. Likewise, there aren't many programs without data. Whether
writing business apps or games you need some mechanism to store
data.

If you intend on using Java (or a Java-like language) as your
language of choice, you face a challenge in Palm OS data
persistence.

The Palm OS is composed of featureful C APIs, which give you a lot
of programming power, including the DataManager and Palm Database
(pdb) files. (Before you raise an objection -- I know that the Palm
has no filesystem, but I'm using the term generically.)

The Sun KVM release comes with classes for creating simple
interfaces on the Palm, but these are mainly a reference
implementation, not intended to be used for commercial
applications. Among these classes is the
com.sun.kjava.Database, which gives a programmer the
ability to read and write pdb files. However, if the
kjava Database class gave you full access to the pdb API,
it would cease to be portable across other resource-constrained
devices that do not run the Palm OS. Sun's intention is to release a
PDA Profile, which would in theory be usable for Palm Pilots, Windows
CE devices, etc. This has been delayed, and if tradition holds, there
will probably be an announcement related to device profiles at JavaOne
next month.

How does all this relate to data persistence with Waba? If you'll
recall, Waba is also a cross-platform solution, and its creators did
not want to provide a data mechanism which would make it a Palm-only
solution. Like the kjava.Database class, persistence in
Waba understands only byte arrays. This is quite primitive, but Waba
gives you some help that the KVM doesn't.

Creating a Catalog and accessing data.

Waba provides data persistence primarily through the Catalog
class.

The Catalog class extends waba.io.Stream, an abstract
superclass of stream-based I/O operations. To create a Catalog in
Waba, you write

Catalog c = new Catalog("MyCatalog.MYxx.DATA", Catalog.CREATE);

The constructor takes two arguments: the catalog name and the mode
in which to open the Catalog. (Other modes are READ_ONLY,
WRITE_ONLY, and READ_WRITE.)

The Catalog name is actually a concatenation of three things. The
first part is the name proper. This would show up in your Palm pilot
if you selected the menu, then selected info, and scrolled through all
of your installed applications.

The second part is the creator ID, which is a four-byte identifier
for the Palm OS to tie together application resources on the
device. This has to have at least one uppercase letter, and it must be
unique on the device. If you intend on releasing your application to a
wider audience, you should register your creator id on the Palm OS
site. This will ensure that no one else has chosen the same name. The
third is the type of database. This should be hardcoded to
DATA as this is the only kind you can create through the
Catalog class.

Once you have opened the catalog, you have access to it. Before you
actually call any of the write methods, you need to add a record by
the addRecord(int recordSize) method. This will allocate
a new record of the given size for you to write data to.

Most application programmers are accustomed to relational
databases, which provide a wealth of ways to manipulate your data, but
ways that resource-constrained devices can't handle. In other words, a
Catalog is in no way a relational database.

Catalogs support the concept of a key field other than the record
id itself. There isn't a method like
Catalog.findRecord("Smith") that will return a record
id. Access to the data is sequential only; to find a particular
record, you need to search the entire set, looking for a match. There
is a utility method, inspectRecord(byte[] buf, int
recPosition), that is designed to offer a quick look at the
data to see if it contains what you want. The API documentation
cautions that using this method can be unreliable as none of the
parameters are checked for validity. You must avoid running past the
end of the Catalog.

To return to the matter of putting data into a Catalog, everything
needs to be in the form of a byte array before it is written. That can
be quite cumbersome when dealing with Strings, especially since some
of the methods of the java.lang.String class are not
present in the waba.lang.String class. In particular
String.getBytes is not there.

Waba provides some extra help for dealing with such a primitive
persistence facility. Waba allows you to wrap a Catalog in a
DataStream class, which eases the burden. (In fact, a DataStream can
be wrapped around any other stream-oriented class such as a
SerialPort). When you create the Catalog that we looked at previously,
you could wrap this in a DataStream as follows:

We open the Catalog, then after ensuring that we have a valid
handle, we create a new DataStream wrapped around the Catalog
instance. After this, we can add records, then call DataStream's
writeString method, which will convert the String to a byte array so
the Catalog can accept it.

Other methods in the DataStream class to deal with various data
types include writeBoolean, writeBytes,
writeFloat, writeInt,
writeShort, and writeStringArray. These have
matching read methods as well. Armed with these tools, let's look at
another example of reading data from a Catalog, and populating an Edit
control.

We open the Catalog as before (but now in READ_ONLY
mode) and wrap it in a DataStream. After ensuring it is open we get
the count using the getRecordCount() method. This will be
used when we loop through the record set, getting each name,
populating the Edit control. Notice also that we position
the pointer at each record by using the
setRecordPos(record) method. This is necessary to step
through the Catalog.

Then we use the DataStream to get the data by calling the
readString() method.

This demonstrates the basic functionality of Catalogs in
Waba. However, these examples are not what you would likely encounter
in real programming in that they deal with a very rudimentary set of
data (first names only).

If you were writing the contact manager that we began to construct
last month, you'd need to store first name, last name, address phone,
fax, email, and several other fields as well.

Since the Catalog only understands byte arrays, how can you
determine where one field ends and another begins? Using DataStream
can again make your life easier. If you call the methods described
above to write the data to the Catalog, then getting it out will be
equally easy. For example, if you write the data like this,

As long as the proper order is maintained, the data will be
properly delimited by the DataStream class. This is a major advantage
over the kjava.Database class where delimiting the data
is entirely up to you, either by offset or with some character.

Performance Considerations and Conduits

When coding for a resource-constrained device such as the Palm
Pilot, Visor, etc., you need to bear in mind that memory is at a
premium. Treating small devices like they are PCs with 256MB RAM can
definitely get you into trouble. For example, our contact manager
application may have 150 records in it. Reading these into a Hashtable
or some other collection object could quickly max out the allocated
heap space. Instead, you should consider working with the record Ids
only. Then when you need a particular record you can use the id to
look it up, and you don't need to risk the overhead of dealing with a
large collection of objects.

The final thing to note is that to write an app that goes the
distance, you will likely need a conduit which will synchronize you
Palm Database with the desktop using the HotSync technology. Palm has
a couple of Conduit Development Kits available, one in C, the other in
Java. The kjava.Database class is not synchronizable
since it does not conform to the pdb format, which a conduit expects
-- at least, not with the Conduit Kit for Java. It is possible that
one could do it using the Conduit Kit for C, but if you know C, why
write your Palm app in Java at all?

Guilherme Hazan, who's added several of the SuperWaba classes to
the API, has written conduits in Java to synchronize a Waba Catalog,
and he assures me that it can indeed be done. Writing a conduit is a
big subject in itself, and if your application calls for it, I would
encourage you to download the Conduit Development Kit from the Palm OS
site. It includes samples and a tutorial.

Given the arsenal of Waba tools you now have, you can certainly
begin experimenting with a very useful and well-performing platform
for Palm OS application development.