Building Palm Conduits, Part 4

This final installment from Programming Visual Basic for the Palm OS covers data formats, including packing and unpacking record data, in a Palm application.

Data Formats

Until now, our focus has been on how to interface with the HotSync manager, and how to use the Sync Suite API to synchronize databases and records. It was easy to read and write the actual record data, because we only supported ASCII strings. A real Palm program is going to have a much more complicated record structure, with a mixture of string and binary data.

The sample code for this section includes a new AppForge project, Ch4b.vbp., and a new conduit project, Ch4bCond.vbp. Together, these show how to handle the low-level chores associated with packing and unpacking record data, converting numbers between the Intel and Motorola formats, and handling differences between Palm and Windows date formats.

Note that if you use the AppForge database utilities and the Universal Conduit, you don't have to worry about packing and unpacking records and fields, or converting between Palm PDA and Windows data types. We covered the database utilities in Chapter 3, and will look at the Universal Conduit in Chapter 5.

Creating Packed Record Data

Our new application creates a database that consists of structured records with four fields: a Date value, a Time value, a Boolean value, an Integer value, a Long integer, and a String with 20 characters. Figure 4-9 shows the application's user interface on the Palm PDA.

Figure 4-9.Mains screen for sample application Ch4b

First, let's look at how to create a packed record on the Palm. You'll find this code in the form module Ch4b.frm in the new application.

Building Palm Conduits, Part 1
This excerpt from Programming VB for PalmOS, offers an introduction to building conduits, synchronization software that connects Palm apps and data stores.

The AppForge documentation provides the size of each of the built-in data types: four bytes for a Date or Time field, one byte for a Boolean, two bytes for an Integer, four bytes for a Long, and one byte for each character in a string variable.

When the user presses the WriteDB button in the application, we use PDBCreateRecord to allocate a record large enough to hold the six fields.

We pack the fields into our record in this order: Date, Time, Boolean, Integer, Long, and then String. Knowing the layout of the fields--their type, order and size--is critical when unpacking the data. AppForge lists the sizes of the basic data types in their documentation. As under Windows, both Date and Time values are stored in the Date data type.

The PDBWriteFieldByOffset stores the binary representation of a value into the record. Here's how to write a single Long integer field into the Palm record:

Note that this function requires that the field offset into the record be specified. In this example, we are writing a Long value at offset 5. The AppForge VB runtime engine already knows that the size of this variable is 4 bytes, so we don't have to pass that also.

After we have written all six fields, we call PDBUpdateRecord to flush the new record into the Palm database.

Reading Packed Record Data

Now let's look at how to unpack the field data in the conduit. There are two main issues we address when unpacking the record data: finding the start and end of fields, and converting formats between the Palm device and Windows. You'll find the code for this section in Ch4bLogic.cls--note the `b'--in the new conduit. This conduit is structured just like the first one.

Locating the field data is just the reverse of how the record was created. First we get the record bytes into the variant array Data, using one of the PDRecordAdapterRead iterator functions. Because the array data is a byte-wise copy of the Palm record, the offsets used for writing are identical to those used for reading.

If the data is byte- or character-oriented, such as String or Boolean data, simply copy the needed bytes from the array:

Dim b As Boolean
b = Data(8)

For String data, you need to know the size ahead of time. In this example, we read from offset 15 until the end of the record data:

It is probably overkill to use the UBound and LBound functions here. These are two of the rare VB functions that index from zero, not one, which is why there is an extra +1 in the ByteArrayToBSTR function call.

Numeric data, such as Integer or Long values, must be converted from Motorola to Intel byte ordering. The PDUtility object has methods that explicitly do this: SwapWORD and SwapDWORD take 16-bit and 32-bit quantities and reverse the high and low bytes as appropriate.

There is another issue when converting numbers: the data in the array is byte-oriented, while we want whole Integer or Long values. Here's how we convert both at once in the sample conduit:

Dim i As Integer
pdUtil.ByteArrayToWORD Data, 9, True, i

The ByteArrayToWORD function locates the two bytes at offset 9 in the data array and converts them to an Integer quantity, and it swaps the byte ordering as well. We request the swapping by passing True as the third parameter; if you don't want the implicit conversion, pass False instead. To convert a 32-bit quantity, use the ByteArrayToDWORD function.

Converting Dates and Times

Dates and times are the hardest values to convert, because they are a blend of application code and operating system convention. Dates and times on the Palm are represented in the Palm operating system as seconds since January 1, 1904. How these values are stored in database files, however, varies wildly from application to application.

The Palm native applications, for example, use a packed unsigned 16-bit quantity to represent the date: 7 bits for the year since 1904, 4 bits for the month, and 5 bits for the day. These applications use an unsigned 16-bit quantity to represent the time: 8 bits for the hour, and 8 bits for the minute.[6]

Our application stores dates and times exactly as returned by the AppForge VB runtime functions Date and Time. These values are the Palm operating system values: seconds since January 1, 1904. Depending on how the value is constructed, a Date value may or may not have the day or time component:

Dim d As Date
d = Date ' No time component
d = Date + Time ' Has time component

Although it's not documented, AppForge Date values are 32-bit quantities.

First we extract the 32-bit date from the packed record data using ByteArrayToDWORD (not shown). Then we call PalmLongToDate to convert the Long value to a VB Date value. The code for PalmLongToDate is shown in Example 4-14.

First we convert the Long argument, which represents seconds since January 1, 1904, into a double. We also adjust the argument if it is negative. This is necessary because the Palm data type is unsigned, so a negative number indicates that we have lost a bit! Adding the huge UnsignedLngMax brings it back.[7]

Next, we can calculate how many days have passed since January 1, 1904.

DaysSince1904 = SecsSince1904 / SecsPerDay

That's the hard part. Finally, we create a VB Date with the initial magic value using the DateSerial function, and add the correct number of days to it to obtain the converted date:

PalmLongToDate = DateSerial(1904, 1, 1) + CLng(DaysSince1904)

By using VB date arithmetic, we avoid issues such as leap year, which are better handled by the operating system and runtime libraries.

WARNING:PalmLongToDate is only accurate if it is given a pure date--one that has no time component. During normal integer division, the remainder is silently discarded. But because the calculation to get DaysSince1904 is done in Double arithmetic, this truncation doesn't occur. This can cause the function to be inaccurate with some inputs.

Compared to getting the Date, the Time routine is almost trivial. It is shown in Example 4-15.

The Time value is stored in the lower 17 bits of the 32-bit quantity. This means that we don't have to worry about overflow while converting between signed and unsigned formats:

If T < 0 Then T = T + &H10000
T = T And &H1FFFF

We can discard any high-order bits that belong to a Date component, which means that PalmLongToTime may be safely called with any valid Date.

At this point, the variable T holds the number of seconds since midnight; this is converted to hours, minutes, and seconds. Adding these together with the VB TimeSerial function gives us the correct time, which we return as the value of the function.

Resources

This chapter presented the fundamentals of conduit development using VB. At this point, you should understand what a conduit is and how it fits into Palm's HotSync architecture. You should be comfortable with building and running conduits under the Palm HotSync manager, and you should be able to address the issues encountered when designing a conduit.

There are many resources available from Palm to assist in the conduit development process. The Palm OS development web site has a page for conduit development at http://www.palmos.com/dev/tech/conduits. This web page has several resources, including an active mailing list, the Palm knowledge base, and links to documentation.

There are two sets of official Palm documents for the CDK. One set is based on the COM specification, and the other is based on the C/C++ language. Like all Palm documentation, the sets come in two parts: a Reference and a Companion. The companion document explains high-level concepts; the reference provides a description of every class, method, and property in the CDK. Other documentation on the web site includes presentations from the PalmSource developer conferences on conduit development.

Palm OS Programming (O'Reilly & Associates, Inc.), now in its second edition, provides a lot of detail and insight into conduit logic and the inner workings of the HotSync manager. Be aware, however, that the book is intended primarily for C/C++ developers.

1.
There are other HotSync managers for enterprise use that allow remote connections. Conduits can also be built using C/C++ or Java, but we won't discuss either of those languages in this book.

2.
You can use the InstallAide functions to do this, which are part of the Palm Desktop. These functions are described in the CDK documentation; we don't discuss them in this book.

3.
You want CDK release 4.01 DR1, which was the latest release available as of this writing. This release is the first that exposes a COM interface specifically for VB development.

4.
The Palm device currently uses the Motorola 68000 series processor. This CPU represents numbers in little-Endian order, which is different from the Intel 80x86 processors. Your conduit must handle the conversion if your application stores numeric data.

5.
The slow sync logic given here fails to handle this important case. If the user syncs with his desktop, then does a delete on the handheld, and then syncs with another desktop, the deleted record on the handheld is gone. Now, if the user syncs with his desktop, the conduit doesn't see the deleted record on the handheld, and so it shouldn't delete it on the desktop. Whew. You'll need to iterate through the records on the desktop. Any that aren't on the handheld (and aren't new) have been deleted, and must be deleted from the desktop (unless, of course, they've been modified on the desktop).

6.
We don't cover the native Palm date and time formats here. The AppForge knowledge base has an article with a code sample on how to convert those formats.

7.
Additionally, this is why we convert to a double internally. The conversion to unsigned 32-bit data would cause a silent overflow and our dates would be incorrect.

Patrick Burton has been programming in C/C++ for most of his career. His experience inlcludes algorithm development for embedded satellite receivers, Linux system programming, and Windows programming using the Win32 API and Microsoft Found Classes (MFC).

Matthew Holmes has been developing computer software for 15 years. He cherishs his liberal arts degree in Foreign Affairs from the University of Virginia, and he holds a graduate degree in Computer Science from George Mason University.