KnowledgeBase 00106: Handling Binary Data in QM

Introduction

Historically, multivalue environments have often required programmers to use
special encodings when storing binary data such as transforming the data to a
stream of hexadecimal character values. This is because, for example,

Some systems follow the C programming convention and use the ASCII null
character (character 0) as a string terminator internally.

Some systems cannot store data containing the item mark (also known as the
record mark or segment mark).

The media format used by the Pick T.DUMP and ACCOUNT.SAVE tools reserves
sequences such as an item mark followed by a text mark for internal use.

Binary Data in the QM File System

QM does not reserve any special character sequences within the file system.
It is therefore possible to save binary data such as scanned images without any
encoding. When using directory files, it is important that the file is used
with mark mapping disabled so that the default action of treating field marks
in the data as newlines in the file is suppressed.

QM provides a limited emulation of the Pick data save tools. It is important
to remember that these may not be able to save all data constructs found in QM
because they must adhere to the Pick media format definition.

Use of Null Terminators

Although the QM file system is "binary clean", the underlying operating system
requires use of null terminated strings in some places. For example, a
filename cannot include the ASCII null character.

QMClient

The QMClient API for Visual Basic is binary clean in that it handles
strings using the BSTR data type which has a length associated with the
actual string data. Unfortunatly, Microsoft have modified Visual Basic to use
null terminators internally so the API is effectively no longer binary clean.

The C version of the QMClient API, also used by several wrappers for other
languages, has never been binary clean as it was designed to work with null
terminated C strings.