Package com.sleepycat.bind.tuple Description

The serialization format for tuple bindings are designed for compactness,
serialization speed and proper default sorting.

When a format is used for database keys, it is important to use default
sorting for best performance. Although a custom comparator may be specified
for a database or
entity
index, custom comparators often reduce performance because comparators are
called very frequently during Btree operations.

For proper default sorting, the byte array of the stored format must be
designed so that a byte-by-byte unsigned comparison results in the natural sort
order, as defined by the Comparable.compareTo(T) method of the
data type. For example, the natural sort order for integers is the standard
mathematical definition, and is implemented by Integer.compareTo,
Long.compareTo, etc. This is called default natural
sorting.

Although most tuple formats provide default natural sorting, not all of them
do. Certain formats do not provide default natural sorting for historical
reasons (see the discussion of packed integer and float formats below.) Other
formats sacrifice default natural sorting for other performance factors (see
the discussion of BigDecimal formats below.)

Another performance factor has to do with amount of memory used by keys in
the Btree. Keys are stored in their serialized form in the Btree. If keys are
small (currently 16 bytes or less), Btree memory can be optimized. Optimized
memory storage is based on the maximum size of all keys in a single Btree
node. A single Btree node holds N adjacent key values, where N is 128 by
default and can be configured for each database or index.

Strings are stored as a byte array of UTF encoded characters, either where
the length must be known by the application, or the byte array is
zero-terminated. The UTF encoding is described below.

Null strings are UTF encoded as { 0xFF }, which is not allowed in a
standard UTF encoding. This allows null strings, as distinct from empty or
zero length strings, to be represented. Using default sorting, null strings
will be ordered last.

Zero (0x0000) character values are UTF encoded as non-zero values, and
therefore embedded zeros in the string are supported. The sequence { 0xC0,
0x80 } is used to encode a zero character. This UTF encoding is the same one
used by the native Java UTF libraries and is called
Modified UTF-8.
However, this encoding of zero does impact the lexicographical ordering, and
zeros will not be sorted first (the natural order) or last.

For all character values other than zero, the standard UTF encoding is
used, and the default sorting is the same as the Unicode lexicographical
character ordering.

Binding classes and methods are provided for zero-terminated and
known-length String values.

Fixed Size Integer Formats

The size of the stored value depends on the type, and ranges (as one would
expect) from 1 byte for type byte and class Byte, to 8 bytes for
type long and class Long.

Signed numbers are stored in the buffer in MSB (most significant byte first)
order with their sign bit (high-order bit) inverted to cause negative numbers
to be sorted first when comparing values as unsigned byte arrays, as done in a
database.

Unsigned numbers, including characters, are stored in MSB order with no
change to their sign bit. Arrays of characters and unsigned bytes may also be
stored and may be treated as String values. For booleans, true
is stored as the unsigned byte value one and false as the unsigned byte
value zero.

Packed Integer Formats

The packed integer format stores integers with small absolute values in a
single byte. The size increases as the absolute value increases, up to a
maximum of 5 bytes for int values and 9 bytes for long
values.

The packed integer format can be used for integer values between Long.MIN_VALUE and Long.MAX_VALUE. However,
different bindings and methods are provided for type int and long, to avoid unsafe casting from long to int when int values are used.

Because the same packed format is used for int and long
values, stored int values may be expanded to long values
without introducing a format incompatibility. In other words, you can treat
previously stored packed int values as packed long values.

Packed integer formats come in two varieties: those that support default
natural sorting and those that don't. The formats of the two varieties are
incompatible. For new applications, the format that supports default natural
sorting should normally be used. There is no performance advantage to using
the unsorted format.

The format with support for default natural sorting stores values in the
inclusive range [-119,120] in a single byte.

The unsorted packed integer format is an older, legacy format that is used
internally and supported for compatibility. It stores values in the inclusive
range [-119,119] in a single byte. Because default natural sorting is not
supported, this format should not be used for keys. However, it so happens
that packed integers in the inclusive range [0,630] are sorted correctly by
default, and this may be useful for some applications.

BigInteger Formats

BigInteger values are variable length and are stored as signed
values with a preceding byte length. The length has the same sign as the
value, in order to support default natural sorting.

The length is stored as a 2-byte (short), fixed size, signed integer.
Supported values are therefore limited to those with a byte array (BigInteger.toByteArray()) representation with a size of 0x7fff bytes
or less. The maximum BigInteger value is (20x3fff7 - 1) and
the minimum value is (-20x3fff7).

Floats and doubles are stored in a fixed size, 4 and 8 byte format,
respectively. Floats and doubles are stored using two different
representations: a representation with default natural sorting, and an
unsorted, integer-bit (IEEE 754) representation. For new applications, the
format that supports default natural sorting should normally be used. There is
no performance advantage to using the unsorted format.

For float values, Float.floatToIntBits and the following
bit manipulations are used to convert the signed float value to a
representation that is sorted correctly by default.

BigDecimal values are stored using two different, variable length
representations: a representation that supports default natural sorting, and an
unsorted representation. Differences between the two formats are:

The BigDecimal format with default natural sorting should normally
be used for database keys.

Default natural sorting is supported.

The stored value is around 3 bytes larger than the unsorted format,
more or less, and is a minimum of 8 bytes.

More computation is required for serialization than the unsorted
format.

Trailing zeros after the decimal place are stripped, meaning that
precision is not preserved.

The unsorted BigDecimal format should normally be used for non-key
values.

Default natural sorting is not supported.

The stored value is around 3 bytes smaller than the sorted format, more
or less, and is a minimum of 3 bytes.

Less computation is required for serialization than the sorted
format.

Trailing zeros after the decimal place are preserved, meaning that
precision is preserved.

Both formats store the scale or exponent separately from the unscaled value,
and the stored size does not increase proportionally as the absolute value of
the scale or exponent increases.