Profile

User's Modules

STATUS

This module and related encoding format are still under development. Do
not use it anywhere near production. Input is welcome.

DESCRIPTION

Bifcode implements the bifcode serialisation format, a mixed
binary/text encoding with support for the following data types:

Primitive:

Undefined(null)

Booleans(true/false)

Integer numbers

Floating point numbers

UTF8 strings

Binary strings

Structured:

Arrays(lists)

Hashes(dictionaries)

The encoding is simple to construct and relatively easy to parse. There
is no need to escape special characters in strings. It is not
considered human readable, but as it is mostly text it can usually be
visually debugged.

Bifcode can only be constructed canonically; i.e. there is only one
possible encoding per data structure. This property makes it suitable
for comparing structures (using cryptographic hashes) across networks.

In terms of size the encoding is similar to minified JSON. In terms of
speed this module compares well with other pure Perl encoding modules
with the same features.

MOTIVATION & GOALS

Bifcode was created for a project because none of currently available
serialization formats (Bencode, JSON, MsgPack, Sereal, YAML, etc) met
the requirements of:

Support for undef

Support for UTF8 strings

Support for binary data

Trivial to construct on the fly from within SQLite triggers

Universally-recognized canonical form for hashing

There no lofty goals or intentions to promote this outside of my
specific case. Use it or not, as you please, based on your own
requirements. Constructive discussion is welcome.

SPECIFICATION

The encoding is defined as follows:

BIFCODE_UNDEF

A null or undefined value correspond to '~'.

BIFCODE_TRUE and BIFCODE_FALSE

Boolean values are represented by '1' and '0'.

BIFCODE_UTF8

A UTF8 string is 'U' followed by the octet length of the decoded string
as a base ten number followed by a colon and the decoded string. For
example "\x{df}" corresponds to "U2:\x{c3}\x{9f}".

BIFCODE_BYTES

Opaque data is 'B' followed by the octet length of the data as a base
ten number followed by a colon and then the data itself. For example a
three-byte blob 'xyz' corresponds to 'B3:xyz'.

BIFCODE_INTEGER

Integers are represented by an 'I' followed by the number in base 10
followed by a ','. For example 'I3,' corresponds to 3 and 'I-3,'
corresponds to -3. Integers have no size limitation. 'I-0,' is invalid.
All encodings with a leading zero, such as 'I03,', are invalid, other
than 'I0,', which of course corresponds to 0.

BIFCODE_FLOAT

Floats are represented by an 'F' followed by a decimal number in base
10 followed by a 'e' followed by an exponent followed by a ','. For
example 'F3.0e-1,' corresponds to 0.3 and 'F-0.1e0,' corresponds to
-0.1. Floats have no size limitation. 'F-0.0,' is invalid. All
encodings with an extraneous leading zero, such as 'F03.0e0,', are
invalid.

BIFCODE_LIST

Lists are encoded as a '[' followed by their elements (also bifcode
encoded) followed by a ']'. For example '[U4:spamU4:eggs]' corresponds
to ['spam', 'eggs'].

BIFCODE_DICT

Dictionaries are encoded as a '{' followed by a list of alternating
keys and their corresponding values followed by a '}'. For example,
'{U3:cowU3:mooU4:spamU4:eggs}' corresponds to {'cow': 'moo', 'spam':
'eggs'} and '{U4:spam[U1:aU1:b]}' corresponds to {'spam': ['a', 'b']}.
Keys must be BIFCODE_UTF8 or BIFCODE_BYTES and appear in sorted order
(sorted as raw strings, not alphanumerics).

INTERFACE

encode_bifcode( $datastructure )

Takes a single argument which may be a scalar, or may be a reference to
either a scalar, an array or a hash. Arrays and hashes may in turn
contain values of these same types. Returns a byte string.

The mapping from Perl to bifcode is as follows:

'undef' maps directly to BIFCODE_UNDEF.

The global package variables $Bifcode::TRUE and $Bifcode::FALSE
encode to BIFCODE_TRUE and BIFCODE_FALSE.

Plain scalars that look like canonically represented integers
will be serialised as BIFCODE_INTEGER. Otherwise they are treated as
BIFCODE_UTF8.

SCALAR references become BIFCODE_BYTES.

ARRAY references become BIFCODE_LIST.

HASH references become BIFCODE_DICT.

You can force scalars to be encoded a particular way by passing a
reference to them blessed as Bifcode::BYTES, Bifcode::INTEGER or
Bifcode::UTF8. The force_bifcode function below can help with
creating such references.

decode_bifcode( $string [, $max_depth ] )

If you pass an integer for the second option, it will croak when
attempting to parse dictionaries nested deeper than this level, to
prevent DoS attacks using maliciously crafted input.

bifcode types are mapped back to Perl in the reverse way to the
encode_bifcode function, with the exception that any scalars which
were "forced" to a particular type (using blessed references) will
decode as unblessed scalars.

Croaks on malformed data.

force_bifcode( $scalar, $type )

Returns a reference to $scalar blessed as Bifcode::$TYPE. The value of
$type is not checked, but the encode_bifcode function will only
accept the resulting reference where $type is one of 'bytes',
'integer', or 'utf8'.

DIAGNOSTICS

trailing garbage at %s

Your data does not end after the first encode_bifcode-serialised
item.

You may also get this error if a malformed item follows.

garbage at %s

Your data is malformed.

unexpected end of data at %s

Your data is truncated.

unexpected end of string data starting at %s

Your data includes a string declared to be longer than the available
data.

malformed string length at %s

Your data contained a string with negative length or a length with
leading zeroes.

malformed integer data at %s

Your data contained something that was supposed to be an integer but
didn't make sense.

Lock::Socket provides cooperative inter-process locking for applications
that need to ensure that only one process is running at a time. This
module works by binding an INET socket to a port on a loopback
(127.0.0.0/8) address which the operating system conveniently restricts
to a single process.

The "lock_socket()" and "try_lock_socket()" functions both take a
mandatory port number and an optional IP address as arguments, and
return a Lock::Socket object on success. "lock_socket()" will raise an
exception if the lock cannot be taken whereas "try_lock_socket()" will
return "undef".

This scheme provides something of an automatic per-user lock for a given
$PORT, provided there is no user ID greater than 65536. The calculated
address can be read back via the "addr()" method.

As soon as the Lock::Socket object goes out of scope the port is closed
and the lock can be obtained by someone else.

If you want to keep holding onto a lock socket after a call to "exec"
(perhaps after forking) read about the $^F variable in perlvar, as you
have to set it before creating a lock socket to ensure the it will not
be closed on exec. See the example/solo file in the distribution for a
demonstration:

This module combines two ideas: the first is declarative syntax made successful by Moose. The second idea is to have a command processing module that doesn't stop with options, but also evaluates command arguments. The documentation s not quite complete, but the implementation is pretty close to being finished.