E.1. Transaction History Log (THL)

The Transaction History Log (THL) stores transactional data from different
data servers in a universal format that is then used to exchange and
transfer the information between replicator instances. Because the THL is
stored and independently managed from the data servers that it reads and
writes, the data can be moved, exchanged, and transmuted during processing.

The THL is created by any replicator service acting as a master, where the
information is read from the database using the native format, such as the
MySQL binary log, or Oracle Change Data Capture (CDC), writing the
information to the THL. Once in the THL, the THL data can be exchanged with
other processes, including transmission over the network, and then applied
to a destination database. Within Tungsten Replicator, this process is handled
through the pipeline stages that read and write information between the THL
and internal queues.

Information stored in THL is recorded in a series of event records in
sequential format. The THL therefore acts as a queue of the transactions. On
a replicator reading data from a database, the THL represents the queue of
transactions applied on the source database. On a replicator applying that
information to a database, the THL represents the list of the transactions
to be written. The THL has the following properties:

THL is a sequential list of events

THL events are written to a THL file through a single thread (to enforce
the sequential nature)

THL events can be read from individually or sequentially, and multiple
threads can read the same THL at the same time

THL events are immutable; once stored, the contents of the THL are never
modified or individually deleted (although entire files may be deleted)

THL is written to disk without any buffering to prevent software failure
causing a problem; the operating system buffers are used.

THL data is stored on disk within the thl directory of
your Tungsten Replicator installation. The exact location can configured using
logDir parameter of the THL component. A sample
directory is shown below: