D.1.5. The thl Directory

The transaction history log (THL) retains a copy of the SQL statements
from each master host, and it is the information within the THL that is
transferred between hosts and applied to the database. The THL
information is written to disk and stored in the
thl directory:

THL files are created on both the master and slaves within the cluster.
THL data can be examined using the thl command.

The THL is written into individual files, which are by default, no more
than 1 GByte in size each. From the listing above, you can see that each
file has a unique file index number. A new file is created when the file
size limit is reached, and given the next THL log file number. To
determine the sequence number that is stored within log, use the
thl command:

The THL files are retained for seven days by default, although this
parameter is configurable. Due to the nature and potential size required
to store the information for the THL, you should monitor the disk space
and usage.

The purge is continuous and is based on the date the log file was
written. Each time the replicator finishes the current THL log file, it
checks for files that have exceeded the defined retention configuration
and spawns a job within the replicator to delete files older than the
retention policy. Old files are only removed when the current THL log
file rotates.

D.1.5.1. Purging THL Log Information on a Slave

Warning

Purging the THL on a slave node can potentially remove information
that has not yet been applied to the database. Please check and
ensure that the THL data that you are purging has been applied to
the database before continuing.

The THL files can be explicitly purged to recover disk space, but you
should ensure that the currently applied sequence no to the database
is not purged, and that additional hosts are not reading the THL
information.

To purge the logs on a SLAVE node:

Determine the highest sequence number from the THL that you want
to delete. To purge the logs up until the latest sequence number,
you can use trepctl to determine the highest
applied sequence number:

D.1.5.2. Purging THL Log Information on a Master

Warning

Purging the THL on a Master node can potentially remove information
that has not yet been applied to the slave databases. Please check
and ensure that the THL data that you are purging has been applied
to the database on all slaves before continuing.

Warning

Follow the below steps with great caution! Failure to follow best
practices will result in slaves unable to apply transactions,
forcing a full re-provisioning. For those steps, please see
Section 8.6, “Provision or Reprovision a Slave”.

The THL files can be explicitly purged to recover disk space, but you
should ensure that the currently applied sequence no to the database
is not purged, and that additional hosts are not reading the THL
information.

To purge the logs on a MASTER node:

Determine the highest sequence number from the THL that you want
to delete. To purge the logs up until the latest sequence number,
you can use trepctl to determine the highest
applied sequence number:

Create a new directory, or attach a new filesystem and location
on which the THL content will be located. You can use a
directory on another filesystem or connect to a SAN, NFS or
other filesystem where the new directory will be located. For
example:

shell> mkdir /mnt/data/thl

Copy the existing THL directory to the new directory location.
For example:

shell> rsync -r /opt/continuent/thl/* /mnt/data/thl/

Move the existing directory to a temporary location:

shell> mv /opt/continuent/thl /opt/continuent/old-thl

Create a symbolic link from the new directory to the original
directory location:

Create a new directory, or attach a new filesystem and location
on which the THL content will be located. You can use a
directory on another filesystem or connect to a SAN, NFS or
other filesystem where the new directory will be located. For
example:

shell> mkdir /mnt/data/thl

Copy the existing THL directory to the new directory location.
For example:

shell> rsync -r /opt/continuent/thl/* /mnt/data/thl/

Change the directory location using tpm to
update the configuration for a specific host:

D.1.5.4. Changing the THL Retention Times

THL files are by default retained for seven days, but the retention
period can be adjusted according the to requirements of the service.
Longer times retain the logs for longer, increasing disk space usage
while allowing access to the THL information for longer. Shorter logs
reduce disk space usage while reducing the amount of log data
available.

Note

The files are automatically managed by Tungsten Replication. Old THL files
are deleted only when new data is written to the current files. If
there has been no THL activity, the log files remain until new THL
information is written.