13 Data Loading Format (DLF) Specification

This chapter describes version 1.0 of the Data Loading Format (DLF), which is the standard format to describe translated messages and seed data loaded into the database by the TransX utility. It contains the following topics:

Introduction to DLF

DLF defines a standard format for loading data with the TransX utility. It is intended to supersede loading data with SQL scripts. DLF provides the following advantages:

Format validation. Validation results in fewer errors during the translation and loading processes.

Ease of use. The user does not have to maintain the character encoding of each data file to correspond with the language used in the data file.

DLF is based on the XML 1.0 specification.

Naming Conventions for DLF

This section describes the naming conventions used in this document.

Elements and Attributes

The following naming conventions for elements and attributes are used throughout this specification:

Standard English letters

Lowercase letters only

Hyphen (-) may be used for concatenation

Attribute names must be consistently defined throughout

Industry-standard terminology should be followed wherever possible

Values

Values are case-sensitive except for some attribute values used for column names. All predefined attribute values are lowercase. No element values are defined by this specification.

File Extensions

No file extension is recommended by this specification. A future version of this specification may recommend that documents use an extension that conforms to an 8.3 standard.

General Structure of DLF

Data Loading Format is XML, so it begins with an XML declaration. After the XML declaration comes the DLF document itself, enclosed within the <table> element. A DLF document is composed of the following required sections:

The <lookup-key> element contains a list of column names that determine whether existing rows in the database are duplicates of the rows in the dataset definition included in the <dataset> element.

The <columns> element contains metadata about the <dataset> such as the names, datatypes, and attributes of columns.

The <dataset> element contains a <row> element for each row, which in turn contains a <col> element that corresponds to a piece of data that is loaded in a database column. In this way a DLF document looks similar to the familiar tabular format in printing data in the database and allows easy editing.

DLF provides one optional section, which is enclosed within a <translation> element. This section may precede the required sections.

In addition, DLF provides information about TransX utility processing. Such information includes but is not limited to the following:

The <query> element is used to retrieve the value to be loaded to the column from a SQL query.

The sequence attribute is used to retrieve the value to be loaded to the column from a sequence object in the database.

The constant attribute is used to specify a constant value to the column.

The language attribute is used to specify the language identifier to be loaded to the column.

Tree Structure of DLF

This section shows the possible structure of a DLF document as a tree. Each element is represented as <element_name>, where element_name is the name of an element. Attributes have no markup. Each element and attribute is followed by notation indicating its possible occurrence. Table 13-1 describes the occurrence notation.

DLF Specifications

XML Declaration in DLF

The XML declaration starts an XML entity. It indicates the XML version and can be used to declare the encoding of the file, as in the following example:

<?xml version="1.0" encoding="iso-8859-1" ?>

As in all XML files, the default encoding for a DLF file is assumed to be either UTF-8, which is a superset of the 7-bit ASCII character set, or UTF-16, which is conceptually UCS-2 with surrogate pairs for code points above 65,535. Thus, for these character sets, the encoding declaration is not necessary. Furthermore, all XML parsers support these character sets. If the encoding is UTF-16, then the first character of the file must be the Unicode Byte-Order-Mark, #xFEFF, which indicates the endianness of the file.

Other character sets supported by Oracle XML parsers include all Oracle character sets and commonly used IANA character set and Java encodings. The names of these character sets can be found in the parser documentation. You must declare these with encoding declarations if the document does not have an external source of encoding information such as from the execution environment or the network protocol. Therefore, it is recommended that you use a Unicode character encoding so that you can dispense with the encoding declaration. The recommended practice is to encode the document in UTF-8 and use the following declaration:

<?xml version="1.0" ?>

Entity References in DLF

There are five entity references predefined by XML. These entity references are listed in Table 13-2. The &lt; and &amp; entity references must always be used in place of the character they reference.

Lookup Key Elements

Contains the <column> element(s) based on which the TransX recognizes the rows as identical or duplicate.

name

None

Zero or more <column> elements

<column>

A <column> element under <lookup-key> element indicates a column to be used to recognize the rows as identical or duplicate. Columns with the same values in specified column(s) are considered duplicate, regardless of the values in the other column(s). A lookup key <column> must have corresponding <col>umns in the <dataset> portion or be declared as a <column> with a constant expression or a <query> in the <columns> section.

Specifies a column that corresponds to <col> elements under the <dataset> element. Once a <column> is defined the corresponding <col> element must appear in every <row> unless the column has the sequence, constant or query attribute.

Specifies an instance of a piece of data to be loaded to a database column, or in the case of a virtual column, a piece of data to be used to obtain an actual data to be loaded to a database column.

name

trans-key

Data for use by applications

Attributes in DLF

This section lists the various attributes used in the DLF elements. An attribute is never specified more than once for each element. Along with some of the attributes are the Recommended Attribute Values. Values for these attributes are case sensitive.

The values of the attribute are language identifiers as defined by [IETFRFC4646].

This attribute does not affect data loading operation in any way.

None; if absent, "en" is assumed

<table>

normalize-langtag

Specifies how to normalize the case of language tag.

"none", "standard", "uppercase" or "lowercase".

The meanings are as follows:

none - no normalization. the values in the language column on DLF are used as they are

standard - the style as recommended by the standards

* lowercase for the 2 letter language code

* uppercase for the 2 letter country code

* titlecase for the 4 letter script code

* lowercase for others

uppercase - uppercase everything

lowercase - lowercase everything

none

<table>

space

Specifies how white spaces (ASCII spaces, tabs and line-breaks) should be treated.

"default" or "preserve"

The value "default" signals that applications' default white-space processing modes are acceptable; the value "preserve" indicates the intent that applications preserve all whitespace. If this intent is declared at the root table element, it is considered to apply to all string data elements in the whole document. If declared at column level, it is considered to apply to the specified column of every row. If this attribute is declared in the dataset section, the intent applies to the immediate text data only. Declaration at the document or column level may be overridden with another instance of the space attribute.

"default"

<table>, <column>, <col>

name

Specifies the name of an object such as table, column, restype, and so forth.

String: This is a database table name for the <table> element, and a column name for the <column> or <col> element.

n/a

<table>, <column>, <col>

type

The datatype of a column in the dataset. This attribute specifies the kind of text contained in the <col> element in the dataset. Depending on this type, TransX may apply different processes to the data.

Because implicit datatype conversion is provided by XSU and JDBC, TransX does not do its own parsing based on this type information. It uses this attribute to choose appropriate intermediate data types in Java for columns of date or dateTime type, in which case the standard date formats are accepted.

The lexical representation of a value of number type should be supplied in the SQL language syntax, no matter what the current locale is. The SQL syntax uses no digit grouping separator (usually comma), but uses a dot as the decimal separator (usually dot).For the binary data type, the data value specified in a text field between the col tags indicates the name of a file that contains the actual binary data. Raw data cannot be embedded in the text field.For the other data types (string, date, and dateTime) the representation is constrained by the corresponding types in the XML Schema specification.For the sake of simplicity, DLF only accepts standard date formats of XML Schema in the form "CCYY-MM-DDThh:mm:ss" (dateTime) or "CCYY-MM-DD" (date). No other date format is recognized.

TransX uses this attribute for the following:

Bind virtual columns to parameters of a query

Bind the result of a query to a corresponding column

n/a

<column>

translate

Indicates whether or not the text of this column or parameter should be translated.

Either "yes" or "no"

"no"

<column>, <parameter>

constant

Specifies a constant value for this column or parameter.

The value of this column for every row

n/a

<column>, <parameter>

language

Specifies language identifier for this column

Language identifier or a placeholder. "%x" gets the value from the xml:lang attribute of the root table element.

n/a

<column>

sequence

Specifies a sequence in the database used to fill in the value for this column.

String: The name of a sequence in the database

n/a

<column>

virtual

Indicates that this column provides data used to construct another piece of data, which in turn is loaded into the database. A virtual column does not exist in the database. It is typically used to provide a value of a parameter in a query. A virtual column cannot be a lookup-key column. A virtual column with a query throws the result away.

Either "yes" or "no"

"no"

<column>

useforupdate

Indicates whether the value of this column is used for the update when uploading seed data. This attribute has no effect unless TransX is in the mode to update duplicate rows. A virtual column cannot have this attribute set to yes.

Either "yes" or "no". If this attribute is set to "no", then the value of the column remains unchanged on the update operation.

"yes"

<column>

maxsize

Specifies the maximum size for the data for this column.

Numeric value in the unit specified by the size-unit attribute. If this attribute is set and the size-unit is not set, the size is assumed to be in characters.

None

<column>

size-unit

Specifies the unit of size specified in the maxsize attribute.

Units. Recommended values are "char" for characters, "byte" for bytes.

For supplemental characters, they take two "char" units.

"char"

<column>

restype

Indicates the type of data contained in this column.

A resource type. The value must match with the name of a <restype> element.

None

<column>

expansion

Indicates the maximum size up to which translated strings are allowed to become longer for this type of resource.

A numeric value in percentage of increase.

0%

<restype>

text

Specifies a SQL query statement to obtain a value to put in the column to which the query belongs.

A SQL statement. Zero or more parameters can be specified with an identifier preceded by a colon. The statement should return a single row of a single value. Any excessive result is discarded.

n/a

<query>

id

Specifies a placeholder used in a SQL query statement with parameters. The value of the column specified by the sibling col attribute is associated as a parameter to the query.

String: an identifier that appears in the text attribute of the parent query element.

Empty string

<parameter>

col

Specifies a column to be associated with a placeholder in the query specified by the sibling id attribute.

String: a column name. The column must be other than the column this attribute is a part of.

n/a

<parameter>

trans-key

Specifies a key for translation.

String: a translation key. The value has to be unique in a translation domain.

XML Namespace Attributes

Specifies how whitespace (ASCII spaces, tabs and line-breaks) should be treated.

"default" or "preserve"

The value "default" signals that applications' default whitespace processing modes are acceptable for this element; the value "preserve" indicates the intent that applications preserve all the whitespace. This declared intent is considered to apply to all elements within the content of the element where it is specified, unless overridden with another instance of the xml:space attribute.