Import set performance

SAVE AS PDF

Import set performance

The algorithm transforms import sets from their staging table into their final
destination.

Importing data via an import set requires two phases:

The data is loaded from a data source into a staging table

The data is transformed from the staging table into a target table

The transform algorithm operates in "blocks" of 100 records at a time as opposed to the
previous algorithm which transformed a single record at a time. The newer approach allows the
application server to pre-fetch a variety of information relevant to each block of records,
reducing the number of unique interactions with the database and improving throughput.

Who should expect to see a performance improvement?

Any customer using import sets should expect to see a performance improvement in large
transformations.

What kinds of transformations benefit the most from these changes?

Transformations with a large number of reference or choice type columns see the largest
improvement.

What kind of transformations benefit the least?

Transformations that with complex or unkeyed coalesce conditions see a proportionately smaller
benefit.