Optimizing Data Cleansing Performance

The following are recommendations for achieving good performance with data cleansing when working with large volumes of data:

Include only new or recently modified records in the batch data cleansing process.

Cleansing all records in the Siebel database each time a data cleansing is performed can cause performance issues. Include an Object WHERE clause when you submit your batch job, as shown in Table 28. Split the tasks into smaller tasks and run them concurrently.

Table 28. Recommended Data Cleansing Object WHERE Clause Solutions

To Cleanse

Use This in Your Object WHERE Clause

Updated records

[Last Clnse Date] < [Updated]

New records

[Last Clnse Date] IS NULL

Updated and new records

[Last Clnse Date] < [Updated] OR [Last Clnse Date] IS NULL

To speed up the data cleansing task for large databases, run batch jobs to cleanse a smaller number of records at a time using an Object WHERE clause. For more information about data cleansing for large batches, see Cleansing Data Using Batch Jobs.