In all of the above the customer complained about the system
inability to process the amount of data, either being too slow or it
simply could not be completed at all meaning the process erred out.In all of the above examples an escalated defect was logged, and development worked troubleshooting the problem.

An
additional common theme between all the above examples and many more is
that there was no real need to have a product fix although performance
improvement opportunities may have been identified, they were not the
final solution that addressed the problem.What helped was
understanding that there is no real need to process all the data and
recognizing the data that really needed to be loaded. The data could
have been loaded in a more efficient way using some best practices and
creative thinking.

The scope of this document is to provide some guidance how to troubleshoot such customer situations.The document will focus on three main areas:

Learning and understanding the business needs

Understanding the actual data that needs to be loaded, understand
the gap between this number and the number of rows that the customer
actually tries to load.

Provide some best practices that can help the customer work with the data efficiently.

This document will not deal with the initial Data load processes
although some of the best practices can be adopted for such situations
as well.We will illustrate the concepts in this document using three
real life examples of service requests and/or defects that were logged
on behalf of the customer.