Insurance data mining

Modern companies accumulate staggering amounts of information related to their business activities. Record keeping is by no means a recent development, but traditional methods of sorting through data and making sense of it are no longer capable of handling the volume afforded by current technology. The contemporary solution to the seemingly infinite quantity of information is data mining, which uses statistics and mathematical techniques along with artificial intelligence processes to condense large sets of data into useful information and reveal patterns.

Data mining can be applied to virtually any field, and one that has made extensive use of the process is the insurance industry. The insurance marketplace is fairly saturated and intensely competitive, largely due to the fact that it is a take-away business. New customers generally must be enticed to leave a competitor. This factor alone makes data mining valuable to insurance companies who want to determine how best to meet potential customer's needs while remaining profitable, but another aspect of the insurance industry relies on data mining even more heavily.

The prime determiner of profitability for an insurance company is the cost related to claims payment and claims handling. Claims, especially complicated bodily injury claims, traditionally involved massive amounts of paperwork and frequently were resolved with liberal use of personal judgment. Today, claims adjusters employ proprietary personal injury evaluation systems for decision support. Adjusters routinely use data mining to identify patterns in prior claims databases, investigative databases, medical procedure cost databases, and many other resources to assist them in the claim process.

In order to be advantageous, data mining applications must have certain characteristics. Data mining is intended to identify patterns that uncover associations, predictions, clusters, and sequential relationships. For a data mining application to be valuable it should address key processes, such as understanding the purpose behind the scrutiny of data as well as understanding the data that is relevant to that purpose. Claims adjusters make use of categorical data such as age, sex, and vehicle models as well as numerical data like collision frequency and average costs. Insurance companies need a data mining application to accurately relate that data to the goals of predicting claim amounts and preventing fraud. A good data mining application should also efficiently preprocess data, which consists of consolidation, cleaning, transformation, and reduction. Additionally, the data mining application ought to use the most applicable model-building, evaluation, and deployment methods that suit the organization using it.

The data mining applications used by claims adjusters have been refined over the years. Their initial objective was to fill a gap caused by downsizing that resulted in fewer experienced senior adjusters. Less skilled adjusters were expected to make judgments they were not fully qualified to assess. Introducing data mining along with other decision support tools not only solved the problem of inexperienced claims adjusters that insurance companies faced, it simultaneously presented opportunities that have been exploited ever since. Costs have been reduced because lower level adjusters require less training and less pay. They rely on personal injury evaluation systems rather than "gut feelings" and other fallacious decision methods. Criminals attempting to submit fraudulent claims are now frequently identified and kept from successfully bilking insurance companies.

Data mining is a powerful tool, and data mining methods are usually complex and esoteric. The most frequently used method, classification, can involve techniques such as k-fold cross examination, bootstrapping, and decision tree analysis. Other methods employ determining optimal clusters, fuzzy logic, apriori algorithms, and even artificial neural networks that mimic the structure of the brain. The concepts that make data mining possible are far beyond the average user of the application, whose focus is on results rather than methods. For that reason, it is important for data mining applications to be both flexible and simple to make use of. End users need to be given useful information while being steered away from common errors related to data mining.

Ease of use and flexibility are important for data mining applications to potentially prevent mistakes like choosing the wrong problem, not allowing enough time for data preparation, looking at aggregated results without considering individual records, and not keeping track of procedures and results. Other problems a clear and understandable data mining application can help avoid are disregarding suspicious findings and believing everything said about data and data mining analysis. In the example of less experienced claims adjusters, the user has no option but to use the provided personal injury evaluation system. The benefit gained by using such a system would be nonexistent if the system, including the data mining portion, was not straightforward and adaptable to each claim.