We often think of the relavence of data when we want to include or exclude it from analysis or process. However, are you thinking about relavence as part of your data quality effort?

Just as you focus data quality efforts to clean existing information, there are invariably records that can’t be cleansed or enhanced. They have no value in either business analytics or business process. They are noise, similar to the noise you have when there is bad data. To save and maintain them in your database can affect your ability to accurately analyze information, continue to deflate confidence in data, and if a significant percentage of your database, will cause problems in performance and added maintenance. Developing an archival strategy as part of your data quality practice is a significant component that should not be overlooked.

Benefits of Data Relevance

Trust in data

Enables process

Accuracy of analysis

Supports decisions

Database optimization

It can be tempting to simply delete records from your databases. Though, this can have a detrimental affect due to data dependencies within your databases as well as causing non-compliance in regulated environments. Instead, it is best to formulate a strategy that flags non-relevant data removing or suppressing it from user interfaces and analytics.

Components of Archiving Strategy

Data decay rates – Attributes of records that loose relevance over time. This component is a good guide on the frequency at which you will focus cleansing efforts. It also provides an indicator on when data is approaching a horizon when a record will lose its relevance. Age of the data and activity related to a record, even if a record is complete, can signify whether the data is relavant and open to archiving.

Minimum requirements of record viability – Records should continually be assessed to determine if they meet the minimum standards of use. Failure to meet minimum requirements is a leading indicator that the record is a candidate for archiving.

Relevance of record to analysis, process, decisions – If a record is not going to be used in analysis, process, or decision making, there is not need to keep it in use. This may be the case if processes have been optimized and certain information is no longer needed. Or, it could be that it was a candidate for archiving due to decay rates and minimum data requirements. Additionally, relavance may be determined when integrating systems where old records with old transaction history is not relevant to the existing or new business.

Regulatory compliance – In highly regulated environments like health care, there are standards on what you can and cannot remove. Records may not be useful in existing process, analysis, and decision making, but might be required in certification or other compliance related activities. Archiving ensures that information is not deleted from primary systems. Although, you may have to provide a mechanism that provides adequate access to data for compliance.

An archiving strategy is a critical component of data quality best practices. It will continually help you focus on improving and refining your data quality projects as well as thinking strategically about how you use and manage your data on a daily basis. Establish an archiving strategy at the forefront of your data quality initiatives and you start your efforts off on the right foot.

How many times have you heard that? It’s become the standard mantra. It is so ubiquitous that I don’t think anyone questions anymore the validity of the statement. It just is. However, this is probably the hardest part to facilitate when building out you business intelligence practice. Facilitating decisions is what makes BI stragetic.

Just what is the business decision? What does a business decision look like?

A typical approach during the business analysis phase for BI is to at business decisions across a business process and where questions are asked to change behavior in that process. Although, the difficulty with this level of granularity is that it is too deep. These transition points are tactical. Intelligence across this process and at these decision points is important, but you don’t get the strategic value of BI at this level. You need to look at the outcome of the process and provide a platform that supports the decision of what to do next. This is the unstated question.

Let’s take an example. Sales management will always want a perspective on the pipeline and forecast. This shows them how they are meeting their numbers quarter to quarter. However, outside of conversion and volume, there are business decisions that sales managers need to make. Should they adjust their territories to capture new opportunity or shore up existing business? Are there changes needed in commissions to incent sales people along certain products and services to improve profitability or revenue? BI can lead sales management with insights that will guide them to optimize their processes and management rather than just data.

To align BI to the business decision it is important to include executives in the discussion. Get beyond the reports they want to see and ask the question about how they manage their business. Walk through scenarios of what they ask as changes in the market or the business arise and how information can help them make a decision. The better able you are to see how they manage their business, the more valuable the BI practice will be to supporting the business.

Reading the buzz on the Jim Davis’s presentation at SAS Global Executive Forum, what it made me realize is that if as an industry we can’t agree on what Business Intelligence is or Business Analytics, how are we supposed to make sense of it in implementation?

You have analytics players, enterprise application vendors, business process consultants, and analysts all trying to sell the ‘hype’ of a better way to analyze your business and makes decisions. SAS wants to sell their analytic solution that really pioneered data mining in businesses. Oracle and IBM wants to push dashboard solutions that links to business processes and their enterprise applications. Gartner that tries to tie together people, process, and technology but is really is focused on what technology to buy. Then, you have consultants that are trying to help you implement the technology even as they document your processes. The problem is that it’s all boiling down to the one with the best tool wins.

Enter in the ‘Business’ and now you have a problem. All they want to know is how they can meet their business objectives. IT is trying to sell the solution and make them understand the technology, and the business glazes over and can’t figure out what to focus on. I’ve sat in these discussions where IT tells me, “You tell us what to do, we’ll do it. Don’t worry about the solution.” It is open ended. This leads to IT unable to work towards tangible goals and results. The business walks away frustrated, projects run from months into years, and original budgets are thrown out the window. I liken these projects to Boston’s Big Dig.

“I don’t like the term business analytics; it doesn’t tell me anything. Frankly, I think business intelligence as a term is downright laughable, too. What does that mean? Is integrating data intelligence? Is generating reports intelligence? Maybe its informing, but isn’t intelligence something you HAVE not something you do? Does doing what we call BI lead to intelligence, or just some information? A long time ago we called this decision support, and that gets my vote.”

So here’s my take on what steps to take when and how to venture into BI and analytic solutions.

Steps:

What decisions need to be made?

At what point in our business and business processes are these decisions made?

What information is needed at these points?

How should our applications and data provide this information – triggers or visualization?

See the steps? It starts with the business decion and ends in the technology. So, when you begin to review vendors and solutions, make sure you have steps 1,2,3 in mind before you determine how to solve step 4.