Blogs

David Corrigan is director of product marketing for IBM's InfoSphere portfolio, which is focused on managing Trusted Information. His primary role is driving the messaging and strategy for the portfolio of information integration, data quality, master data management, data lifecycle management, and data privacy and security capabilities. Prior to his current role, David has led the product management and product marketing teams for Master Data Management (MDM), and has worked in the Information Management space for over 14 years. David holds an MBA from York University's Schulich School of Business, and an undergraduate degree from the University of Toronto. Follow David on Twitter @dcorrigan.

Data refinement is one of the most important revelations in the big data market. The idea is simple: you want to take advantage of and use all sources of big data. But when each individual user needs only information relevant to them, what’s needed is a data refinery. It automatically cleans,...

Last month at Insight 2014, IBM made an exciting announcement: IBM DataWorks is available now. The reaction was overwhelmingly positive; clients who have or will soon have a data lake were very keen on the notion of data refinement. In fact, a data refinery is a natural fit with a data lake.

It seems that everyone these days is interested in big data: using more data, more quickly and making better decisions from it. How does your company interact with data? Or, more specifically, how do business users interact with data?

Confidence in information is vital for business users who make impactful decisions based on analytical insights. While easy to define, confidence has proven difficult to measure and visualize. To address this need, Aberdeen Group has developed the Information Confidence Calculator (ICC). The ICC is...

There have been a number of high-profile missteps with big data in the news. One recent example is the story of OfficeMax and the “Dead Daughter” marketing mailing (read more about it here http://onforb.es/1hhjVCL). OfficeMax accidently sent a mailing with “Mike Seay – Daughter Killer in Car Crash...

I participated in a really rich Twitter chat yesterday on the topic of "Evolving integration and governance for big data" with thought leaders Jim Harris (@ocdqblog), James Kobielus (@jameskobielus), Tim Crawford (@tcrawford) and Richard Lee (@InfoMgmtExec).
Sensitivity and error
The first topic...

Big data means more data – from more sources, in more formats. It also involves data that is created at a more rapid pace. All of those factors make it harder to establish context. Where did this data come from? How much do you trust it? What steps were taken to correct, or massage, the data?
In...

Protecting and security sensitive big data is necessary to ensure data is shared for new forms of analysis. Before the owners of that data will share it (yes, political silos still exist, and yes individuals still feel they own data and can say no to sharing it), they want to ensure it is...

Confidence in big data is highly variable. Some data sources have inherent uncertainty. So why shouldn’t you spend as much time as needed to make big data perfect? Time. You simply don’t have enough time to sort out every data irregularity, every ambiguity, every incomplete attribute. And for many...