Global CIO: Data Mining Faces The Supreme Court Test

A high court hearing raises a question critical to the number-crunching business world: How much should individuals be able to control the data companies have about them?

Pharmaceutical companies know an astonishing level of detail about individual doctors -- things like how often a doctor prescribes a certain drug, if he’s started prescribing a rival drug, or if he’s dropped name brand drugs to prescribe mostly generics.

A Supreme Court hearing today is going deep inside the business of pharmaceutical data mining. Consumer interest is rising into what companies know about whom, amid fresh examples of data gathering by companies, including revelations that Apple's iPhone collects location data on the device and on computers used to sync the device. As worries rise about how much companies know about us, and how they use the data, lawmakers are sure to look for ways to regulate such information. The Supreme Court case, Sorrell v. IMS Health, could determine how far lawmakers can go in restricting data use.

The case focuses on a Vermont law that would let doctors decide whether their names could be sold to pharma companies for marketing purposes.

Doctors generally don't have any say in whether drug companies get information about their prescribing habits. Pharmacists sell the data to data mining companies, which then sell data and analysis to pharmaceutical companies with patient names removed or encrypted. That data fuels drug companies' CRM efforts -- what's known in pharma as "detailing" a doctor or practice. Pharma companies tailor their marketing to doctors based on those profiles, anything from giving free samples to docs currently prescribing a rival drug to sending marketing materials and research only to docs likely treating patients with a rare condition.

Under the Vermont law, doctors had to give consent for names to be included in data sold to drug companies; without consent, doctor names were kept secret, just like patient names. Vermont lawmakers worried that there was a "massive imbalance in information" -- that doctors were being bombarded by pharma marketing, which was driving up costs as doctors prescribed branded drugs over generics. The state argues that pharmacists have the doctors' prescription history only because state law requires them to keep it, so the state can control what happens with that data. From Vermont's Supreme Court brief:

"Any doctor that finds this form of marketing beneficial may consent, and communications to that doctor will be unaffected. By letting doctors, rather than the State, control the use of this information for marketing, the legislature avoided impinging on the "protected interest" in communication between pharmaceutical manufacturers and willing doctors."

Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.

Why should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.