BIG DATA revisited – a survey

I have the good fortune of attending some presentations by people from Cisco, IBM, PwC, TCS Research Lab today in a Business Conference in Calcutta. The focus on the issue was how to harness the business opportunities arising out of these ‘strange’, ‘disruptive’, ‘fast-changing’ environment.

I heard all the experts, read material recommended by them, checked with their clients, discussed with some and here is the WLSfBG (Wordsmith Labour Saving for BIG DATA)

1. There is a great need, rather the most-important need for a disinterested (means the document is not going to sell some BIG DATA product / solution at the start, middle or end using fear, hope, confused smugness, intelligent sensors radiating non-sense and the fail-safe method: a fool and his money parts relatively easily) document explaining the fundamental core of BIG DATA.

2. Human beings, most intelligent ones of the species have awakened to the discovery of a profound ability-gap. The ability to produce data and the ability of managing data points to a Malthusian formula. Data is produced at exponential rate and managing capability is happening in non-exponential rate.

3. There are two kinds of data : structured and unstructured. Generations before had built their mansions and fortune by selling us boxes and tools to manage structured data. Present generation (business included) is desperate to have their larger mansions and unstructured data is the area now.

4. Some presenters were so grave as if narrating a tragedy that we are analyzing barely 10% of the data being generated. What nuggets are left from the 90% that remains analysed. This is a good business opportunity. This hope however pre-supposes, a priori that the left-over has treasure. I say a priori because it is a hypothesis and a premise, not tested or validated yet (see ability gap 2 above)

5. One gentleman, a banker actually gave a clue who said that his bank does 90 million transactions each day and how to visualize this thing or to bring this within our cognition. I give up. I cannot visualize this. May be, there is nothing to visualize. This is something like a zen koan – the sound of a single hand clapping or visualizing flood evacuation procedure in sub-saharan Africa.

6. One young architect, with great glee told that the day of the engineers have arrived. It was nasty bad business driving engineering but now with BIG DATA, Analytic etc, the table is turned. His enthusiasm was infectious as youth itself is an infection of genes where they start sending so many chemicals in the bloodstream and the world appears so fresh and new. I remembered in the end of the 1990s similar feeling prevailed among young telecom and network engineers. This will have implications in the future, I mean, this feeling of the table being turned. I also foresee many companies horoscope showing definitive trend towards meeting bankruptcy judges. (I have used a proprietary BIG DATA process to predict future. The process is real cutting-edge, I assure you.)

7. Another interesting idea was to see who non-IT thing will interact with the I-o-T (Internet of Everything) fabric ? In other words, what will happen if we put sensors in everything we have been using since the beginning of time considering them ‘passive’. Now your shoe will have sensors and if you lose your job and throw your shoe in frustration, your mobile will get a sms recording ‘Abuse Score Alert’ and after a certain point, your Insurance will alert of ‘potential risk score high from the accelerator controlling bionic arm (actually your leg)’. The river of forgetfulness bottle you are supposed to open to down your pain will refuse to accept the bar code and God fobid if you throw the bottle down when the gyro will record, auto-adjust, alert the closest police beat and you will be picked up. Does it sound very pleasant ?

8. No one gave us any information as how pricing models for these services are being developed. This reminds me a story where I asked a Social Media Marketeer as how much is he paid per month for a client and on what metric. He mentioned a dismally low figure. I asked him : how are you managing at this pricing ? He answered : ‘ At such price, they have no right to ask about RoI’. Do not laugh. This is a very very subtle and tested business method. And I give you the clue :