from atoms to bits and back again

Menu

From Data to Knowledge and Beyond…

There has always been plenty of disagreement about what defines data, information and knowledge. If you are using these words when defining system requirements, it can be beneficial to calibrate your definitions with potential suppliers, partners and customers.

These are the descriptions that I use. Definitions provided in quotes are taken from the Word IQ web site:

Data

“A datum is a statement accepted at face value (a “given”). Data is the plural of datum.”

Data is a set of discrete facts that describe something – an object, event, whatever. There is no interpretation or judgement. Data alone will not tell you what to do, or what action (if any) should be taken.

Accuracy of data is critical since all information will be based on it. In most scenarios, computers are better than people at recording, calculating and retrieving data – in terms of efficiency, accuracy and consistency.

Information

“Information is a message, something to be communicated from the sender to the receiver, as opposed to noise, which is something that inhibits the flow of communication.”

Information is created by interpreting data and making judgements. It is a message that should change the way the recipient perceives something. The process of creating information is consistent – the interpretation and judgements may change as the underlying data changes, but the rules used to make those interpretations and judgements remain constant unless there is an explicit reason to change them.

The process of creating and sending information is increasingly being automated by technology. Computers are starting to do a better job than humans at categorising and analysing data, spotting patterns and anomalies, and presenting recommendations. Recent reports have even shown computers out-performing doctors at interpreting x-rays… Fortunately for us, in systems centred around people, at least one recipient in the process does still need to be a human.

Knowledge

“Knowledge is the awareness and understanding of facts, truths or information gained in the form of experience or learning.”

Knowledge is created by people experimenting with different sources of information in different contexts. Because much of the process takes place in the mind, it cannot be easily observed or recorded (or automated by technology for that matter, humans still rule here). Knowledge comes from testing, breaking and adapting the rules used to create information. The environment in which knowledge is created is unique – the exact mix of information and context cannot be reproduced and the knowledge created may, or may not, be applicable to other scenarios. In short, knowledge can be messy and unpredictable.

Too many knowledge management systems downgrade knowledge to information in order to retain and re-use it. But failing to capture the reasoning behind knowledgeable decisions risks losing the true value of the knowledge.

Wisdom

“Wisdom is about making the best use of available knowledge”

Wisdom is the combination of experience with knowledge to distinguish between the plausible and the possible, with increasing accuracy and speed as experience and knowledge accumulate over time.

The following excerpt from the Cafe Hayek blog, written by Don Boudreaux in September 2004, compares wisdom to its poor relation, cleverness (and be warned, knowledge can lead to cleverness more quickly than it leads to wisdom):

Cleverness is not wisdom. it is not necessarily inconsistent with wisdom, but too often cleverness is mistaken for wisdom. And also too often, cleverness crowds out wisdom… It’s important to be aware that the range of the possible is enormously larger than the range of the plausible. Wise people focus on the latter. Stupid clever people get all giddy and excited about the former.

When Ray Kurzweil talks about computing capacity surpassing brain capacity, I believe it is true in relation to information, but not for knowledge and certainly not wisdom. A computer is a long long way from understanding why animals make what can seem to be irrational decisions. That irrationality comes from behaviours driven by emotion, something a computer does not (yet) have. Emotion has a strong influence on knowledge and wisdom.

A simple scenario

To demonstrate the journey from data to knowledge and beyond:

Scenario:

September 2005, driving from the office to home. News report mentions that anti-fuel tax protestors are planning a demonstration to mark the anniversary of the 2001 protest. Fuel companies urge people not to panic-buy, stressing there is plenty of stock to meet normal demand.

Data:

Fact: Journey from office to home = 83 miles

Fact: Petrol remaining in tank (according to computer in car) = 117 miles

Fact: Petrol stations located on journey home: 2 near to work, 1 at mid-point on motorway, 2 close to home

Fact: Remaining petrol in car, on reaching destination, will last approximately 30 miles

Information:

Interpretation: The car has enough petrol to complete this journey, but will run out early in to the next journey. However, 30 miles is ample fuel to reach one of the petrol stations
close to home, as has been successfully tested on previous occasions. The news reports are stressing that there is no problem with fuel supplies at petrol stations and there is no need to panic.

Normal Decision rule: It is a bit soon to fill up when departing from office (the tank is still 1/4 full); won’t want to go to motorway service station because the junction will be busy due to rush hour traffic; won’t want to be bothered by the time I get close to home – by then, eating food will take priority

Normal Judgement: Fill up the car at the start of the next journey

Knowledge:

Context: The anti-fuel tax protest in 2001 led to panic-buying – long queues and petrol stations running out of supplies. Chances are, people will ignore requests not to panic-buy this time in fear that the same will happen again

Temporary edit to decision rule: Check first petrol station on journey home to compare against news report. Surprise, no queues. Perhaps people are being sensible this time…

Judgement update: Decide to fill up at the last petrol station before home. I think panic-buying will start to happen and I don’t want to risk being unable to fill up in the morning with only 30 miles of fuel left in the tank

Cleverness:

Context: People do panic and start going to petrol stations to fill up, even if they’ve still got three quarters of a tank full. News reports on the radio talk about queues building and petrol stations running out of fuel, more people start to panic, and so the vicious circle begins…

Wisdom:

New decision rule introduced: The normal rule is to never, ever, ever, queue for anything – I hate queues, queues are to be avoided at (nearly) any cost. Whoever said the British like queuing didn’t ask me

Context: Last petrol station from home has a queue

Judgement update: Normal rule to be ignored; will resist urge to abandon queue and fill up in the morning. Risk that petrol stations will be empty in the morning has now been upgraded to high. Will sit in the queue, cursing the stupid clever people (clearly visible by the short amount of time it is taking for them to fill up their cars)

Outcome:

Had to queue for 20 minutes at local petrol station. Muttered about stupid clever people to bloke filling up alongside (his car was empty too). Both local petrol stations ran out of petrol within 30 minutes of opening the following morning… 36 hours later, people finally twigged there was no blockade at the fuel depots, and normality was restored.

Ah, the benefit of wisdom and knowledge…

Interesting side note on this story – different areas of the country responded differently to this mini-crisis. The likelihood of petrol stations running out of fuel correlated to the reporting style of the local news channels. Reporting in some areas focused on the concerns about running out of fuel, whilst other areas focused on why there was no need to panic because stocks were being kept higher to prevent a repeat of 2001…

References

Book: Working Knowledge, by Thomas H. Davenport and Larry Prusack (1997, Harvard Business School Press)