In his post A Dictionary for Deep Space the other day, Kunal Sen used the scenario of communication with an alien world to raise a deep question about information. Coincidentally, he hit upon a pet hobbyhorse of mine. Sen’s conclusion was this:

The sum total of all the text we have collectively produced over the ages does not add up to anything more than a gigantic closed system with no real information value outside of this closed system.

Here’s another way of saying the same thing: “Information” does not exist as an independent entity or substance; what we call information is a mental event purely. That we think of it as something out there, in the world, is an instance of thinking by metonymy.

I am talking about a particular sense of the word “information,” what I call the engineering sense. As we use it in everyday speech, “information” refers to what we read in the newspaper, look up in Britannica, or hear from the person chatting to us on the train. But the meaning of the word for engineers and people who design computers and things like the Internet is different. They think in terms of bits – the elements that make up what it is that computers and networks store and manipulate; in short, the 1’s and 0’s.

What does it mean? I’ve no idea. I just made it up. But suppose it were the output of some measuring device, or in other words that it were data. The information value of each bit is, in simple terms, a measure of how surprising it is. That is, if we can’t predict what the next digit will be, then it has informational value; if we can, it doesn’t. As the dictionary puts it, information in this sense is “the attribute inherent in and communicated by one of two or more alternative sequences.”

Here’s an illustration of this point using letters instead of numbers: If I write

Happy Thanks*ivin*

you can make out what is intended despite my (not very clever) attempt to hide my meaning. So if I then offer to tell you what the missing letters are for, say, a dollar, you won’t buy because you know you won’t learn anything new. There’s no information value in the deal.

Now consider this simpler binary string:

1010101010101

You don’t know what it signifies, if anything. But if I tell you that it is the binary representation of a natural number, and if you are quick at mental math, you can do the conversion and arrive at the decimal number 5,461. So that’s the information contained in the string, you conclude.

But wait. I lied. The string actually represents the results of a series of coin flips, where 0=heads and 1=tails. Well, fine, you say; you understand that each flip of the coin is an independent event whose outcome is unpredictable, so you judge that this is genuine information in the string.

But wait. I lied again. The string actually represents successive positions of a light switch, where 0=off and 1=on. This is different. Given the first digit, every other digit is predictable, right out to infinity. From the “off” position the switch can only go one other place, and ditto from “on.” The information content of the string, you are forced to concede now, is thus zero.

In all three cases the string of digits, which you supposed to somehow contain information, is the same. What changed? The frame of interpretation you brought to bear. You said, in one case, “I will interpret this string as representing a binary number.” Then you said “I will interpret this string as representing coin flips.” And finally you said “OK, now I will interpret this string as representing switch positions.” Until I told you what frame was appropriate, you couldn’t make anything of it; nor could you discover on your own what was the appropriate frame, because the string itself provides no clue.

Absent some frame of interpretation, the string is just a set of marks, intrinsically meaningless. No information content. The “information” we think we derive from it is something that exists only in our heads as a result of this purely mental process of interpretation.So why do we speak of “information” in these digital matters? This mental process of interpreting a signal to derive meaning is precisely the same as in the everyday experience of reading, listening, and so on – all the ways in which we gain what the dictionary gives as the first sense of the word: “knowledge obtained from investigation, study, or instruction.” The metonymy consists in labeling the stimulus – some pattern of marks or energy out there – with the name properly belonging to the interpreted result in here.