Information is Random, Complex and Interesting

Prompt: Read Gleick 324-354 on randomness and information. Note the distinction between interesting and uninteresting numbers. As we start thinking about the mid-term, I'll be looking for interesting essays from you. What might that mean?

Chapter 12 of Gleick's "The Information" titled "The Sense of Randomness" adds to the definition of Information. The main character of the chapter, Gregory Chaitin, is a New Yorker mathematician. His  studies furthered theory about algorithms. While Gleick explained that messages being random, complex and interesting translate to containing more information, I kept thinking about Berry's Paradox (179). It helped me understand the purpose of an algorithm. If a complex idea can be stated in fewer terms (or fewer bits) than an algorithm helps us compute a message.

"But Shannon also considered redundancy within a message: the pattern, the regularity, the order that makes a message comprehensible. The more regularity in a message, the more predictable it is. The more predictable, the more redundant. The more redundant a message is, the less information it contains," Gleick writes on page 329. So it seems that the easier it is to create an algorithm for message, the less information it contains. Those messages which require more complicated algorithms to illustrate become more "interesting." And in this chapter, "interesting" has become another measurer of information.

Later in the chapter, Andrei Nikolaevich Kolmogorov calls this randomness complexity: "the size, in bits, of the shortest algorithm needed to generate it. This is also the amount of information" (337).

The more random a message is, the more interesting it is, and the more information it contains. And "random" and "interesting" are not subjective terms in this discussion: "Ignorance is subjective. It is a quality of the observer. Presumably randomness – if it exists at all – should be a quality of the thing itself" (326). 


On page 331 when pi π is explained to be a random number but not in the subjective sense, we learn that math and science can help us pull meaning out of seemingly random numbers. Gleick states that if a number can be computed or defined, it reflects on its randomness. And since algorithms produce patterns, looking at the size of an algorithm reveals computability. Since an algorithm is measurable, w gauge the size of a message in bits (332). Pi is a random number (its numerals do not repeat in patterns) but since we can compute it, it loses the 'random' tag. Same with the number 1729, or the Hardy-Ramanujan number (339).

It's been difficult reading these sections because I can't put my finger yet on how this translates to information in general. Are we talking about decoding foreign, animal or alien languages? Or just quantifying/commodifying Information? 

John von Neumann said, "Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin" (327). Since now we do have algorithmic ways of producing sets of random numbers (as Gleick states earlier on that page), perhaps what Neumann was getting at was Information. New Information cannot be produced arithmetically – it requires "outside the box algorithm" thinking.

Ultimately, when Professor Edwards collects our midterms, he will be looking for "interesting essays." If he is using the objective term as Gleick and Chaitin do in the chapter, then he will grade with the expectation that we have come up with sentences and thoughts that are not just arithmetically concocted from our readings and discussions (our class "algorithms"). The less formulaic our essays are – with quotes pieced together and commentary inserted that has already been mulled over in class – the more interesting they will be. We must work outside our given algorithm ("given" meaning what has been bestowed upon us – we have to seek out and bring in new things).

However, Gleick later brings up the fact that algorithmic information theory is basically the way scientists operate. "When humans or computers learn from experience, they are using induction: recognizing regularities amid irregular streams of information. From this point of view, the laws of science represent data compression in action. A theoretical physicist acts like a very clever coding algorithm" (345).

"...perfect understanding is meant to remain elusive. If one could find the bottom it would be a bore." This applies to human questions, Gleick says, like art, biology and intelligence (353). 

Charles H. Bennett pulls it all together. He stated that "it has been appreciated that information per se is not a good measure of message value" (353). Information is valuable when it is "neither too cryptic nor too shallow, but somewhere in between" (354). Valuable Information is computable but requires us to keep thinking to expand and contextualize the message.