On 2007-12-06, Doug Wedel <dougwedel from earthlink.net> wrote:
> I have never posted in here but have read a great deal from the Chowder
> group. Now I wish to pose a question.
>> You use Shannon's definition of information as "decrease in the uncertainty
> of a receiver." However as you well know this definition contains a nasty
> paradox: information defined this way is at its maximum in random numbers.
>> The idea that information is at its maximum in random numbers is not just
> different from our intuition, but more like opposite to it. As Richard
> Feynman put it in his Lectures on Computation, "How can a random string
> contain any information, let alone the maximum amount? Surely we must be
> using the wrong definition of 'information.'"
>> How do you deal with this paradox in Shannon's definition of information?
Where did you see Shannon's definition of information stated as a
"decrease in the uncertainty of the receiver"? If anything, I believe
that should be an increase in uncertainty; as perfect certainty equals
zero information transmitted. Shannon's entropic formulation of
information amounts to the number of yes/no questions one must ask
before the "message" is known. As this number of questions increases,
(at least statistically), then the information content is viewed as
increasing. Shannon generally thought in terms of an "alphabet" as the
set of possible messages, (i believe he was thinking in terms of
teletypes and noisy transmission lines). If there are equal
probabilities of all possible members of this set then the information
content is maximized. Is this equal probability concept where you
introduce your "random numbers"?
Perhaps others are aware that there is a paradox associated with
Shannon's work connecting entropic formulation to information content,
but I don't see it myself.
I hope this was of use to you.
--
r
