Can anyone help me support the following statement with a published reference?
"Fundamental information theory concepts—entropy, mutual information,
relative entropy, and channel capacity—are functions of a more
primitive concept, the surprisal (S). "
I am aware of use of the term surprisal by Tribus, but I have not
found an information theory textbook that defines entropy as the
expected value of the "surprisal".
Thanks in advance for your help.
Bill Benish
