Guy Hoelzer  wrote or quoted:
> in article chsg65$1hqg$1{at}darwin.ediacara.org, Tim Tyler at tim{at}tt1lock.org:
> > Guy Hoelzer  wrote or quoted:

> >> In this way, the sensitivity of p_i to individual differences in
> >> perception is minimized.
> >> This transition in the meaning of p_i happened without much
notice for many
> >> (most? The vast majority?) information scientists because it
increased the
> >> extent of objectivity and measurement agreement among scientists, and
> >> extended the utility of information theory throughout the
physical sciences.
> > 
> > Nothing fitting this description ever happened.
> 
> Are you arguing that treating p_i as frequency is almost never done, or that
> this practice has not increased in frequency?  Or are you just arguing that
> you don't think it has become sufficiently common to call it a transition?

p_i is /always/ the probability of the i'th symbol arising.

Sometimes the probabilities are determined completly by symbol frequencies 
- but the p_i's are never frequencies.

They always add up to 1.0 - like probabilities do.

> >>> An observer who knows what symbol is coming next (because he
> >>> has seen a very similar message before) will assign different
> >>> probabilites to the symbols from an observer who is seeing
> >>> the message for the first time - and both will assign different
> >>> probabilities from an observer who is not even familiar with
> >>> the language the message is written in.
> >> 
> >> This is a nice description of the (severe IMHO) limitations of the
> >> "telegraph"-context-laden version of the theory
that Shannon originally
> >> devised for his telegraph-company employer.  With all of your
protestation
> >> about my lack of fidelity to Shannon's original context, you haven't
> >> suggested any reasons why treating p_i as a frequency, rather than a
> >> probability, is problematic.  Can you think of any such problems? [...]
> > 
> > The ones above?
> 
> I didn't see any problems suggested in your previous post or in the material
> I snipped above, which was a description of how probabilities and
> frequencies differ.  Your argument then seemed to consist merely of saying
> that Shannon originally meant p_i to be a probability, rather than a
> frequency, to which I already agreed.  None of that addresses my question.

If you used freqencies, it would be equivalent to considering what
new would be learned by an observer with very little brain - whose only 
knowledge of the future consits of measuring symbol frequencies, and 
assuming what has happened before will happen again.

Such observers are surely not common.

> > p_i can only be treated as a frequency, *if* the source is something like
> > a RNG - where the probability of symbols being emitted is constant - and
> > does not depend on the history of the stream or environmental conditions.
> 
> That may be the condition under which a probability and a frequency are
> interchangeable, but it still does not address the issue at hand.  Given the
> differences between probabilities and frequencies, why isn't it better to
> think of p_i as a frequency instead of a probability as Shannon first had in
> mind?

Currently Shannon's information represents the "suprise value" of
a message - an estimate of how unlikely the observer thinks it is
to be received.

I.e.:

``Shannon's entropy measure came to be taken as a measure of the 
  uncertainty about the realization of a random variable. It thus served 
  as a proxy capturing the concept of information contained in a message 
  as opposed to the portion of the message that is strictly determined 
  (hence predictable) by inherent structures.''

 -
http://www.all-science-fair-projects.com/science_fair_projects_encyclopedia/Information_entropy

An estimate of how likely a particular, stupid observer thinks it
is to be received seems likely to be lacking in utility by comparison.

> > That is certainly not the general case - and it is not the case with many
> > common sorts of message streams either.
> >  
> >> If not, then don't you think it is worth considering the more extensive
> >> version of the theory?
> > 
> > It isn't "more extensive".
> 
> It is more absolutely extensive in its potential application and the breadth
> of its explanatory power because it overcomes the limitations of trying to
> approximate identical states among observers before they can agree upon the
> information content in a data set (an observation).

It is equivalent to calculating Shannon information for a rather dumb 
observer who is unable to perform simple logical reasoning :-(

It doesn't completely overcome the supposed "problem" of information being 
subjective  - since agents can still differ on the issue of the frequency 
of source symbols - depending on how much of the stream they have seen.

....and it would mean that the term "information" no longer represented
the "suprise value" of a message to most observers - and that's pretty
much the point of the term.

I don't see the resulting metric as being of much interest.

The term "information" already has a good meaning - and what you are
describing isn't it.
-- 
__________
 |im |yler  http://timtyler.org/  tim{at}tt1lock.org  Remove lock to reply.
---
þ RIMEGate(tm)/RGXPost V1.14 at BBSWORLD * Info{at}bbsworld.com

---
 * RIMEGate(tm)V10.2áÿ* RelayNet(tm) NNTP Gateway * MoonDog BBS
 * RgateImp.MoonDog.BBS at 9/11/04 9:47:06 PM