Zipf on language change/structure

[Joan Bresnan]

Matt, there is a relevant, up-to-date discussion of Zipf's law(s) in
Chapter 1 of

  Christopher D. Manning and Hnrich Schuetze (1999) _Foundations of
Statistical Natural Language Processing_, The MIT Press.

In addition to discussing Zipf's work and subsequent work by
Mandelbrot (1954) which provides an improved fit to texts over Zipf's
law, they make the following observation (pp. 28-9):

  "As a final remark on Zipf's law, we note that there is a debate on
how surprising and interesting Zipf's law and `power laws' in general
are as a description of natural phenomena.  It has been argued that
randomly generated text exhibits Zipf's law (Li 1992).  To show this,
we construct a generator that randomly produces characters from the 26
characters of the alphabet and the blank (that is, each of these 27
symbols has an equal chance of being generated next.) ... One can show
that the words generated by such a generator obey a power law of the
form Mandelbrot suggested.  The key insights are (i) that there are 27
times more words of length n + 1 than length n, and (ii) that there is
a  constant ratio by which words of length n are more frequent than
words of length n + 1.  These two opposing trends combine into the
regularity of Mandelbrot's law. ...

   "There is in fact a broad class of probability distributions that
obey power laws when the same procedure is applied to them that is
used to compute the Zipf distribution: first counting events, then
ranking them according to their frequency (Guenter et al. 1996).  Seen
from this angle, Zipf's law seems less valuable as a characterization
of language.  But the basic insight remains: what makes
frequency-based approaches to language hard is that almost all words
are rare.  Zipf's law is a good way to encapsulate this insight."


Best wishes,

Joan

> Hi folks,
>
>      In connection to the recent posting by Bill Croft and the recent
> interest in a usage-based explanation of language change-frequency inter
> alia (Bybee and Hopper 2001)-I would like to know how Zipf's early work on
> frequency and language structure has been (re)assessed, if at all. I know
> Jack Hawkins is trying to formalize Zipf's insights and to connect them with
> the processing theory. We also recall Haiman's (1983) utilization of Zipf's
> economic motivation in his discussion of reflexives/middles and reciprocals.
>
>    The following is the quote from the relevant sections from Zipf, G.
> (1935) THE PSYCHO-BIOLOGY OF LANGUAGE.
>
>
>
> Thanks,
>
> Matt Shibatani
>
>
>
> "In view of the evidence of the stream of speech we may say that the length
> of a word tends to bear an inverse relationship to its relative frequency;
> and in view of the influence of high frequency on the shortenings from
> truncation and from durable and temporary abbreviatory substitution, it
> seems a plausible deduction that, as the relative frequency of a word
> increases, it tends to diminish in magnitude. This tendency of a decreasing
> magnitude to result from an increase in relative frequency, maybe
> tentatively named the Law of Abbreviation." (38)
>
>    "The law of abbreviation seems to reflect on the one hand an impulse in
> language toward the maintenance of an equilibrium between length and
> frequency, and on the other hand an underlying law of economy as the causa
> causans of this impulse toward equilibrium." (38)
>
>
>
>     "The magnitude of complexity of speech-configuration which bears an
>
> inverse (not necessarily proportionate) relationship to its relative
> frequency,
>
> reflects also in an inverse (not necessarily proportionate) way the extent
> to
>
> which  the category is familiar in common usage." (272)
>
>
>
> "The degree of distinctness of meaning.seem[s] to bear an inverse
> relationship to
>
> F[requency] and C[rystalization] [of the configuration]."  (157)
>
>
>
> "the minimal degree of complexity necessary for comprehensible speech
>
>   between persons reflects the degree of unusualness (in their group) of the
>
>  experiences spoken of, or, somewhat more precisely stated, the unusualness
>
>  of speech about those experiences." (273)