Zipf-Mandelbrot law

Revision as of 03:27, 4 January 2008 by Jackbot (talk | contribs) (Robot: Automated text replacement (-(?ms)^(.*)$ +\1 {{WikiDoc Sources}}))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Zipf-Mandelbrot
Probability mass function
Cumulative distribution function
Parameters (integer)
(real)
(real)
Support
Probability mass function (pmf)
Cumulative distribution function (cdf)
Mean
Median
Mode
Variance
Skewness
Excess kurtosis
Entropy
Moment-generating function (mgf)
Characteristic function

In probability theory and statistics, the Zipf-Mandelbrot law is a discrete probability distribution. Also known as the Pareto-Zipf law, it is a power-law distribution on ranked data, named after the Harvard linguistics professor George Kingsley Zipf (1902-1950) who suggested a simpler distribution called Zipf's law, and the mathematician Benoît Mandelbrot (born November 20, 1924), who subsequently generalized it.

The probability mass function is given by:

where is given by:

which may be thought of as a generalization of a harmonic number. In the limit as approaches infinity, this becomes the Hurwitz zeta function . For finite and the Zipf-Mandelbrot law becomes Zipf's law. For infinite and it becomes a Zeta distribution.

Applications

The distribution of words ranked by their frequency in a random corpus of writing is generally a power-law distribution, known as Zipf's law.

If one plots the frequency rank of words contained in a large corpus of text data versus the number of occurrences or actual frequencies, one obtains a power-law distribution, with exponent close to one (but see Gelbukh and Sidorov 2001).

References and links

  • B. Mandelbrot (1965). "Information Theory and Psycholinguistics". In B.B. Wolman and E. Nagel. Scientific psychology. Basic Books. Reprinted as
  • Z. K. Silagadze: Citations and the Zipf-Mandelbrot's law
  • NIST: Zipf's law
  • W. Li's References on Zipf's law
  • Gelbukh and Sidorov 2001: Zipf and Heaps Laws’ Coefficients Depend on Language

bn:জিফ-মান্ডেলব্রট বিধি



Linked-in.jpg