The Ubiquitous Power Law

Published in IEEE Spectrum Magazine, May 2008

I’ve long been fascinated with the omnipresence of power law statistics in so many natural and social phenomena.  A good example is Zipf’s law for the frequency of usage of English words.  The most common word is “the”, which is used twice as frequently as the second most popular word (“of”) and three times as much as the third (“and”).   Similarly, the nth most popular word has a relative frequency of use of 1/n.

Thus the curve of popularity versus rank shows a steep decline at first, followed by a long tail that looks rather flat when plotted on a linear scale.  (On a log-log plot, of course, this becomes a straight line.)  A word like “omnipresence” is way out on the tail at popularity position 74,228, right before the word “Borodin” (the Russian composer) according to wordcount.org.  We might expect to see about 74,000 occurrences of “the” before we encountered “omnipresence” -- notwithstanding this essay, of course.

All of the most common words are short, resulting in a very efficient transmission of information.  I imagine our distant ancestors sitting around the fire, drawing information theory equations with sticks in the mud to come up with an optimally parsimonious language, after which they would decide that they shouldn’t have used the word “parsimonious” (popularity number 49,309), when something like “compact” would have sufficed.

The result is rather a perfect blend – a hundred or so popular words used in everyday conversation and writing, together with about a hundred thousand more esoteric words that get sprinkled in for effect or special purpose.

Many other phenomena exhibit power law (i.e., polynomial) statistics – the population of cities, wealth of individuals, strength of earthquakes, hits on web sites, sales of individual books on-line, and so forth.   I would even imagine that it applies to something like the knowledge distribution in a field like electrical engineering.  All of us know Ohm’s law, for example, but perhaps only a tenth of us are familiar with the basic concepts in communications.  Then maybe only one engineer in a thousand is familiar with a particular protocol, and only one in a hundred thousand might be conversant with a particular theoretical paper in an IEEE Transactions.  But this is what makes the world go round; we have a lot of things in common, but there is a long tail of specialties that makes each individual unique.

Although power law statistics have been long known, the subject has gotten much recent attention under the name “the long tail.”   Discussions have been prompted by the difference between sales in the physical world where inventories are limited to the popular items, and those in the virtual world of the Internet where there is no inventory constraint that eliminates all the rare items on the long tail.  In the virtual world the many combined small sales out on the long tail of rare items approximately equal the sales of the few most popular items.  There is a perfect balance between the few popular and the many rare.

In most cases there are fundamental reasons why statistics behave like a power law.  For example, even though it might seem like individual choices should be uniformly distributed among alternatives, an individual’s choice is often influenced by the choices of others.  This gives a herd-like behavior with a flocking around popular choices and a long tail of individual dissent.     

Moreover, how could it be otherwise?  Suppose for a moment that power law statistics weren’t the norm, and that choices were uniformly distributed.  What would the world be like?  With all hundred thousand or so words equally likely, books would be long and turgid, but there would be little interest in them because there would be so few subjects of common concern.  And of course it would be almost impossible to learn a foreign language.

If the World Wide Web existed at all, it would be a strange place with millions of small, niche amateur sites, each getting its small fair share of the overall traffic.  People would have so little in common that there wouldn’t be much to talk about among the masses.  Instead small groups would congregate online around each of millions of special interests.

Population would be uniformly scattered about the earth.  There would be no cities, and whole countries would be like New Jersey, where I have to describe where I live by the exit number on the Parkway.  For better or for worse, wealth would be uniformly distributed, and perhaps neither cathedrals nor slums would be so prevalent.

I’m sure that readers can provide their own suppositions, but perhaps we could all agree that it wouldn’t be a world we’d want to inhabit.  Those ancient ancestors around the fire figured this out a long time ago.

Robert Lucky