Why does Ziff’s law work?

שו”ת › Category: philosophy › Why does Ziff’s law work?

asked 2 years ago

Hello Rabbi,
There is an empirical statistical law called Zipf’s law that says that if you take some text and arrange the words in it in order of frequency from highest to lowest, then the graph will behave like 1 divided by n, where n is the position of the word in the frequency order. For example, the first word will have twice the frequency of the second word after it, and 3 times the frequency of the word after it, and so on. This is a puzzling fact in itself, discovered by the linguist Zipf, after whom the law is named. Not only that, but it turns out that this phenomenon also exists in contexts completely different from linguistic contexts, for example, city population sizes, income distributions, and earthquake intensities, all obey Zipf’s law. My question is a bit philosophical. I ask how it is possible that all these unrelated things all obey Zipf’s law? Why this law and not another, and why is there any regularity to their distribution at all?
Best regards,

Discover more from הרב מיכאל אברהם

Subscribe to get the latest posts sent to your email.

Leave a ReplyCancel reply

0 Answers

0 Vote Up Vote Down

מיכי Staff answered 2 years ago

Zipf’s law is a well-known law that states that different things are distributed according to some law of power, where the powers can be different. I have not researched the matter, and I will write what seems to me to be correct. Ostensibly, this means that the collection of these phenomena is distributed according to a power law because of their nature (phenomena of some nature are distributed according to appropriate laws). After all, there are other phenomena that are not distributed according to power laws but according to another law. This law deals with those phenomena that are distributed according to a power law, and if we collect them in this way, there is no wonder in it. But when I first heard about this law, I suspected that there was nothing special here, except that when you arrange a set of phenomena in ascending or descending order, there are many cases in which it is possible to approximate distributions by some power. Even then, the deviations always exist, so I am a little doubtful to what extent this law really exists and the deviations are statistical or that there is actually no power law but it is possible to approximate many distributions by power. In any case, it only exists for phenomena in which this is the distribution, but there are other distributions as well.

Discover more from הרב מיכאל אברהם

Subscribe to get the latest posts sent to your email.

אורן replied 2 years ago

But it's not just some arbitrary power law like one over x^2.6, it's 1 over x, and not only that, it's always the same power of one over x. It doesn't change to sometimes be one over x^2, for example. For example, there is no Zipf2 law of phenomena that are distributed by one over x^2. There is only one Zipf law, with a certain, round power.

מיכי Staff replied 2 years ago

As far as I remember, I've seen that there are several different exponent laws. But even if the exponent is different, it's possible that within a certain range it can be approximated by one divided by x.

אורן replied 2 years ago

I checked again and it seems you are right, there is an extension of Ziff's law called the Ziff-Mandelbrot law that talks about more general powers.