Probability and Statistics – Lecture 23

This is an English translation (via GPT-5.4). Read the original Hebrew version.

This transcript was produced automatically using artificial intelligence. There may be inaccuracies in the transcribed content and in speaker identification.

🔗 Link to the original lecture

🔗 Link to the transcript on Sofer.AI

[29:54] The problem with small samples and coherence
[31:17] The risk of errors in district data
[33:07] Kahneman’s law of large numbers
[34:17] The Gates Foundation and small schools – a study
[36:20] Small schools at the Gaussian extremes
[42:21] The difficulty of standing up in military moral situations
[43:41] “Occupation corrupts” – applying the law of small numbers
[48:14] Accusations against Breaking the Silence and a statistical error
[54:48] Alternative medicine and the connection to Breaking the Silence

Full Transcript

[Rabbi Michael Abraham] Okay, we’re dealing with doubt and statistics, and last time we talked a bit about diagnosing miracles. And I said that very often, or in my opinion most of the time, that diagnosis is based on statistical mistakes. And one of the reasons—and I was really talking about the involvement of the Holy One, blessed be He, in the world—we spoke about two kinds of lotteries and the assumptions underlying the distinction between the two kinds, and about the fact that the Holy One, blessed be He, is not really involved in nature on an ongoing basis. There are certain parameters or certain mechanisms through which we might be able to cause Him to be involved, and that’s what is called a lot, as distinct from a lottery. And then I got a bit into the question of divine involvement in the world, and I said that many people—usually this is the accepted view in the religious world—that the Holy One, blessed be He, is involved in the world to one degree or another, starting from Nachmanides’ view at the end of Parashat Bo, where he says that everything that happens to us is nothing but miracles, meaning that the Holy One, blessed be He, basically does everything that happens here. And then there are somewhat more moderate views saying that He is involved in terms of reward and punishment: He helps those who deserve it, doesn’t help those who don’t deserve it, gives reward, gives punishment, but not necessarily that everything that happens in the world comes directly from Him. And in any case, whether according to this view or that one, it seems to me that when we talk about lotteries it’s very hard to assume—in other words, according to both of these views you could conclude that even when we talk about lotteries, the result is really the work of the Holy One, blessed be He. And I said that I think the distinction between lots and lotteries indicates that this is not true. Lotteries are the way of the world; lots are something else. And lotteries operate according to the laws of nature, and the purpose of a lottery is not to reveal—usually it is not to reveal—some true answer, where the lottery is really the means that allows the Holy One, blessed be He, to reveal to us the true answer. Rather, the purpose is to divide the chances in a situation where we can’t divide the thing itself—from that ship there in Liverpool, and continuing with the field of partners, a small field belonging to partners that can’t be divided, and continuing with all sorts of examples we discussed. And in that case, all I need is for the lottery to be fair, so that naturally each of the two participants gets an equal chance. Why do people come to the conclusion that everything is in the hands of the Holy One, blessed be He? First of all, there are sources for this in the Torah and so on; it’s not that people just invent it. But somehow, following those conceptions, people really do manage to discern the involvement of the Holy One, blessed be He—the handiwork of the Holy One, blessed be He—in every event that happens to them in the world. So when, I don’t know, someone is killed in a terror attack, injured in a terror attack, people immediately say: the Holy One, blessed be He, knows His accounting of the world; He has His own calculations that we can’t understand. Some people try to explain why it happened; doesn’t matter. But both these and those assume that what happened happened by the hand of the Holy One, blessed be He. Whereas if we’re talking about natural processes without divine involvement, then there’s no need to come up with explanations, and there’s also no need to say that the Holy One, blessed be He, makes calculations we don’t understand. He simply—it’s not His calculation; it’s something that happened because that is ultimately what nature produced. In this context, I didn’t get too deeply into this issue of divine involvement because that really isn’t our topic, but in this context I’ll maybe just add something I didn’t mention: Rabbeinu Chananel in Chagigah. The Talmud says there: “There are those swept away without justice.” “Swept away without justice”—so the Talmud says, and Rabbeinu Chananel says, for example a person who killed his fellow. That’s called “swept away without justice.” What does he mean? “Swept away without justice” means someone who dies even though he didn’t deserve to die—not that the Holy One, blessed be He, decided he would die, but that he died by chance. So what is “swept away without justice”? Rabbeinu Chananel says: a person who killed his fellow. What does he mean? He probably means, if I understand correctly, that if there is a natural death here, then natural death is in the hands of the Holy One, blessed be He, because He governs nature. But if it is a human decision to kill another person, that is free choice. And if I chose to kill someone, I can succeed in killing him even if according to the calculations of the Holy One, blessed be He, he did not deserve to die. And that is called “swept away without justice.” In other words, anything that is the product of human choice is not in the hands of Heaven. That is basically what Rabbeinu Chananel says, or what the Talmud says according to Rabbeinu Chananel. I would add to that that in my view, even things that happen not by human hands but by nature are also usually not in the hands of Heaven. Again, there may be occasional cases here and there; you can’t know. But usually that is not the case. Usually these are natural processes, and the example of the lottery basically makes that clear. Now what leads people to these conclusions—how they see the hand of God in various places—is what I mentioned last time: the law of small numbers. Yes, that’s a term from Kahneman. Kahneman says that the law of large numbers says that if things happen randomly—without outside involvement—then they will be distributed according to the a priori distribution. Now take a fair die: the chance for each face is one-sixth. If we roll it, I don’t know, ten times, twenty times, it won’t really come out one-sixth, one-sixth, one-sixth. But if we roll it ten billion times, then it will come out one-sixth, one-sixth, one-sixth. In other words, with large numbers the whole business converges toward the a priori probability; the distribution of the results converges toward the a priori distribution. That’s the law of large numbers. And the law of small numbers says that what is true for large numbers is also true for small numbers. That is obviously nonsense, but that is how people very often relate to events that happen to them. They draw conclusions from them as if they had made some choice or conducted some examination on a broad sample, a controlled systematic examination, and so on, even though in fact we are talking about two examples or a few isolated examples they happened to encounter. Very often this involves selective choice of results, or failure to notice that in fact you are looking at very, very specific results and not some representative sample. And from that you infer all kinds of conclusions. I gave various examples of this: the miracles in Gush Katif and a Book of Psalms in your pocket saving a soldier from being hit, or things of that kind—which are really all expressions of the law of small numbers. I want to go into this in a bit more detail.

[Speaker B] Baruch Pag’ner, what? Yes, I’ll mute here.

[Rabbi Michael Abraham] I got into this topic because I was discussing the involvement of the Holy One, blessed be He, but this is actually part of the planned structure of the series. We’re talking about doubt and statistics, so I want to talk about statistical fallacies. So the involvement of the Holy One, blessed be He, is really just the link I’m making in order to get to this chapter in the series—the chapter on fallacies in statistical inference. I dealt with this in several columns on the website. The main column devoted to this topic is Column 38, and it’s called: What Do Breaking the Silence and Alternative Medicine Have in Common? Yes, there’s something I identify as common to both these phenomena, and I think that behind it really lies the law of small numbers. So I want to maybe follow what I wrote there in that column. The phenomenon of Breaking the Silence has always fascinated me. At first, yes, it’s very infuriating, but afterward, when I hear more from these people—if I meet them up close—I’m even more struck by this. We’re not talking about evil people scheming to harm the IDF or the State of Israel. We’re talking about people who sincerely are convinced that we are evil. They sincerely are convinced that the IDF behaves terribly, that occupation corrupts, that people are cruel, and so on and so on and so on. And some of them are very good people, very good Zionists, everything is fine, and they sincerely believe this. Now on the one hand that’s one side of the picture. On the other hand, my impression—and again, I’m just one person too, and I haven’t done statistics either—but my impression is that the situation is not like that. Both from my own service in the army and reserves, and from what I hear around me, I really get the impression that it’s not true. Not that there are no deviations—of course there are deviations—but overall the thing does function with some kind of moral standards. I don’t know whether we are the most moral army in the world, but it seems to me that we definitely do generally behave properly. And that raises this dissonance for me, or this question: so how does this happen? If I’m right—if my impression is not blatantly mistaken—then what they say requires explanation. In other words, unless they are evil and it’s all plots of the European Union and the New Israel Fund. But I don’t buy those statements. Not that the European Union doesn’t fund them or the New Israel Fund doesn’t fund them—they do fund them, that’s true—but they don’t seem to me like evil, malicious agents who have come to harm and slander. Rather, they sincerely are convinced that this is the situation. And the funding they obtain is funding from the sources that of course want to promote this agenda. But these are people—people who, a large portion of them, I don’t know, I haven’t checked, but those I met, those I heard, sound like people who sincerely are convinced that this is the situation. That they are sincerely warning about real problems and are convinced this is the case. So how does that fit with my impression, which is completely different? That’s really the question that bothered me. So again, I’m not looking at these complaints and—I’m not trying right now to determine whether we are moral or not. I’m using this example to raise a question that is really a question in statistics. So this is not—without getting now into arguments about Breaking the Silence and to what extent they are right, and whether they are right, and so on. I think they are not right on many levels, both on the factual-statistical level, and beyond that also in their lack of, or disregard for, the background to the matter. Meaning, it’s as if we’re living in Switzerland and how can it be that we don’t give equal treatment to all residents. Even if we do not give equal treatment to all residents, we are not in Switzerland. Since we are not in Switzerland, one can certainly understand—even if not accept—but understand where this unequal treatment of different populations comes from. There has been a war here for more than a hundred years, and you can’t ignore that. That kind of ignoring seems completely bizarre to me. So I’m not intending here to argue in favor of Breaking the Silence or to say they are right. On the contrary, I assume they are not right, I assume they are biased. But I’m asking why. Why are good people biased in this way, assuming I’m right and they are biased? Assuming my own impression is not the biased one, but that my impression is what I see, and I really think it’s correct. And if that is so, then I myself have to account to myself for what happened to them. And here it seems to me that I enter into this issue of the law of small numbers. Daniel Kahneman, yes, was an Israeli psychologist from the Center for Rationality at the Hebrew University. He was there for quite a few years, together with Amos Tversky, with whom he did most of the work. At some point he moved to the United States, lives there to this day, and works there—Daniel Kahneman. He won the Nobel Prize already as an American, but we, you know, count him as one of the Israelis who won the Nobel Prize. There’s some justice to that too, but that’s not the issue right now. In any case, he is a well-known man, a Nobel laureate, an intelligent person. And his main livelihood—and by the way, that of the Center for Rationality generally—is from studying failures in thinking in general, and specifically failures in statistical thinking. And that is what he made his living from, and that is what he got the Nobel Prize for. Some of his findings are described in a more popular book called Thinking, Fast and Slow. There he describes various phenomena and failures in statistical thinking, and I think it is definitely a useful and important book for anyone who wants to improve his statistical thinking—thinking in general, and statistical thinking in particular. Among other things he refers to a fallacy he calls WYSIATI—what you see is all there is. In Hebrew I abbreviated it differently, but the idea is the same. The meaning is: I saw something, from that I built the whole picture, and I ignore the fact that this is only what I saw, and there may be many other things I did not see, and perhaps they are different. And what I saw—that is all there is. In other words, I relate to that and infer my conclusions from it, even though that information is of course partial and selective and perhaps also tendentious, and so on. But this is what I encountered, and since this is what I encountered, I draw very clear conclusions from it. And again I say: I’m like that too. The conclusion I mentioned before—the conclusion I mentioned before, that I think Breaking the Silence are mistaken—that too is not based on facts and statistical reasoning or some comprehensive statistical survey I conducted, but on an impression. So it is entirely possible that I too am mistaken here. But still, one of the two sides here is clearly not right. And the question is how we explain the side that is not right—which I naturally tend to think is them. Kahneman argues that people’s confidence in their conclusions is usually not built on the quality or quantity of the sample and the evidence, but on the quality of the story built around the examples they encountered. There is a fascinating property of a small number of examples. That property basically says that when I have a small sample of examples—a small number of examples—it succeeds in producing a more coherent picture than pictures that are obtained on the basis of large samples. In a large sample, many, many things happen. In a small sample, it may be that only things of a certain kind happen. Then I can easily infer from that that this is the natural phenomenon, this is what always happens. Therefore, the smaller the number of examples we encountered, the more our conclusions will be, on the one hand, more extreme, and on the other hand more credible and more coherent. Now you have to pay close attention: these are two phenomena that deal a mortal blow to our ability to draw conclusions. Consistency somehow matters to us more than reliability, and with small samples it is easy to arrive at consistent findings. So let me give a few examples.

[Speaker C] First example—and there’s also confirmation bias, where people are constantly looking for the examples that will confirm what they already believe.

[Rabbi Michael Abraham] That’s part of the issue, something additional, but I’ll get to that later too—confirmation bias, as it’s called. In chapter ten of his book, Kahneman brings a study that examined the kidney-cancer mortality rate in different counties in the United States. There were 3,141 counties. Counties meaning areas—not committing right now to the exact size. They examined 3,141 areas or places and checked the kidney-cancer mortality rate in all those places, trying to see whether we could find parameters that have an effect. It turned out that the counties with the lowest kidney-cancer mortality—the healthiest counties—were rural counties with sparse populations, located in the Midwest, the South, and the West of the United States, and they had a tradition of support for the Republican Party. That’s what they found; those are the facts. Now the question is what we can learn from this. So let’s try a preliminary filtering of the parameters, because we’re talking about rural counties with sparse populations; the location is in the Midwest; they support the Republican Party; and they are very traditional places. Okay, those are all the characteristics. Which of these characteristics is responsible for the low mortality? It seems to me that support for the Republican Party is not the first parameter I would choose, even if I were an ardent Trump supporter. I don’t think I would attribute people’s health or their avoidance of kidney cancer to that. So what would I say? The obvious conclusion was—and that’s what was done there, he describes it—that the root of the matter is rural life. You’re not in the city: there’s open space, good air, you eat healthy and fresh food, you drink unpolluted water, you live calmly rather than in urban neuroticism. So somehow you get a healthier body, a better immune system, I don’t know exactly what, you get sick less often, and therefore there is less kidney cancer. That was the claim. Fine, sounds logical and reasonable. At least to me. Next step: if we want to do serious work and not just stay with guesses, let’s check the counties where kidney-cancer mortality is highest. Okay? What turns out? It turns out that these too are rural counties with sparse populations, located in the Midwest, South, and West of the United States, and of course with Republican leanings, support for Trump, and so on. Yes, exactly those same counties. How does this miracle happen? Apparently rural life is probably not a guaranteed immunity against kidney cancer, nor healthy and fresh food either—at least that’s not what emerges from this study. So what then? Well, you can draw a conclusion. There is poverty there, lack of education, lack of access to medical centers, heavy alcohol consumption. So apparently these things cause kidney-cancer mortality. Again, we also have a very convincing explanation for the findings of the second study. The problem is that these two studies do not fit with one another. Since those same sparsely populated counties in the American Midwest that are traditional and support the Republican Party also eat healthy food and have fresh air and calm and so forth—and supposedly that causes greater health—but they also have poverty and lack of access to medical centers and so on, and that causes greater mortality. So I don’t understand: do the parameters characterizing these counties lead to lower mortality or higher mortality? What is true here? And we might expect some answer like: well, the truth is somewhere in the middle. The truth is somewhere in the middle. No, it is not in the middle. These are two contradictory conclusions, and the truth is somewhere else entirely—not in the middle. What probably happens is that among the various counties—and after all there are 3,141 counties—some of them are densely populated counties, big cities, with millions of inhabitants, crowded populations and so on. And there are counties with a hundred residents, even a thousand residents, I don’t know exactly, spread over the same area. Because this is rural land, there are large farms and the like. Now most of the 3,141 counties were small counties. That’s how it was distributed there, and naturally that’s how it is—there aren’t that many counties with millions of residents. So if you want that many counties and divide the land more or less evenly, most counties will be sparsely populated. Now in sparsely populated counties we are really dealing with groups—small-group samples. Let’s say in such a county there are a hundred residents, okay? And opposite it there is a county with a million residents. Now among a million residents, the kidney-cancer mortality rate will be more or less similar—this is the law of large numbers—it will be more or less like its proportion in the general population, because this is a very large sample. Unless, again, city life itself has some effect here, but if I ignore that for the moment, it is clear that with a million residents the distribution will be close to the a priori probability. By contrast—and I can say more or less how many will die there. If I have worldwide data, then I can tell you roughly how many will die in New York, Chicago, the big cities. But in small places with a hundred residents, or even a thousand residents, which is still a small place, the mortality rate will vary greatly from county to county. There will be one county where, say, if the average mortality rate is—I don’t know—two percent, I’m just throwing out a number—then in one county it will be three percent, in another one percent, in another maybe even five percent. Overall, in each thousand people we take, mortality rates will differ. In very, very large numbers it will already concentrate and it will always be two percent, almost always two percent plus or minus a little. But in small samples, mortality rates can be very high, or deviate sharply upward from the average, or deviate sharply downward from the average. Therefore, if I draw all the counties on some Gaussian curve distributed around the average—say the average is two percent—then the Gaussian runs from, I don’t know, half a percent to four percent or three and a half percent when the average is two. Okay? So the average is two. Who sits in the middle of the Gaussian? The big cities. There is a large number of people there. For a large number of people, the distribution will be more or less the same. There’s no difference—when you roll a die a billion times and then again roll a die a billion times, there will be no difference between the results. The difference will be minuscule. The results will be the same every time. But if you roll a die ten times, then one time you’ll get one distribution and another time you’ll get a completely, completely different distribution. Not only unrelated to one-sixth, one-sixth, one-sixth of course, but beyond that it can come out totally different. So let’s think about rolling a die ten times. Okay? Or twelve times, because we have six faces—twelve times. Then of course it never comes out two, two, two, two, right? Two ones, two twos, two threes—it never happens. But what does happen? One time you get five ones and then the rest divided differently. Another time you get ten threes and the remaining two divided some other way. Another time you get four of this and four of that and four among all the rest. There will be very great variance when the number of rolls is small; the variance is very great. And among other things, I definitely have a chance of finding cases where all ten or twelve rolls came up one. If I do twelve-roll sequences many times, you understand there will be such dozens, yes? Such sets where they are distributed exactly two-two-two, or all twelve fell on one number. That can happen too. So if I look for extreme results, I will always find them in the small samples. In other words, if I ask where I will find a situation where all the results came up one—in small samples of course, right? In large samples it won’t happen. In small samples it will happen. Of course not in all small samples—on the contrary, in a very small number of small samples—but there it will happen. And of course the opposite will happen too: the exact uniform distribution, two-two-two-two, that too will occur in small samples. Okay, and therefore in small samples I will always find the extreme results. Or in other words, if the patients are somehow distributed across the United States, then there will be counties of a hundred people in which all the people have kidney cancer. There will be many other counties in which there won’t be a single kidney-cancer patient. So there will be counties with one hundred percent sick—I’m exaggerating—ten percent with kidney cancer, and counties with zero kidney-cancer cases. Therefore, if I look across the counties, the extreme counties will always be the small counties—small in terms of the number of people. Sparsely populated counties in the Midwest. And therefore both the places where kidney-cancer mortality is greatest and the places where kidney-cancer mortality is lowest are found in the South, in the American Midwest, among Trump supporters. It just so happens that they are Trump supporters too; that probably has nothing to do with it. Okay? So understand that when we do a survey like this and ask where the mortality is lowest, you can then read in the newspaper that rural life—then they also interview someone who reached the age of 110. They ask him: how did you do it? Because I ate two chewing gums every day. So now we have a newspaper item: eating two chewing gums every day guarantees long life. That is of course complete nonsense. After all, it is obvious that every person who reached the age of 110 did something. It does not mean that because of that thing he reached the age of 110. He reached the age of 110, and besides that he also did something. If we latch onto the thing he did and see that as the reason for his longevity, we won’t get far. How should this really be checked? You have to check all those who reached 110 and see whether they all chewed gum twice a day. Or check all those who chew gum twice a day and see whether their life expectancy is indeed greater than that of those who do not chew gum twice a day. That could get us closer to valid findings. But to take some isolated case or small sample, see what happens there, and infer conclusions from it—that is a guarantee of error. So people did a study of kidney-cancer mortality, and different studies came to conclusions that were exactly opposite. One study concluded that rural life guarantees immunity from kidney cancer, and another concluded that rural life guarantees getting kidney cancer. And each of these studies had a story in the background. That’s the nice part of the whole matter. It’s easy—once our results are striking results, we see very clearly, we have a small sample. A small sample: there is a chance it will be very coherent. Meaning, something very pronounced will happen in it. Because in a large sample many things always happen. In a small sample, you may have caught the three cases where exactly the same thing happens, and therefore it’s terribly coherent. That strongly tempts us to develop a story around it. The story of rural life, peace and quiet, healthy and fresh food—all of that fits perfectly with good health. The story of ignorance and poverty and lack of access to medical centers explains high mortality wonderfully. But both these stories are fairy tales. They are stories we develop on the basis of a small sample that happened, by chance, to come out coherent. In a small sample there’s a decent chance it will come out coherent. If we make many small samples, then of course we will also find among them small samples that are coherent like this.

[Speaker C] That also shows that we learned nothing. We came with the story already from the start.

[Rabbi Michael Abraham] No, no, not from the start. Again, no.

[Speaker C] Because we dressed up this side and dressed up that side with thoughts we already came with in advance.

[Rabbi Michael Abraham] No, because the insights—the findings—we found them in the study. But of course we have certain a priori intuitions, and they create a story for us. And by the way, they may well contain truth. All in all, healthy and fresh food seems to me to be beneficial to health. I think there are updated findings that support that. But those intuitions help us build a story around the small sample in a way that convinces us that the law of large numbers also applies to small numbers. Because it sounds so logical; it fits with what we think. And therefore when we see a phenomenon that is externally consistent—externally meaning coherent, everything happens in the same direction, in the same way, everyone there has kidney cancer—well then obviously there must be something in that place underlying it all. It can’t be a coincidence that all hundred residents here have kidney cancer. The answer is that yes, it can be. If you have 3,141 counties, then one of them will have dozens of percent with kidney cancer. It just happened statistically, with no connection at all. Don’t go looking there for the source of the issue; it doesn’t have to be the source of these things.

[Speaker D] But I didn’t really understand. If we come to check, say, rural areas with suitable nutrition, then if we take a sufficient number of such villages—twenty thousand containing twenty million residents—then we could judge, no?

[Rabbi Michael Abraham] That’s a good way to do the study, but what they

[Speaker D] did was something else.

[Rabbi Michael Abraham] I want to check which counties are the counties with the lowest mortality. That’s the question they asked. They went through 3,141 counties and discovered that the bottom three were all areas in the Midwest, and so on and so on. That’s what they found. Now ask yourself why. Apparently rural life is very healthy. The same thing in the reverse study: which counties have the highest mortality rate? Which are the three or five with the highest mortality rate? And once again they found that it was in the Midwest. Then it gets fitted onto some story, and immediately you build some theory out of it, and everything works out wonderfully. And this is even before confirmation bias and so on and so on. We’re still just talking about a statistical fallacy. A statistical fallacy that basically says: once I have coherent phenomena, I look for a story to explain them. I found a story that explains them, so as far as I’m concerned the law of large numbers applies here. So this is a representative sample, I have a theory, I’m very decisive about it, everything fits neatly. In other words, this is the point—yes, Kahneman phrases it like this. He says there is a sentence—listen to this sentence: large samples are more accurate than small samples, more reliable than small samples. That sounds reasonable to all of us, right? That is basically the law of large numbers. Now look at the next sentence: small samples yield results that are more extreme and more coherent, and often less correct, than large samples. The second sentence is equivalent to the first sentence. The first sentence seems obvious to us. Very few people think about the second sentence, even though it is completely equivalent to the first. And the second sentence is what really underlies these mistakes of small samples. Look at another similar story, for example. Here too I’m giving you a small sample. I took one study, two studies on kidney-cancer mortality, and now I’m taking another story. From these stories I’m now building a story, and now I’m selling you a story too. Altogether I showed you three examples. Here too you can suspect that maybe I’m spinning a story. Statistical inference is a very tricky business. But let me tell you the second story before you attack me. There is the Gates Foundation, the philanthropic foundation of Bill Gates and his wife, aimed at advancing education. The foundation invested 1.7 billion dollars in tracking the characteristics of successful schools. They wanted to find out what the recipe is for successful schools. Another example Kahneman brings, okay? So they went through all the schools and checked which schools had the best results, and they discovered that these were usually small schools. Small schools—immediately, what’s the story? In small schools there’s personal attention, they know each student, they can relate to each one, they follow the students, it’s not a mass-production factory. Right? Small classes—remember the story about small classes? So obviously small schools are a good recipe, or a good mechanism, for improving student achievement. The Gates Foundation invested billions in shrinking schools, splitting schools up, making them smaller, creating small schools—until some wise guy came along and said to them: wait a second, let’s check what happens with small schools—sorry, who are the less successful schools? It turns out, once again, that they are small schools. The least successful schools are small schools. Well then: small schools, poor areas, they send students there, they don’t have money to pay, the teachers there are of low quality—we have a story again. We can find a story for this too. There are also fewer options in small schools, by the way. There aren’t different tracks, you can’t divide the students, there’s less ability to advance students in directions that suit them because the range is smaller. So we have a really good explanation for why the small schools are actually less successful. Except that once again we have the same tangle. So what’s going on? Are small schools a recipe for success or a recipe for failure? The answer is that they are a recipe for nothing. Small schools will always be found at the extremes of the Gaussian, both at the right-hand extreme and at the left-hand extreme. You will also find many small schools in the middle, with middling achievements, good achievements, bad achievements, middling achievements. Because small schools are—think about a school with three students. It’s extreme, yes? But suppose. Then you understand that in a school of three students—say many schools of three students—you understand that some of them will have all three with an average score of 100 on matriculation exams, and there will be schools whose students all have an average of 100 on matriculation exams. And there will be other schools where all the students failed, all three of them failed their matriculation exams, in every subject. Okay? Why? Because with three students, if by chance you have three weak students, then they all fail. If you have three very strong students, then they all succeed. But in a school with ten thousand students, it won’t happen that everyone fails, and it won’t happen that everyone succeeds. It will be distributed somehow the way success is generally distributed in the population. In other words, this is the same phenomenon as with kidney cancer in the schools. And notice: we’re talking here about people whose intelligence no one disputes. Bill Gates is not an idiot, and I assume those who run his foundation are not idiots either. And they invested billions on the basis of the findings of this study, until it became clear that it was all nonsense. By the way, Kahneman himself argues that a more solid study—he doesn’t say exactly where he gets this from, but that’s what he claims—supports the claim that larger schools are better. Larger schools are better because they leave you more options. There are more teachers, more tracks, more possibilities, so students can find their place in a path more suited to them, and therefore they have a better chance of succeeding in larger schools. They also have a better chance of finding friends they connect with, of feeling good at school. In a small class you may suddenly find that you don’t fit, that you have no friends because nobody wants to be your friend. In a large school you’ll always find some group more suited to you; they’ll be your friends, you’ll feel better with them, you’ll feel better there, and so on. Doesn’t matter—maybe he’s right, maybe not. I’m not getting into that issue at all. Because the question I’m dealing with is not how to succeed on matriculation exams, nor how to avoid kidney cancer, but to use these examples to show the phenomenon of the law of small numbers. When I look at a few phenomena that I encounter, and I have a good story behind them, I am convinced that this is the absolute truth—and that, in a nutshell, is the law of small numbers. Okay, yes, a small school, then students—how can they pay, there’s a story for every result. What exactly—what exactly does this mean? I want—can’t hear? Did someone say something? No.

[Speaker E] To ask—so how can you study this? How can you really study those small schools? Meaning, if you do want to reach conclusions that are actually correct?

[Rabbi Michael Abraham] You want to know whether a small school is helpful or not helpful? You need to examine lots of small schools and see how success is distributed within the small schools, not take all the schools and pick the three least successful ones and discover that they happened to be small schools. Of course they’ll be small. And there are proper ways to do this; you don’t need to be a brilliant statistician for that, it’s just common sense. You only need to be aware that you have to work with common sense, that’s all. If you want to know whether a small school succeeds more than a large school, take many small schools, take many large schools, compute an average for the schools. For the small schools, with a standard deviation, same for the large schools, and compare the results. But to take all the schools and ask who are the five least successful or who are the five most successful and draw conclusions from that—that’s a statistical mistake. So I’m not saying this to discourage us from statistical thinking and say there’s no chance of drawing correct conclusions. Quite the opposite. I want to say that if we become aware of the importance of orderly statistical thinking, we’ll be able to draw better conclusions and not fail in all sorts of failures that people fall into in studies of this kind. By the way, not only ordinary people fail in these things—researchers fail in them too. It’s not just laymen. Serious people in their fields, intelligent people, can also fail in these things, sometimes in a really embarrassing way. You look at it and say: this is a total blackout. But the important point in the story is what Kahneman adds, because he’s a psychologist, not only a statistician. Statisticians can say: the law of small numbers—a small sample can give you any result. But he’s a psychologist. And as a psychologist he tells us: friends, pay attention, the story matters here. I have a small number of schools—I have a number of small schools that are very successful. It immediately gets dressed up in a story: a small school invests a lot in each student, rich parents can pay for a small school, and therefore they succeed. I immediately find a fitting story. And once I have a fitting story, that’s a winning card. And of course the same goes for small schools that fail. Same thing with kidney cancer. The story here is very, very important. Now together with the story—and we’ll get to this in a moment—beyond the story there’s also confirmation bias and all those things; we’ll get to that later. But first of all I want you to notice the phenomenon itself. This phenomenon of the law of small numbers is very dangerous. I think, or suspect, that this is the phenomenon underlying this whole matter of Breaking the Silence. What happens there—some of them, say, are soldiers who themselves experienced some problematic incident, where people behaved badly toward the local population, and unjustifiably. By the way, sometimes they behave badly but justifiably, yet someone with any sensitivity still recoils from that behavior. It’s hard to stand in situations like that. I was in situations like that, and it really is hard to stand in that kind of situation even when it’s justified. It’s hard. You see a family, you enter a family’s home in the middle of the night, you see children crying there, and you take the father. It’s not easy to stand in such a situation. Understand, the Shin Bet person told you: this father is a terrorist. More than once, and not just twice, it happened to me myself in reserve duty—I accompanied Shin Bet people on arrests. Yes, they needed military security, so we accompanied them. It’s very hard to see that. I can easily see myself afterward going out with Breaking the Silence on some worldwide campaign saying that we are basically Nazis who snatch people from their homes and families in front of their crying children in the middle of the night. Of course we do it in the middle of the night; during the day I won’t catch him, he’ll be hiding. You understand? Even when the phenomenon is justified, there’s something there that is hard to digest for a person with reasonable moral sensitivity, and I think most soldiers are like that. And all the more so if it isn’t justified, and you do things that are unjustified, then it’s even harder to digest, and that happens too. And once you experience such a phenomenon—you saw three such cases—as far as you’re concerned, the occupation corrupts. The story that “the occupation corrupts” is the story that gets attached to the law of small numbers; that’s the interpretation I’m offering of the phenomenon. Or maybe I’m biased too, as I said earlier—but that’s the interpretation I’m offering. People who experienced, or heard firsthand, it doesn’t matter, about problematic cases or hard cases—even if they weren’t problematic, they were hard because it’s hard to accept this kind of treatment of people. And immediately it gets attached in their minds to this idea that the occupation corrupts. They have a good story: listen, we have power, you know, and power is not exactly a guarantee of gentle behavior. It sounds very logical to us. And so you immediately reach the conclusion that the whole army basically behaves this way, or a significant part of the army behaves this way, and your immediate conclusion is that our army is cruel and terrible. And that is exactly the same as the small schools and kidney cancer and all the rest—it’s really the same thing, or at least it could be the same thing. Again, maybe I’m the one who’s wrong here. Maybe the reason I think the army behaves properly is because of the small sample that I experienced. Maybe, I don’t know. But one of the two sides here is operating with the law of small numbers, and the story that attaches itself to that only strengthens our attachment to the conclusion. After all, when I ask myself—when I say “the occupation corrupts”—I have evidence. I saw that they shot here, and that they took people out in the middle of the night, and at checkpoints they behaved very badly and impatiently, and so on and so on. And that’s true—cases do happen, and they definitely do happen. On the other hand, the soldiers themselves are pushed around quite a bit, and there are terrorists threatening them, and they do shifts, and this whole story is, after all, because of the Arabs—none of us enjoys doing this—and so there is a very natural and very understandable anger toward the Arab population in general. I have such anger too, not only other people; I find it in myself as well. And that anger can lead to all kinds of results, and I understand that those results happen. Sometimes they’re justified, sometimes they’re unjustified. Yes, it can happen. Now when I say that the occupation corrupts, what does that mean, that the occupation corrupts? When I want to say that we as soldiers are more violent—what does “more violent” mean, compared to whom? Compared to the British army? Compared to peaceful civilians living in their homes in Tel Aviv, Petah Tikva, or Metula, or I don’t know, some kibbutz? Obviously yes—or not obviously, but probably yes. An army is a place where violence is used; that’s what it’s there for. The army’s job is to use violence so that the other civilians can live in peace. That doesn’t mean the occupation corrupts; it means that the one who has to carry out the violent actions is the soldier. When you were a soldier, you did it; when I’m a soldier, I do it. And then when you sit at home, you’ll be less violent than me; when you’re the soldier and I’m at home, you’ll be more violent than me. It’s a division of roles. Does that mean that soldiers are more violent than someone else? To say that, you need to compare, I don’t know, such soldiers to soldiers in another situation and see whether the violence increases, or whether unjustified violence increases. You have to see the context to understand what causes this violence. Are there justified factors? If so, then it’s not true that the occupation corrupts. The occupation puts us in situations that require us to behave more violently, but really more violence is needed there than without the occupation—that’s true. It’s not because the occupation corrupts. Meaning, there’s something here—now I’m not saying, I don’t know, maybe the occupation does corrupt, I don’t know. The Torah itself says, “And He will grant you compassion and have compassion on you,” yes? The ethicists say this about the idolatrous city. Why does the Torah say, “And He will grant you compassion and have compassion on you”? Because someone who does such a deed, it can plant cruelty in him. Yes, that can definitely happen; it’s a phenomenon I’m willing to hear about. Emotion, sensitivity, becomes dulled when you encounter such cases. The story is a story that can also be true, but that true story, in my opinion, gets attached to the law of small numbers and produces a phenomenon that looks to the person who experienced that small number of cases like a general phenomenon. And that’s the explanation—that’s the general situation—and here there is a failure. And I think that when we suddenly look at it from this angle, all these bizarre statistical phenomena that we might laugh about suddenly turn out to be something that exists inside our own home. We ourselves constantly fail in these things and constantly don’t notice it. And we accuse Breaking the Silence of being anti-Semites, of the New Israel Fund, and all these things—and they’re not—but it may be that they’re mistaken because of the law of small numbers, just as we too can be mistaken about the same thing in other contexts. Why? Because they like the story that the occupation corrupts, and that story gets attached to their small data set, and I like the story about small schools being wonderfully successful, and that gets attached to my story. Each person with his own stories and samples. You need some kind of combination: a small and coherent sample that you encountered—coherent because it’s small—together with a story that seems very compelling and very logical to you and gets attached to that small sample, and that creates in you some kind of broad picture, a general conception. And we all fall into this. In this matter I think almost no one comes out clean. You really have to be, first, very skilled, and second, very aware. To check yourself all the time with your conclusions, to see whether you’re falling into one of these failures. And that’s why it’s important to know these failures, even though they’re not very sophisticated. Any child can understand them; you don’t need advanced mathematics for this. But you do need to be aware of these failures and be alert to the fact that you yourself can fall into them. You have to keep checking yourself, because otherwise there’s no one who doesn’t fall—no one. And many times, when we’re on the other side and someone else falls, we immediately accuse him of being an anti-Semite, or this kind of wicked person, or that kind of wicked person. And of course he accuses us of being wicked, and anti-Semites, and so on, or chauvinists, and so on and so on. And all those accusations are basically rooted in mistakes—statistical mistakes, sometimes, not always, sometimes. And therefore I think it’s very, very important to know these statistical failures and get used to acting properly.

[Speaker D] Rabbi, rabbi, can I say something? Yes, yes. Maybe one could say in their favor—in quotation marks—of Breaking the Silence, that when you bring a human factor into it, then it’s already a little different from the story of cancer and schools. Because how do people talk about an incident of desecration of God’s name? Why is that so severe? After all, statistically it represents nothing—so why is it such a…?

[Rabbi Michael Abraham] No, but that’s the beginning of something that can lead to a great deal more, and if it’s caused by something, you need to deal with it.

[Speaker D] I mean to say—

[Rabbi Michael Abraham] There are many claims of Breaking the Silence that are justified claims. I have no doubt about that. None whatsoever. I have no doubt that they have justified claims—not only about events that happened, but also about the fact that the army didn’t handle them properly. Clearly. There are things that were also publicized, checked, and it turned out that it was indeed true. So I have no doubt, and therefore—the fact that this needs to be dealt with, and that people need to come out and warn about it—I don’t know whether abroad, maybe even abroad if they see that here we’re not doing it properly—anything is possible. I’m speaking only about one aspect. The question is whether this is a general phenomenon. If you were to tell me no, it’s not a general phenomenon, but it’s still important to deal with it, and it’s a desecration of God’s name, and it isn’t being addressed—fine, then that has to be discussed on its own merits. Every case has to be examined, and maybe you’re right. I’m talking about a very, very specific aspect: the aspect that presents this as a phenomenon or a general picture. That, in my opinion, is not correct.

[Speaker D] Fine. If, say, I were to argue that the occupation can bring out the worst in you—and by the way it can also bring out the best in you. There are incredibly good people there who help, and there are people who do the worst things as a result of the same factor.

[Rabbi Michael Abraham] So I’m saying: to that I can agree, but I know that a priori. I don’t need Breaking the Silence’s examples for that. That’s obvious. The argument is not over whether the occupation brings out the worst in you, but whether it is necessary. And that, for some reason, Breaking the Silence people tend to ignore, in my opinion. And if I have to occupy because I have no other choice, then yes, it brings out the worst in me, but what can I do? I have no better option. But I accept the claim that the occupation brings out bad things in me, because when you give power to a person it brings out—or can bring out—bad things in him. I agree completely. True. On the other hand, that doesn’t mean it happens to everyone. It doesn’t mean it’s a general phenomenon. And third, it doesn’t mean it can be prevented. Maybe we have no choice—what can we do? I don’t want to be an occupier, but I don’t have much choice. Let those who want to be occupiers go free, but there are very many people in the public who do not want to be occupiers. They think we have no choice. So how does it help me if you tell me that the occupation corrupts to the very marrow? Fine, it corrupts—what am I supposed to do? I can try to make sure it doesn’t corrupt. The army by and large tries to keep it from corrupting; it tries to set norms, standards, to check things. Again, this is a large system, and overall let’s not forget—it’s the IDF. We all know how the IDF functions. Fine. But within those limitations, that’s what there is. People try to do their best in the circumstances. The question is whether those circumstances are forced on us or not; one can argue about that. I think they are. It seems to me that for now at least we have no option without occupation. So therefore it won’t help me even if you convince me that occupation corrupts. But again, that’s really a different discussion. I don’t want to enter the political or moral debate; I’m speaking only about the statistical aspect. About the statistical aspect. The moment you present it as a picture—I spoke to people from Breaking the Silence. And they say: what do you mean? Things like this happen every day. People take a machine gun in Hebron and just fire into a population without provocation. Not that they weren’t aiming exactly at the provocateur and happened to hit someone—not at all. Just for sport. Now I don’t know. Maybe such a thing happened once; I almost can’t bring myself to imagine that such a thing could happen. I don’t know. When I served in reserves, that could never have happened. Never. I don’t know—maybe there are especially wild units, or a squad with a problematic sergeant, or a problematic platoon commander, I don’t know. It’s possible. But to say that this is the regular phenomenon, that this is what always happens—I heard with my own ears that they say this. This is what happens; this is the standard. No, not just some case that he saw. That can’t be, I don’t know. It can’t be. I don’t accept it. From my experience, from what I know, it simply cannot be. I am not willing to accept that as a matter of fact. I do understand that maybe there was a case that he saw. And even about that, I don’t know how he interpreted it or what exactly happened there—but perhaps. That case should be handled. But the question is whether I can infer from here a general picture. The moment you put it together with the story that the occupation corrupts, then it’s lost. The coherence of the picture plus the story—there’s no chance. That’s the alternative medicine I mentioned in that context, because the title was: what do Breaking the Silence and alternative medicine have to do with each other? Alternative medicine is almost entirely built on exactly these nonsense ideas. Exactly. Yes, there’s a book by Simon Singh—The Code Book, Big Bang, and so on. He’s a very well-known British-Indian writer, very well known, actually a physicist. And he wrote several books. A less well-known one of his is called Trick or Treatment? It was also translated into Hebrew. It deals with alternative medicine. He wrote it, by the way, with a professor of alternative medicine. I don’t really understand how that professor is a professor of alternative medicine if he attacks the field in that way. It’s not clear to me. Maybe he was a professor of it and realized there was nothing there, and then went off to knit socks, I don’t know. But from the way he presents it, there is no field, so I don’t know what he researches. Maybe he researches refuting it? Maybe he specializes in refuting the field. Once, when I studied physics at Bar-Ilan—a very small physics department—I had the idea of running for the student union, recruiting the whole physics department, which was maybe twenty people from the advanced degrees. Altogether twenty people could get me elected head of the student union, because who goes to vote in those ridiculous elections anyway? So with twenty people I conquer the student union and declare its dissolution, and that’s the end of the… Of course I didn’t do it, because once I had solved the problem theoretically, I had no reason to invest a gram of energy in it. But maybe that professor is doing the same thing. He studied alternative medicine in order to refute and dismantle the field. Anyway, what they do in that book is go through technique by technique in alternative medicine—simply, each chapter is devoted to another alternative medicine technique. They go through all the studies, at least the ones they found that were conducted systematically, controlled, double-blind—yes, everything required in scientific research—and they show that there is no significant study, not one study showing a significant result for any of the alternative medicine techniques. That’s what the book says. I didn’t check myself; maybe there are other studies they didn’t bring, or studies that came after the book, but that’s what they write there. And that basically means that all of alternative medicine—and anyone who listens a little to people can… Back when I still had energy and argued with people on the AGR bulletin board in Lod, all kinds of people would post about alternative medicine working wonders, and every single time I warned people not to invest a penny in that stuff. And I had bitter arguments with various people—by the way, some of them with advanced degrees in the sciences—who explained to me that their grandmother, all her doctors had given up on her, and then she took Bach drops and two days later she was floating in the air like a swan. So you don’t know what to do with these people. I mean, this is the law of small numbers in the flesh. People have stories of one person who was healed by this technique or that technique. By the way, in alternative medicine they always put a story on top of it. I once went to someone—I had an illness, not a simple one, for a long period, and we really weren’t managing to make progress with regular medicine. They succeeded in dragging me—I was very young—to a few of these magicians, not many, but two or three. And one of them started telling me stories about… he did, I don’t even remember exactly what he did, about meridians and all kinds of… not the Chinese meridians, he had other meridians—energy lines and things of that sort. Now by then I was already in a master’s degree in physics, and I told him: spare me the business about energy lines and meridians. Do whatever you want and let’s see if it helps. That’s all. What happens is that these people always have a story. In alternative medicine too they always have a theory. What do they study in colleges of alternative medicine? They study the theory. Now there is no theory. It’s all nonsense. But what happens is that he has some phenomena that for some reason he observed—usually this is the law of small numbers—and on that he drapes some theory, that there are energy lines that run from the leg to the left ear and from the left ear to the right eye, and if you press here and something comes out there then it can affect you and you’ll be cured of cancer or a cold. Okay? And one time out of a hundred it even happens. Yes, one in a hundred it even happens, and that unequivocally confirms his story, and the story gets attached to the phenomena of small numbers and it becomes a theory, and colleges are built on it, and people spend billions on it, this market turns over billions, and it has already entered the health funds, because once there is demand, the health fund provides the service—which in my view is irresponsible, but that’s what they do. Now I say again: I haven’t checked; maybe there are techniques that do stand up to a statistical test and passed serious studies—maybe, I don’t know. Simon Singh doesn’t mention any such cases. Maybe there are. No, in any case I’m not impressed that there are, because I know that if it passed a statistically significant study, conventional medicine would become interested in it too. It would either find an explanation for it or not, but they use techniques with significant results even if there’s no explanation. As long as it’s statistically significant, as long as it’s been examined seriously. And therefore, as far as I understand, alternative medicine is by definition medicine that doesn’t work. If it worked, it would have been adopted into conventional medicine. So alternative medicine is medicine that doesn’t work. To me that’s a synonymous definition, because otherwise it wouldn’t be alternative. And again, I’m not saying conventional medicine is free of interests, and pharmaceutical companies—everything could be true. But still, I don’t know—without orderly research, I have no ability to trust your findings. The law of small numbers always works. This whole story of the law of small numbers—I’ll maybe give a few examples that make it a bit clearer. Look at the divine kabbalist Oren Zarif, for example. Okay? Some magician from Bnei Brak, doesn’t matter. Probably not religious, with some ponytail like that, and he’s a divine kabbalist. And all sorts of people go to him and pay millions, and there are miracles and wonders in his stories. The miracles and wonders mostly come out of him, but here and there there are also a few people who tell miracle stories about him. I even heard one myself once. Okay? Then I tried to check a little what exactly the miracle was, and it turned out there was nothing there. But stories like that happen. Now where does it come from? It comes from the fact that a person comes, does all sorts of Ali Baba and the Forty Thieves on you, performs magical incantations and all sorts of things like that. Three people out of the thousand who came to him weren’t sick at all—they only imagined they were sick. Three people were healed by the placebo effect. Another three people had spontaneous healing that would have happened even without Oren Zarif. So ten out of the thousand recovered. Those ten tell miracle stories about the talents of Oren Zarif, who succeeded where the doctors failed. The other 990 go home mourning and dejected; no one hears what happened to them or didn’t happen to them. And there you have your stories about the all-powerful magician who can cure every disease. That’s how these stories are created. Now maybe there are some people who really have powers—I’m not ruling that out, it’s possible, I haven’t checked. But there are very many people who boast of such powers, and this is the source. That’s where it comes from—from this statistical failure, the law of small numbers. And that is exactly what scientific research in conventional medicine, or scientific research in general, tries to do: to neutralize all these aspects, because there are such aspects and they’ve been tested experimentally. Placebo is a well-established phenomenon. Healing that comes from your believing that you were given a medicine—not because the medicine works, but because that belief itself, or the belief in the person treating you, can itself heal you, can heal you of a real illness, really heal you from the illness. Not by mistake, not illusions, not mere suggestion. Suggestion has physiological indicators that can be measured afterward. So this is a known medical phenomenon. But you understand that I can do that too, just like Oren Zarif, like anyone else. A lot of psychology, in my opinion, is built on that too—in my interpretation, in some heretical remark. But the point is that the law of small numbers, together with the story draped over it, is an immensely powerful thing. It’s something people spend billions on, swear by, and stake their lives on. And it’s unbelievable how many people—some of them very intelligent, very educated—go after this nonsense. And again I’m saying: I’m not one of those fanatical rationalists—I think I’m not—who deny all these phenomena a priori. If you convince me that it exists, and I check it statistically and see that indeed, in a significant way, you succeed beyond placebo and beyond all the other things, I’ll simply conduct a controlled double-blind study on you. No problem. I’m willing to accept that you have powers; I’m not ruling it out in advance. But usually when people get into these stories and check, they discover there’s nothing there. It’s all nonsense. And it’s a whole world—a whole world of people, scientists, colleges that teach it, billions of dollars, health funds going along with it—it’s unbelievable. Unbelievable. And again I say: maybe here and there there are real things too. I haven’t personally checked the field, and maybe there are techniques here and there that were also tested experimentally. I don’t know. I tend to think not, because again, I believe it would move into regular medicine. But maybe yes, I don’t know. But there is a great deal that doesn’t. A great deal that doesn’t. What I checked of these things, nothing worked for me. Nothing. I went to three such people who tried all sorts of things on me; nothing worked. And of course I also had no confidence in it, so maybe they’ll tell me it was because I had no confidence. Right—the placebo didn’t work on me because I didn’t believe them. Maybe there are people who do believe them, so the placebo works. But that’s only placebo—at least until someone shows me really special cases of another kind. Another example—yes, I think I mentioned it in one of the previous workshops, maybe—a thriller that begins with a man receiving an envelope with a letter. The letter tells him that the sender says: look, I’m a person with supernatural powers, I can guess things you think, and if you believe in me I’ll make you a millionaire. To test me, look—there’s a small envelope inside the large envelope. Inside the large envelope was a letter and a small envelope. There’s the small envelope, fine. Choose a number between one and one thousand. Open the small envelope and you’ll see that inside there’s a note with the number you chose. If I was right and guessed in advance what you would choose, send me ten thousand dollars and I’ll make you a millionaire. Fine, the man guessed, I don’t know, 772. He opened the small envelope—unbelievable, 772. He immediately sent ten thousand dollars, and to this day he never heard from the sender again. What turns out at the end of the book? The sender sent ten thousand envelopes to ten thousand people. Inside each one there was a small envelope with a number between one and one thousand. Yes—there were ten envelopes with the number 1, ten envelopes with the number 2, all the way to 1000, okay? Now assuming that a person chooses a random number between one and a thousand at random, then on average we should expect ten people to find in the small envelope the number they guessed, right? That’s the average. Say they choose at random a number between one and one thousand, and there are ten thousand people, so there should be ten people for whom I guessed their number correctly. Those ten people will send me ten thousand dollars because each is sure that he’s the only one, and that I guessed the number he thought of. So I’ve received a hundred thousand dollars, I’ll buy myself a small apartment in Yeruham, and everything is fine, everything is excellent. That’s the secret. Now think about yourselves if you were in such a situation. You got a letter, you guessed 772, you opened the small envelope and it says 772—why, you’d fall for it on the spot. It would be obvious to you that the person who sent this is Ali Baba and the forty Oren Zarifs. And what is it? The whole story is statistics, the law of small numbers. Okay? That’s a wonderful example of this statistical phenomenon, the law of small numbers. The next example—and they already pulled me on this last time, I said I’d get to it—how can we do without the story of the miracle in Gedera, my miracle, something that happened to me personally. We came back from a celebration in Kfar Chabad—yes, you won’t believe it, we have relatives in Kfar Chabad—and we were there at some family celebration. We came back with the whole family to Yeruham, a big car with eight people, six children and my wife and me. We were coming back from Kfar Chabad in the middle of the night, around one in the morning. We got to Gedera, and suddenly from the side of the road some young woman came out—she had a stop sign, but she didn’t stop, she pulled out. The car in front of me braked, I hit it, my car was done for, completely totaled. After a few minutes I realized this was a young woman who had gotten her license that very day, didn’t stop at the stop sign, and so on and so on. Never mind, it even ended up in court later, not important. But all that became clear only afterward. I’m describing things as they happened. At the moment of the collision, a second after the collision, a large empty car stopped next to us, with my neighbor from Yeruham in it. This was in Gedera. I still hadn’t had time to despair over what I was going to do now with a family of eight in Gedera—how to get to Yeruham and leave the car there for a tow truck or I don’t know exactly what you do at one in the morning—I hadn’t even had time to think all that through, because a large vehicle stopped next to me. It belonged to my neighbor from Yeruham. He said: is there a problem? We said: yes. Elijah the Prophet, welcome in the name of the Lord. We got into his car and all of us drove home, in a large vehicle that holds eight people—not some private car for five including the driver, okay? A large empty vehicle, at exactly the second of the collision, five seconds later, okay? We get into the car, his wife is also sitting there, and she says—actually this was a local Yeruham politician, he used to drive regularly to Jerusalem and back from Jerusalem—he says: listen, I missed the Latrun junction, and at Latrun you turn south. He missed the Latrun junction and somehow reached Ramla and then Gedera. He said: I don’t know, I didn’t know the area at all, we’re driving here and I was about to pass you, I didn’t even see you because the collision had just happened, really just now. His wife says: wait, wait, that’s the Abraham family, stop, maybe they need something. He says: okay, the Abraham family, so what? They still hadn’t seen anything, nothing had happened yet. Well, it had happened, but we still hadn’t gotten out of the car, there still wasn’t anything there you could see that… No, no, stop, maybe they need something. And indeed, we did need something. He stopped, we got into the car, and we drove to Yeruham. And the Sabbath after that, Friday night, I held a tish for the yeshiva guys, my students, and I wanted to tell of the wonders of the Holy One, blessed be He. So I told them: let me bring you a little closer to repentance. I told them about the miracles and wonders that had happened to me, and I asked them: what do you think is the obvious conclusion from this marvelous Arabian Nights adventure? Well then—yes, an open miracle, the Holy One, blessed be He, saved you, and so on and so on. I told them: look, the truth is I don’t know. It really is a fascinating and surprising coincidence. A lot of points in this story are hard to explain. He missed Latrun even though he regularly drives to Jerusalem. He got there, he didn’t see us at all, his wife nevertheless says to stop. In a large vehicle, belonging to my neighbor, and the vehicle is empty, and exactly at the right moment. The whole story sounds like science fiction. That’s true in principle. But on the other hand, I didn’t check how many people got stuck in the middle of the night and no one stopped next to them. Now assuming the chance of such a thing happening is, I don’t know, let’s just say one in a million for the sake of the story, okay? There may have been a million other cases where no one stopped next to you and took you anywhere. Does that mean there was a miracle here? Not necessarily. Maybe, I don’t know—but not necessarily. I have no clear indication. If the chance is one in a million and there were a million cases, then in one of them it will happen, just like the chosen number example. So I was chosen to be the successful case, like the guy with 772 who was the statistically successful guess. Statistics does its thing. Therefore I have no way of knowing whether this was the hand of God, whether it was a miracle or not. I don’t know, I don’t know. I had many other cases where this didn’t happen and I got stuck and remained stuck. Maybe I was especially righteous that day because I went to Kfar Chabad, so by the Rebbe’s merit I was saved—I don’t know exactly what. But I have no indication that there was a miracle here.

[Speaker C] When it happened to you, didn’t you think it was a miracle? Did you immediately feel rationally that this is statistical, I have no…?

[Rabbi Michael Abraham] By then I was an apikores, yes. It was late enough that I was already an apikores. Again, it could be a miracle, I don’t know. But I have no indication of that. I can’t know. It’s impossible to know. There are also—

[Speaker E] Concepts we learned in previous lessons, so I don’t understand why this isn’t considered a special phenomenon. We spoke about an unusual phenomenon and a rare phenomenon.

[Rabbi Michael Abraham] A special phenomenon—if it happens, you need to check what the probability is of it happening, and how many times it happened out of how many times it didn’t happen. I have no problem: it’s special, and it happened, and it’s also rare and unusual. That’s true. But even if it’s rare and unusual, the fact that it’s unusual means you need to examine the matter, because if it isn’t unusual then the rarity means nothing. But if it is unusual, then I check whether it’s rare. In principle it’s rare—but rare relative to what? If the probability is one in a million, that’s rare. But if there were a million people who got stuck and it didn’t happen to them, then no, it’s not rare. If I’m the only one it happened to, then yes, it’s rare. Take another example: the anthropic principle. Hawking, in A Brief History of Time, says suppose there’s a skilled firing squad standing there, shooting at a condemned man—ten skilled shooters, yes?—and they all miss, and he remains alive. Does that mean there was a miracle? He says no. There is some chance that even a skilled shooter will miss. I don’t know, say one percent. Fine? So ten shooters—then it’s ten to the minus twenty. A very, very small chance. But how many executions were there in the world? As many executions as there have been with skilled firing squads around the world, and they always hit—one time they miss. So that’s the time they missed. That’s the atheistic anthropic principle; believers have their version of the anthropic principle, but this is the atheistic version. That’s Hawking’s claim. I think in that case it isn’t correct, for example. I think there, first of all, the number of executions doesn’t seem to reach that level of rarity, and second, it would probably have to be the same firing squad carrying out a billion executions. Different firing squads already make it more complicated; I don’t know. There it’s much more dubious, in my opinion. But still, it’s an argument that has to be checked. And the fact that something happened to you that is both rare and unusual is already a fact worth checking. But it’s worth checking. The conclusion of the investigation is still not clear. It has to be checked. Okay?

[Speaker C] So in your case too, you’d need to find a situation that’s really similar. A large family, a large car, all the complexity of this story—how would you find something parallel and run the same…?

[Rabbi Michael Abraham] I don’t know. I don’t know how one would do that experiment. But I don’t know. How many such phenomena have there been since the invention of the car until today? How many times did someone happen to stop next to somebody and take them home? I don’t know how to do such a test. I don’t think it can be done.

[Speaker C] But you can estimate that a similar story like that would be practically approaching zero.

[Rabbi Michael Abraham] No, no, absolutely not. It doesn’t have to be similar in those same parameters. Why should it be similar? The parameters are accidental parameters. What—why does it need to be similar? If he didn’t stop in Gedera but in, I don’t know, Kentucky, then what? What difference does that make?

[Speaker C] No, but it’s a large family, and that he didn’t take the right turn, and a neighbor of…

[Rabbi Michael Abraham] The whole mistake, and the fact that a large car left from Latrun and missed Latrun—that can happen too. It’s not impossible. Right, it can happen. Fine, so compare it against how many cases there are all over the world, I don’t know. And again I’m saying, without now entering the question of whether it was a miracle or not, I’m trying to show why it’s not necessary that it was a miracle. Now each person can think for himself whether he agrees or doesn’t agree. How many such cases have there been, in your estimation? I don’t know, I have no estimate. But my goal is that you make this calculation before you reach a conclusion. Then reach whatever conclusion you reach—I don’t mind—but remember that this calculation has to be made.

[Speaker F] Rabbi, rabbi, rabbi, this again is built on what the Rabbi started the lesson with today, that we disagree with Nachmanides, right? In terms of how we relate to miracles. Meaning, if it obeys the statistical laws, then again there isn’t really a miracle here and we have no reason to thank Him.

[Rabbi Michael Abraham] No. You want to show the hand of God. If you reach the conclusion… Again, if someone tells me: listen, I believe with complete faith that the Holy One, blessed be He, is involved in everything, and if this happened to me then apparently He did it for me—fine. By the way, He also caused your accident, of course. And He also did whatever He did to everyone else next to whose car nobody stopped. Fine. But then that’s your assumption, and you interpret reality accordingly. No problem. But usually when you bring a case like this, you bring it as evidence that the Holy One, blessed be He, is involved. You’re trying to convince me, who thinks He is not involved: look, in this case it’s proven that He is involved. That kind of proof—there is none. That’s what I want to claim. If you think like Nachmanides and you apply that here, fine. Just of course apply it consistently in the bad as well as in the good.

[Speaker F] Yes, I’m only talking about this approach that says, כביכול, the Holy One, blessed be He… Again, the Rabbi will certainly agree that the Holy One, blessed be He, could have… if He wanted, He could have done what appears in Nachmanides. He could have watched over every worm that Rabbi Kook says gets stuck between two leaves, right? The well-known story. So the Rabbi says no, but He decided to close His eyes and let the laws of nature unfold as they usually do. But He did that; He took responsibility for it. From my perspective, that’s like a person who drove with his eyes closed and crashed and killed someone. He says, I’m not responsible, I didn’t mean to, I wasn’t thinking about you. That doesn’t make so much difference. If the Holy One, blessed be He, could have known that the car would stop after fourteen million… eight children, and He could have closed His eyes or not closed His eyes, that doesn’t change it. The fact is that it happened.

[Rabbi Michael Abraham] That I find hard to accept—that He did it in advance, I find hard to accept—because along the way there are all kinds of human choices, and with choices He can’t know in advance, but that already takes us into other theological issues. Fine. Again, without getting into an argument at all about these examples—let everyone do with them what they want. What I do want to claim is: leave theology aside for a moment, let’s talk about statistics. Statistics says that when you draw a statistical conclusion, know that this is a complicated story. You have to check all these fallacies, and others too, before we reach a conclusion. The law of small numbers is a very powerful law, and you have to be careful with it—a very powerful fallacy, of course, not a very powerful law—you have to be careful with it. Okay, I’ll stop here, and we’ll continue a little more later.

[Speaker G] Just a small comment: the concept of a miracle, conceptually, is complicated, because if we found that this really happens a lot, then it wouldn’t be a miracle. So what are we saying here? If, say, in this description you gave about the car that stopped at night in Yeruham, we ran a check and found that it happens many times, then we’d say okay, it’s probably natural, it’s probably not such a miracle, not so strange.

[Rabbi Michael Abraham] No, if it happened many times, I would still think it was a miracle, because the circumstances show that it’s a miracle. It’s not an observational question. I know how many people get confused, and I know how many people drive a large vehicle, and I know how many people do or don’t stop when they don’t see anything suspicious. So if it happens a lot, then there are many miracles—that wouldn’t change the fact, because my analysis is not because it didn’t happen a lot; my analysis is because of the understanding that a situation like this is an improbable situation.

[Speaker G] It’s the understanding that a natural law—we strip it of being a miracle only because it happens a lot. There’s no difference at all between this and

[Rabbi Michael Abraham] and a miracle except that it happens a lot. No, but I’m explaining to you that it’s a law. If you reach the conclusion that there’s some law here, all the better—explain the law to me; maybe I’ll accept it. But in the meantime, my law says that if a person gets confused in Latrun, there’s no law that says he always gets confused when I’m stuck with my car. If that always happens, and I find some law that does say that whenever I get stuck here, he gets confused there in Latrun, fine, then you’re right. But if it happened many times and I don’t see any connection between the things, then I’ll say there were many miracles. This is a bit connected to the previous series I gave, the series about inference from experience. There we talked about presumptive patterns of three times in the Talmud, right—regarding fixed menstrual cycles or an ox that gores repeatedly, all those things. And there we discussed the Mekor Chaim, who talks about what happens when there are three cases and you don’t have any law tying them together, you don’t have any logic behind them. In such a case, you really don’t infer the conclusion that there is an established presumption. He talks there about leavened food on Passover—Mekor Chaim in the laws of Passover. He writes there about what happens if I found a grain of wheat inside some cooked dish or something like that. After all, if there’s no law that says—it’s seen from the case of a woman

[Speaker G] who is considered lethal, where they don’t say a lethal man. What? You see it from a woman considered lethal, that there is no presumption of a lethal man.

[Rabbi Michael Abraham] The Talmud itself discusses whether it’s the spring causing it or… In the Talmud itself, actually, it’s—

[Speaker G] Yes, you have to arrive at some kind of story, yes, some kind of mechanism.

[Rabbi Michael Abraham] Well, that’s a dispute among the Amoraim there. Seemingly there is a dispute among the Amoraim over whether you need a background story, but the Mekor Chaim claims there’s no dispute at all. Even all the Amoraim require a background story; it’s just that the Amora there who accepts it without a story does so because, from his perspective, mazal is also a story. Fine, and we talked about that in the previous series, and it’s very much connected to here. I said this series would be a continuation of the previous one. Okay, we’ll stop here—it’s already late. Thank you.

[Speaker B] Good night, good night, thank you very much.

Probability and Statistics – Lecture 23

Table of Contents

Full Transcript

Leave a ReplyCancel reply

Table of Contents

Full Transcript

שתף

Leave a ReplyCancel reply