Uncertainty and Statistics – Lecture 6
This transcription was produced automatically using artificial intelligence. There may be inaccuracies in the transcribed content and in speaker identification.
🔗 Link to the original lecture
🔗 Link to the transcript on Sofer.AI
Table of Contents
- Conditional probability and the example of a die
- Medical diagnosis, disease rarity, and the illusion of “99% reliability”
- Sensitivity/specificity, defining test reliability, and reversing the conditioning
- Bayes’ formula as the link between the two questions
- Symptoms as indicators that raise the relevant prevalence
- Evidence in criminal law, “quality of evidence” versus “reliability of the verdict”
- A litigant’s confession and the requirement of some corroborating element in court
- “The burden of proof rests on the claimant,” possession, and the claim of statistical error
- A representative sample and a non-representative subgroup
- The example of health funds and reversal of the condition
- Moving to the halakhic discussion: “Follow the majority,” majority present before us and majority not present before us
Summary
General Overview
The lecture sharpens the idea of conditional probability as the dependence of probability on the amount and type of information we have, and shows how confusion about the direction of conditioning creates mistaken conclusions both in medicine and in law. The central example is a medical test that is very “reliable,” where despite 99% sensitivity/specificity, the probability of actually being sick after a positive result can still be tiny when the disease is rare, and the gap is explained by means of Bayes’ formula. It is then argued that most probability mistakes are not in the mathematics but in imprecise formulation of the question and in the mistaken assumption that a subgroup is a representative sample of the whole. Finally, the lecture opens a preparation for a halakhic discussion of majority: the distinction between a majority present before us and a majority not present before us, and the difficulty of deriving the latter from the verse “Follow the majority.”
Conditional probability and the example of a die
The probability of an event depends on the information given, and additional information removes possibilities and leaves a narrower possibility-space. With a fair die, the probability of rolling a two is one-sixth when no information is given. Given the information that the result is even, the possibilities are two, four, or six, and therefore the probability of two is one-third.
Medical diagnosis, disease rarity, and the illusion of “99% reliability”
A test defined as 99% reliable makes an error 1% of the time in both directions, so a sick person can test healthy and a healthy person can test sick. The answer to the question “I tested positive—what is the chance that I’m really sick?” depends on the prevalence of the disease in the population and is not determined by the 99% figure alone. When the disease appears in one person out of a million, even an excellent test will generate many false positives relative to the few real patients, and therefore most “positives” will actually be healthy, and the chance of being sick after a positive result comes out tiny. The parable of the net and the fish says that the quality of the net alone is not enough when the fish are rare/small relative to what the net “catches,” so the decisive ratio is test reliability versus disease prevalence.
Sensitivity/specificity, defining test reliability, and reversing the conditioning
It is said that the term “test reliability” requires conceptual precision, because there is sensitivity and specificity, and there are false negatives and false positives. Test reliability is defined as the conditional probability in the direction “given that the person is sick, what is the chance that the test will come out positive,” because that is how a test is actually evaluated on people whose status is known by other means. By contrast, “diagnostic reliability” is the opposite question: “given that the test came out positive, what is the chance that the person is sick,” and the two questions can be very far apart when the disease is rare. The gap is presented as the difference between \(P(B\mid A)\) and \(P(A\mid B)\), where the former can be 0.99 and the latter can be close to zero.
Bayes’ formula as the link between the two questions
The identity \(P(B\mid A)\cdot P(A)=P(A\mid B)\cdot P(B)\) is presented as the probability of the joint event \(A\land B\) computed in two different ways. From this one gets Bayes’ formula, which makes it possible to move from a conditional probability in one direction to a conditional probability in the opposite direction by means of the “absolute” probabilities. The application to testing shows that the connection between test reliability and diagnostic reliability is determined by the ratio between disease prevalence and the frequency of a positive test result, not by the 99% figure alone.
Symptoms as indicators that raise the relevant prevalence
When the test is given to the general population without symptoms, a positive result for a rare disease is worth very little. When there are suspicious symptoms, the relevant tested group is a subpopulation in which the disease is far more common, and therefore that same 99% test gives excellent results. The importance of symptoms is explained as their ability to shift the prevalence from one in a million to values like one in ten or one-half, and then the probability after a positive test becomes meaningful.
Evidence in criminal law, “quality of evidence” versus “reliability of the verdict”
The same distinction is then extended to the legal sphere: the quality of evidence is defined as “assuming the defendant is guilty, what is the chance the evidence will point to him,” whereas the reliability of the verdict is “assuming the evidence/verdict determines guilt, what is the chance he is really guilty.” When the number of murderers in the general population is very small, evidence of 99% quality can still lead to the conclusion that the chance the defendant is guilty after positive evidence is very small. When there are legal “symptoms” such as motive, opportunity, and ability, the suspect group is narrowed to a small set in which the prevalence of guilt is high, and then 99% quality evidence becomes decisive.
A litigant’s confession and the requirement of “some corroborating element” in court
The lecture presents the phenomenon that in law one does not convict on a person’s own confession without some additional corroborating element, even though a confession is regarded as very strong evidence. The statistical explanation offered is that the corroborating element is not “more evidence that adds percentages,” but a tool that narrows the relevant population and raises the prevalence of guilt within the group on which the evidence is being applied. The example sets a high-quality confession against rare crime in an enormous population, and argues that without narrowing the population the evidence is “worth nothing” from the standpoint of practical probability.
“The burden of proof rests on the claimant,” possession, and the claim of statistical error
It is said that the basis of the rule “the burden of proof rests on the claimant” is not a statistical presumption that whatever is in someone’s possession is his, but a legal rule of possession. The claim is that statistics about all objects in the world are not relevant to the subgroup of objects that are in legal dispute, and that there is no indication that in this subgroup the plaintiff is more likely to be lying than the defendant. It is explained that once we focus only on cases that came before a religious court, there is no justification for assuming a majority in favor of the defendant, and therefore any attempt to ground the rule in “most objects are with their owners” is a mistake of mixing a general population with a biased subgroup.
A representative sample and a non-representative subgroup
A principle is presented that every subgroup is not a representative sample of the general population, and therefore a statistic that is true of the whole is not necessarily true of the subgroup. The example about animals illustrates that probability changes when we know that the creature is “land-dwelling,” because the information narrows us to a group with a different distribution. The example of election polling in a kibbutz shows that a non-representative sample leads to worthless predictions, whereas the craft of polling institutes is to choose a small but representative sample that predicts with impressive accuracy within the margin of error.
The example of health funds and reversal of the condition
An example is given of two health funds, where the larger includes 75% of the residents and the smaller 25%, and satisfaction is higher in the smaller one (90% versus 80%). The advertising claim “If you’re satisfied, you’re probably with us” is rejected through a calculation showing that most satisfied members come from the larger fund because of its size. The difference is attributed to switching from “given that you are in the fund, what is the chance you’ll be satisfied” to “given that you are satisfied, what is the chance that you are in the fund,” that is, to the direction of the conditioning.
Moving to the halakhic discussion: “Follow the majority,” majority present before us and majority not present before us
It is presented that the verse “Follow the majority” deals with a majority in a religious court, and from it one learns a general principle of following the majority, such as nine kosher stores and one non-kosher one. The Talmud in Chullin 11a distinguishes between a majority present before us, which is a majority physically before us such as the Sanhedrin and the nine stores, and a majority not present before us, such as “a young boy and a young girl” and the claim that most women are not infertile in that way, or “most women give birth at nine months.” It is said that a majority not present before us rests on generalization from experience and from a sample to the whole world, and not on a defined mixture lying before us, and the question remains open where we learn a majority not present before us from if it is not learned directly from the verse about the court.
div>
Full Transcript
[Rabbi Michael Abraham] We’re in the topic of doubt in statistics. Last time I spoke about conditional probability and gave the example of Munchausen syndrome. I did it a bit quickly, and I want to sharpen the point a little, because it’ll also serve us later on. The concept of conditional probability is really based on the idea that when we determine the probability of some event, it depends on how much information we have, or what information we have. Different levels of information can change the probability of a given event. Right? The example I gave was: if you ask what the probability is that when you roll a die the result will be two—so if you have no information, then with a fair die the probability is one-sixth. But if you know that the result that came up was even, then the probability that it was a two is one-third. Because I already know the result was even, that means there are only three possible results, not six: two, four, or six. Assuming the die is fair, the probability of those three results is the same, and therefore each one has probability one-third. And that’s what’s called conditional probability. What does that mean? I can ask what the probability is that a two came up—that’s ordinary probability. I can ask what the probability is that a two came up on condition that I know the result was even—that’s conditional probability. Conditional probability is always the probability of a certain event given some information. What is the probability of this and this event, given that I have such and such information? Usually, all that the information does is remove some of the possible events by telling me they didn’t happen, and leave some possibilities among the events that did happen. So that’s the concept called conditional probability. Now, I talked about Munchausen syndrome and said that what the discussion really was—I’m not going back to Munchausen syndrome, I’ll go back to the issue of medical diagnosis, because that works better for me as an example of what I want to say. Let’s say we have a medical test whose reliability is ninety-nine percent. That is, it misses one percent of cases. It gives a wrong result in one percent of cases, and I’m assuming that works both ways—false positive or false negative. Meaning there’s one percent where a sick person will come out healthy, and one percent where a healthy person will come out sick. In other words, the error in both directions is one percent. Now the doctor sends me to do a test for some disease he suspects I have, and I came out sick. The question is: what’s the chance that I’m really sick? So at first glance—right, I won’t ask you again, we already talked about this last time—at first glance people immediately say ninety-nine percent. Meaning, the test is ninety-nine percent reliable. So if it said I’m sick, then there’s a ninety-nine percent chance I’m sick. But that’s a mistake. You can’t answer the question of what the chance is that I’m sick unless you know the prevalence of the disease in the population. I’ll give an example. Suppose I have a disease that appears in one person out of a million in the population. Okay? Say I have a country—I don’t know—of ten million people, so there are ten sick people. So let’s say the probability—the prevalence, not probability—the prevalence of the disease is one in a million. Okay? Now let’s do the calculation assuming I send all those million people for the test. What will the results be? Actually, you know what? Let’s do it differently. Let’s say it’s one in a million, and I send all one hundred million residents of the country for the test. One hundred million. Let’s take a country of one hundred million people. The numbers come out more comfortably that way. Let’s take a country of one hundred million people, of whom one hundred are sick. Okay? Now I send all of them to do the test, all one hundred million, just for the calculation. What will come out? Well, from the hundred sick people, the test will identify ninety-nine and determine that they’re sick, and it’ll miss one, right? Because it misses one percent, so one out of the hundred sick people will come out healthy. From the rest—the people who are all healthy—there are ten million of them, ten million minus a hundred but approximately ten million, then since the reliability of the test is ninety-nine percent, of them one hundred thousand will come out sick. One hundred thousand out of ten million is one hundred thousand sick people, who of course are healthy—it’s an error—but one percent error out of ten million healthy people means one hundred thousand will come out sick, and all the rest will come out healthy. Meaning, it’s a high percentage, ninety-nine percent of the time it’s right. But the number of sick people we discover there is one hundred thousand. So if we put the whole population through this test, how many sick people do we get altogether? One hundred thousand and one—sorry, one hundred thousand and ninety-nine, right? We got ninety-nine out of the hundred actual sick people, and one hundred thousand from among the healthy people also came out sick. So that’s one hundred thousand and ninety-nine, about one hundred thousand one hundred. Okay. Out of that one hundred thousand one hundred, how many are really sick? A hundred. The ninety-nine are the ones who are really sick; the one hundred thousand are healthy people who came out sick because of a testing error. So basically we have one hundred out of one hundred thousand who are sick, so the chance that I’m sick is one in a thousand. Notice: with a test—right—only one out of every hundred thousand among those who came out sick is actually sick. Okay? Is the calculation clear? It’s a simple calculation. Meaning, one hundred thousand people came out sick on the test, and among them there are only one hundred who are really sick; all the rest are simply false positives. Okay? So if I ask, among those who came out sick on the test, how many are really sick? One hundred divided by one hundred thousand—that is, one in one hundred thousand. One in a thousand, sorry. Okay? So notice what happened here. They sent me to a test whose reliability is ninety-nine percent. I came out sick. What’s the chance that I’m really sick? One in a thousand, one tenth of a percent. Negligible. Meaning I can just crudely ignore it and go home happy and enjoy life. There’s really no need to be troubled by the results of this test when the test is ninety-nine percent reliable. Why is that? Because the disease we’re trying to test for is a rare disease. And I said this—I explained it by way of the parable of the net and the fish. When I want to catch small fish, the holes in the net I use have to be smaller than the size of the fish. So even if I have a very fine net with very, very small holes—that is, a very high-quality net—if I’m dealing with fish whose size is smaller than those holes, the quality of the net won’t help; I won’t catch the fish. Now a test is basically a kind of net. I’m trying to catch a fish, right? Who’s sick—that’s the fish in the analogy. Now if there are very few sick people, meaning the percentage of sick people is very, very small, then the quality of the test has to be enormous for the test to have significance. And the ratio is always: the reliability of the test versus the prevalence of the disease. Okay? If there’s a one percent error in the test, the test can be relevant for a disease that exists in one percent of the population. Then if I came out sick, there’s a meaningful chance that I’m sick—half. But never mind, the probability is already meaningful; you can’t ignore it. Okay? So the relevant orders of magnitude here are disease prevalence versus test reliability. Now what’s so…
[Speaker C] Rabbi, but basically—I completely agree with the Rabbi, of course—but there was really a conceptual naming error here in the term “test reliability.” Because the Rabbi called it test reliability and went on that way, but that doesn’t mean that’s necessarily the right way to define the reliability of a test. There’s sensitivity, but there’s also specificity, false negative, false positive. The term itself matters here, saying “test reliability.”
[Rabbi Michael Abraham] I’m explaining that now. What lies at the basis of the matter is conditional probability. Why? Because when I discuss the reliability of a test, what does that datum mean? What does it mean that the test is ninety-nine percent reliable? So I explained that earlier. It means that out of one hundred sick people, ninety-nine will be correctly identified; or out of one hundred healthy people, ninety-nine will also indeed be found healthy. Okay? That’s the meaning of test reliability. Let’s formulate it in the language…
[Speaker C] But if the Rabbi had defined test reliability as…
[Rabbi Michael Abraham] So test reliability is defined as: given that there are one hundred sick people, or given that the person is sick, what is the chance that the test will detect it? That’s called test reliability; that’s how you test it. Okay? That’s the meaning of this parameter called test reliability. Test reliability means the conditional probability: given that the person is sick, what is the chance that the test will indeed come out that he is sick? Okay? Which is really a conditional probability between two events. One event is that the person is sick; the second event is that the test says he is sick. And the conditional probability is formulated this way, and notice how subtle this is. The probability—test reliability—is defined in this direction, and that’s how it’s defined: if the person is sick, what is the chance that the test will classify him as sick? That’s the definition of this parameter, test reliability. Why define it that way? Simply because that’s how you test it. Okay? I mean that’s how you test the test. How do you test the test? You take people that we know are sick by other means, doesn’t matter how, and I run the test on them and see for how many of them it’s right. Okay? So that shows me the reliability of the test. But notice: when I go to be tested and I come out sick, and now I ask myself, what is the chance that I’m really sick—that’s exactly the opposite question. That’s the reverse conditional probability. Let’s make it a bit clearer; I’ll just write it for you. Look. I define the two events. One of them is the event that I’m sick. I came out sick in the test, okay? That’s event B. Now test reliability is defined like this: the probability—let’s call it P, probability—of B given A. Can you see, or is it too small? Okay? What does that mean? Look, I’ll translate. P is the probability, the chance, of event B given that A is known. Given that it’s known that I’m sick, what is the chance that I’ll come out sick in the test—B? Okay? That’s the notation. Now the question I ask after I’ve been tested is the opposite question. The reliability of the diagnosis, let’s call it that. Okay? The reliability of the diagnosis is defined as P of A given B, not P of B given A. Why? Because the reliability of the diagnosis basically means: given that I came out sick in the test—B, right? That’s the datum. Now I ask myself what is the chance that I’m sick—that’s A. So here it’s not P of B given A but P of A given B. That’s a completely different question. And it turns out that if the disease I’m dealing with is very rare, then there’s a very big difference between this number and this number. This number is ninety-nine percent, 0.99, and this number is one in a thousand, 0.001. A dramatic difference. Okay? When our whole scale runs from zero to one, you have to understand: 0.001 is almost at zero, and 0.99 is almost at one. They’re really at opposite ends. There’s no connection between these two questions, even though at first glance they look very similar. If someone tells me the test is ninety-nine percent reliable, I immediately say okay, if you were tested then you’re probably sick—ninety-nine percent you’re sick. Wrong. A big mistake. So the point—and what you asked earlier—it doesn’t depend on how you define the term test reliability, because that is how test reliability is defined; that’s the datum they give us with tests. That’s how it’s defined. Obviously, if I know somehow the reliability of the diagnosis, then there’s no problem; I just know the result, I don’t need to ask myself. But given that I have the reliability of the test and I ask what the reliability of the diagnosis is, the answer is: there’s no way to know. It depends on the prevalence of the disease. Now of course—and I think I mentioned this—if I have some other indications that I’m sick, say the doctor sends me to do this test. Usually he doesn’t just send me for no reason. Sometimes yes, sometimes they send the whole population to do a coronavirus test, regardless of whether you have symptoms, and then really that’s what it’s worth. But if they’re actually sending me because I have some suspicious symptoms, some indication that maybe the disease is here and therefore we need to test, that’s already a completely different world. Why? Because the moment I have suspicious symptoms, it means that the number of sick people among the people who have symptoms is already much higher than the one in a million I talked about earlier. Among those who have symptoms, the number of sick people might be, say, half. So you still need the test to know whether I’m sick or not—you can’t know with certainty. But once the prevalence of the disease is already half, not one in a million but half, then a ninety-nine percent test is excellent. So the importance of symptoms—even though on the face of it they only tell you there’s a fifty percent chance you’re sick, which doesn’t sound like something terribly significant—is dramatic. Because if there are symptoms that turn my percentage of being sick from one in a million in the general population to one in ten or to half—that’s even better, but even one in ten—a test with ninety-nine percent test reliability gives excellent results. So it depends very much on whether I have symptoms. The same thing I said about implications for legal evidence. A judge wants to know whether Reuven murdered Shimon. When they bring him evidence with strength or quality of ninety-nine percent in this definition, P of D given A, okay? And now the evidence indicates that Reuven murdered Shimon. What’s the chance that Reuven really murdered Shimon? Same thing—notice. The quality of the evidence is defined as: assuming that Reuven murdered Shimon, this evidence will find that he is indeed the murderer—say DNA evidence, doesn’t matter. Okay? Whereas the reliability of the diagnosis in this context is basically the reliability of the verdict. Meaning: assuming he is found by the verdict to be a murderer, what is the chance that he really is a murderer? A completely different question. Now if the number of murderers in the general population is very, very small, then I have excellent evidence and it’s worth nothing. Even if the evidence is at the ninety-nine percent level, it’s worth nothing. If the number of murderers in the population is, say, one in a million, and the quality of the evidence is ninety-nine percent, and I find evidence against me that I’m a murderer, the chance that I’m a murderer is one in a thousand. No chance that I’m a murderer. That’s why it’s very important for doctors and legal professionals to understand these things well. What happens is that even in the legal world there are symptoms just like with disease. Meaning, you were in the area, you had motive, you had the ability—all those stories in criminal law: you need opportunity, you need ability, and you need motive. So assuming all those exist, how many such people are there now? Five. One of those five is the murderer—he is the murderer, okay? Let’s say for the sake of discussion, or at least one of three. Now if I have evidence of ninety-nine percent quality, no problem, because it’s one out of three, not one out of a million. One out of a million is if I ask how many murderers there are in the whole population. But when I have other indications, that narrows the group only to those who have those indications. Among them it’s already one out of five. That’s already a high prevalence, so evidence of ninety-nine percent quality is excellent. Okay?
[Speaker C] Rabbi, Rabbi, it’s interesting that—just joking maybe—with the bitter waters administered to the sotah, to determine the sotah, when they gave her the bitter waters, the Sages actually said that when adulterers became numerous, the practice was discontinued. Seemingly they should have said the opposite—
[Rabbi Michael Abraham] If adulterers became numerous, then the reliability of the test went up. No—because when adulterers became numerous, it wasn’t because the evidence became worse, but because they didn’t want to do it.
[Speaker C] Sure, sure, I’m just saying it.
[Rabbi Michael Abraham] Anyway, so what’s happening here now—okay, how can I know, how can I know the relation between this datum and that datum with regard to tests, or legal evidence, or anything else? We’ll still see more examples of these things. I want to know whether I can somehow determine the relationship. It turns out I can—that’s what’s called Bayes’ formula. And Bayes’ formula is based on the following idea. Sorry for a little formalism, but it’s not difficult formalism. I assume here there shouldn’t be any problem following this. Good. I’m now writing the… okay. What is this doing to me? Right. Good. Let’s write this thing: P of B given A times P of A. Okay? What does that expression mean? Notice: this expression really means the chance of A and B, right? The chance that A happened times the chance that, given A, B also happened—that is exactly the chance that both A and B happened. Okay? I’m saying: I start with A. Let’s say, what’s the chance that A happened? Given the chance that A happened, what’s the chance that B also happened? The product of those two is exactly the chance that both A and B happened. Okay? For example, in the context of the number two on a die, I ask: what’s the chance of getting a two given that the result is even? Okay? So I say: what’s the chance that the result is even? Half. What’s the chance it’s two given that the result is even? One-third, right? Half times one-third is one-sixth, right? Which is exactly the chance that a two came up. Okay? So okay, in exactly the same way I can also write, of course—it’s the same logic exactly, right?—this is also equal to P of A and B. And here it’s a product, here it’s and—right? Not in the same sense, but never mind. Okay? Right? It’s the same thing. Meaning, I can calculate the chance that both A and B happened by starting with A and saying what’s the chance that A happened, and then asking okay, assuming A happened, what’s the chance that B also happened—or by starting from B: what’s the chance that B happened, and then asking, given that B happened, what’s the chance that A happened. In both cases I get the result that this is the chance that both A and B happened. Why is that important? Because once both of these are equal to the same thing, then they’re also equal to each other. Right? So that means this is… this is simply the proof of Bayes’ theorem. That this equals that. Right? Because both are equal to the same quantity. Now you see we got a formula that helps us move from P of B given A to P of A given B. The ratio between them is simply like the ratio of the absolute probabilities. Let’s try to see how this works in our test. Okay? Remember? In our test, A is that I’m sick, B is that I came out sick in the test. Okay? So basically, when I ask myself—the reliability of the test is ninety-nine percent. P of B given A. Okay? That’s 0.99. 0.99 times—what is P of A? What is the chance that I’m sick? That’s the prevalence of the disease. Right? The prevalence of the disease is one in a million. Okay? Equals P of A given B, which is what I’m looking for. Right? I want to know how much that is. Times P of B.
[Speaker D] P of B given A times P of A? You’re supposed to write P of B given A times P of A, right? If you want to get the…
[Rabbi Michael Abraham] Right, so what’s the chance that I’m sick? One in a million, right? That’s the prevalence of the disease.
[Speaker D] Right, right.
[Rabbi Michael Abraham] Times the reliability of the test, which is 0.99. P of B given A is the reliability of the test. The reliability of the test is 0.99 times one in a million equals P of A given B—that’s what I’m really looking for, right? That’s basically one in a million; 0.99 doesn’t matter, it’s one in a million. Okay? Times what’s the chance that I’m sick? What’s the chance that I came out sick on the test? The chance that I came out sick on the test is: if we have a million people, right, then basically one percent of them will come out sick on the test, right? Agreed? If the test makes mistakes, then one percent of the people will come out sick. Overall, one percent of the people will come out sick because of the test error. So you see how we get P of A given B.
[Speaker D] No, it’s not one percent of people who come out sick; it’s one percent of the healthy people who come out sick, and ninety-nine percent of the positive people—
[Rabbi Michael Abraham] Correct, but ninety-nine percent of the sick people is negligible. Obviously, yes, you need to add something here, but it’s negligible. Because the number of sick people is tiny; it’s one in a million. So let’s say I have ten million people—then one in a million is only ten sick people. Ninety-nine percent of them adds another nine. Fine, that has no significance. Another nine on top of the hundred thousand you have—it’s negligible. Okay? And that’s it, so you see we get P of A given B. Okay? That’s basically one in ten thousand—I said one in a thousand before, I don’t know why; it’s one in ten thousand. Okay? So again, leave the mathematical details aside for the moment. I’m just trying to give you a feel for why these two questions are different questions, and the answers to them can be very, very different from one another. And the relation between these two is one thing that interests me. Right? The relation between this and this—that’s what interests me. Right? It basically follows from the ratio between this and this. Right? The ratio between them is exactly the ratio between the rarity of the disease and the quality of the test. Okay? That’s basically what I said earlier in a non-formal way. This is just the calculation that gives it. Now let’s leave this formalism for a moment and take a few examples that I won’t do mathematically, because it’s exactly the same as what I’ve done until now, but you can also understand it without the math. Look: basically the claim is that when I ask the question—by the way, in probability generally, the mathematics, at least of basic probability, is very, very simple. You can’t really go wrong in the mathematics. The mistakes—and there are lots of mistakes in probability, even in simple probability—the mistakes are always the result of imprecise formulation of the question. Meaning, if you formulated the question and the data precisely, then yes, exactly—if you formulated the question and the data precisely, the problem is solved. Meaning, the mathematics is not the issue here. It’s always a question of formulation. And notice what we’ve just seen—that’s exactly it. When we formulate the data and the question carefully, we suddenly see that these are opposite questions. The reliability of the test is the probability P(B given A). The reliability of the diagnosis is P(A given B). So you have to think very carefully about the meaning of the data and how we formulate the question in order to reach the answer. Let’s look at another very similar example. Suppose I’m looking at the reliability of a judge, okay? Reliability of a judge means—say this judge is an excellent judge. What does that mean? Let’s say at the level of ninety percent. What does it mean that he’s an excellent judge at ninety percent? It means that he’ll catch ninety percent of murderers. Meaning, if someone comes before him accused of murder, then in ninety percent of cases the judge will rule that he is a murderer. Okay. Now the judge ruled that someone is a murderer. What’s the chance that he’s really a murderer? Do you see? It’s exactly the same thing we saw before, right? There are very few murderers. If the judge misses both murderers and non-murderers—say at the same rate, ten percent error in both directions—when this excellent judge rules that Reuven murdered, the chance that Reuven murdered is negligible. There are very few murderers. So you have to understand that it’s very important to define the question you’re asking and the datum you have. We have data about the reliability of the judge—think very carefully what that datum means. It means: given that Reuven is a murderer, what’s the chance that the judge will rule that he’s a murderer? Ninety percent, because he’s an excellent judge. Now I ask: the judge ruled that he’s a murderer. I ask myself whether the judge was right—what’s the chance that he really murdered? Zero. An excellent judge—zero. Why? Because the number of murderers in the population is tiny. And the question I’m asking about the reliability of the ruling is the opposite of the question about the quality of the judge. The quality of the judge and the reliability of the ruling are opposite questions. When the phenomenon is not rare, then there is some connection between these two quantities—they’re not the same, but they’re not so far from each other. When the phenomenon is rare, like murder, like crib death, what we talked about with Munchausen syndrome, and so on, that means there is really no true connection between these two data points. And even if I have a court of three marvelous judges, each one correct ninety percent of the time, and they found that so-and-so is a murderer—you can calmly let him go free. He’s not a murderer. Only if I narrow the population of potential murderers, as I said earlier. Meaning, if I know that ninety-nine point nine percent of the population could not have murdered—they weren’t in the area, they had no motive, they weren’t around, they couldn’t use the murder weapon, doesn’t matter, they didn’t have that tool, whatever it is—I managed to eliminate most of the population, and I’m left with only three suspects. Now if I have three excellent judges, each with ninety percent quality, and they rule that specifically Reuven is the murderer and not Shimon or Levi, then probably Reuven is the murderer. Because I reduced the prevalence—or rather I increased the prevalence of the phenomenon. Without the signs, without the additional indications, what’s the prevalence of the phenomenon—how many murderers are there in the general population? One in a million. But once I narrowed the population to three people, and now I ask what’s the chance that one of them is the murderer—meaning the a priori chance—without the evidence, I say one-third. Meaning the prevalence—or not the probability, but the prevalence—of murderers in this new narrowed population is one-third, not one in a million. So this is very, very important. You know, for example, there are legal scholars—they themselves don’t understand why this is, but there is statistical logic behind it. Suppose someone gives a confession. According to legal rules, in Israel at least, you can’t convict a person on his own confession unless there is some additional corroborating element. Some extra indication that he really is the murderer, the criminal, whatever is under discussion. Now why? After all, the confession is the queen of evidence, right? It’s excellent evidence. So why isn’t that enough? Why do you need some extra corroborating element? And the corroborating element really is marginal evidence, not significant evidence. Why is that extra bit so important? The explanation is exactly what I said here. The moment I have that extra bit, it now basically says: let’s take the whole population. I have ten million residents in Israel. I have no corroborating element and nothing else. One out of those ten million committed the murder. I have no idea who it is. Now a person comes and confesses. And let’s say that confession is excellent evidence, ninety-nine point nine percent—whoever confesses is indeed guilty.
[Speaker D] So maybe we’re worried he’s mentally unstable?
[Rabbi Michael Abraham] That’s the famous Maimonides, yes. Never mind, I’m not getting into the explanations right now, only the statistical phenomenon. So I’m saying: there is excellent evidence. But that excellent evidence is trying to capture a phenomenon that is much rarer than the quality of the evidence. It’s worth nothing. But if there’s some corroborating element—say fingerprints found at the scene, or genetic findings found at the scene that match only five people in all of Israel, and among them is the defendant. Okay? Now the prevalence of the phenomenon is one out of five, not one out of ten million. In that situation, evidence of ninety-nine point nine is excellent. Even though this corroborating element is very weak. That corroborating element is only—and certainly not enough to convict; nobody would dream of convicting on the basis of such a thing. It gives a twenty percent chance that he is the murderer. That’s weak evidence that gives you twenty percent that he is the murderer. What is twenty percent? Twenty percent has no legal weight. But once that twenty percent narrows the number of potential murderers, it creates a dramatic reversal of the result.
[Speaker D] So you’re assuming a litigant’s confession is not one hundred out of one hundred?
[Rabbi Michael Abraham] It’s not one hundred, yes. That’s Maimonides’ explanation, what you said earlier. Never mind, those are just explanations. But it’s not one hundred. Yes, it could be that he went crazy, it could be that all kinds of things happened.
[Speaker D] Yes, but if we assume there’s a one-hundred-percent test, a coronavirus test that’s one hundred out of one hundred, then all the statistics…
iv>
[Rabbi Michael Abraham] If it’s one hundred percent, then it’s one hundred percent—there’s nothing to say. So fine, tests that aren’t good aren’t good, that’s true. But there are no such tests. So the point is that this additional element—what does it actually do? It does not constitute evidence against the person. People think there are two pieces of evidence here: he confessed, and there’s also another piece of evidence against him, that other evidence. Fine—but his confession is evidence at the level of ninety-nine point nine percent, and the extra thing adds another twenty percent to the… what difference does it make? That’s not interesting. Why do you need that extra addition? Isn’t ninety-nine point nine enough? The answer is no, it’s not enough. That is exactly the reason—it’s not enough. Only if you have some additional element—he was walking around at the scene, genetics that leave him together with a few other people but not too many, or things like that. Why? Because that’s not evidence against him; it narrows the potential population to which I apply the evidence. That is what determines the prevalence of the phenomenon, not the quality of the evidence. And therefore there has to be some additional element. Without some additional element, it really is worth nothing. This isn’t just nitpicking. It’s a very important point. I don’t know how many jurists understand this, but the fact that they operate this way—that’s the legal rule—and there’s a great deal of logic behind it. Now, the prevalence of the phenomenon, as I said before, is something we can control through that extra element, or the addition, or whatever it may be. What it does is narrow the population I’m looking at, and then good evidence can work properly. As long as the potential population is very, very large—in other words, the prevalence is small—even good evidence, very good evidence, is not good enough. I’ll give you another example that’s very easy to understand intuitively. Look, there’s a rule that the burden of proof rests on the one seeking to extract property from another. Right? If a person—Reuven—sues Shimon, then the burden of proof lies on Reuven. And if Reuven doesn’t bring proof, then Shimon wins the case. Shimon is the current possessor, and he wins. The question is why. There are those who think that this derives from a presumption that whatever is in a person’s possession belongs to him. All right? What does that mean? There’s some such presumption that if something is found with a person, then it probably belongs to him. Now that presumption is not simple at all. Why? First of all, because there are situations where that presumption doesn’t exist. For example, those goats that ate peeled barley—goats wandering around and happening to be in my yard. So the fact that they’re by me is not an indication that they’re mine, and still the burden of proof is on the one suing me. But beyond that, try for a moment to think what this really means. When Reuven sues Shimon, I ask myself: which one of them is the liar? Right? That’s the real question. Is the plaintiff, Reuven, the liar, or is Shimon the liar? Do you really think there’s a greater chance that the plaintiff is the liar? Does the presumption that whatever is in a person’s possession is his really lead to the conclusion that probably the plaintiff is the liar? Nonsense. Where’s the mistake? The same mistake I spoke about earlier. What do I mean? When I look at the population as a whole, I ask myself: if the item is with you, is it yours? The answer is usually yes. Usually yes. Go around all the houses, all the places, and check every object found in someone’s house—does it belong to him or not? The overwhelming majority of things are found with their owners. Unless he lent them out, or they were stolen, things like that—that’s a small minority. Usually things are with their owners, right?
[Speaker D] You’re checking the objects and the owners?
[Rabbi Michael Abraham] The objects and the owners. Checking whether the objects belong to the owners or not. All right? Now clearly there’s an overwhelming majority here, a large majority of the objects. But here we’re not talking about all objects among all people. We’re talking about a very narrow group of objects that appear in cases over which there is a legal dispute. Let’s check—let’s focus only on cases where there is a legal dispute. Reuven sues Shimon over a certain object; the question is whether it belongs to Reuven or to Shimon. Among those cases, and only those, can I still say that in most cases Shimon is telling the truth and Reuven is lying? I see no indication of that at all. Why would I? In that small subgroup I would say it’s fifty-fifty. Fifty-fifty: either Reuven is lying or Shimon is lying. I don’t know. On the contrary, if a person is holding an object, it’s easier for him to lie and say it’s his. Why would someone attack somebody else, where that other person has an object, and lie that it’s his? The Talmud says there is a presumption that a person does not bring a claim unless he has something to claim. I don’t sue for no reason; apparently if I’m suing, then apparently I have some basis. Why did I pounce on this particular person and not a hundred others? Meaning, there is no reason in the world to assume that the plaintiff Reuven is more likely to be a liar than the defendant Shimon. Therefore this has nothing to do with the presumption that whatever is in a person’s possession is his. Why? Because I am looking only at a very narrow subgroup of the population, or of the objects—only those objects involved in a legal dispute. And with respect to those objects involved in a legal dispute, as far as I’m concerned it’s fifty-fifty: half the time the plaintiff is the liar, half the time the defendant is the liar. I have no other indication. And this in no way contradicts the general statistics about all objects in the whole population, where indeed an overwhelming majority of objects are with their owners. That is true.
[Speaker C] So why does the burden of proof rest on the one seeking to extract property from another?
[Rabbi Michael Abraham] I can’t hear.
[Speaker C] So why does the burden of proof rest on the one seeking to extract property from another? Excellent question.
[Rabbi Michael Abraham] Therefore the basis of the rule that the burden of proof rests on the one seeking to extract property from another is not a statistical basis. Whoever thinks that is making a conceptual mistake,
[Speaker D] That’s not correct.
[Rabbi Michael Abraham] It’s a legal rule, not a statistical rule. There is a legal logic to leaving the advantage with the defendant. Because if I don’t do that, then lots of people will pounce on all sorts of others. Suppose I were to say: in such a case we split it. I have no proof, this one is plaintiff, that one is defendant, let’s split it. You understand that it would be unbearable to run a society that way. Because the moment the rule is “split it,” it’s obvious that masses of liars will now come and claim the property of anyone and everyone, and they’ll gain half the property—because we split it.
[Speaker C] Rabbi, rabbi—meaning, the practical difference between the two approaches would be what happens in Sodom and Gomorrah. According to the approach the rabbi is saying, in Sodom and Gomorrah this rule would remain in place, but according to another argument that says this really is statistical, then there, where the presumption is that most objects are not with their owners, where the presumption is that property under a person’s hand is not his, then maybe…
[Rabbi Michael Abraham] Yes, but it’s not a matter of approach. Whoever says that is simply mistaken. It’s not a question of approach. Whoever says it is mistaken. There is no connection between the rule that most objects are with their owners and the law that the burden of proof rests on the one seeking to extract property from another. No connection whatsoever. Statistically it’s a mistake. Simply wrong. And that is exactly the point, because you have to focus on the subgroup, and in that subgroup the statistics are skewed. Not every subgroup is a representative sample of the whole population. It may be that you chose a certain subgroup whose internal distribution is different. Okay, take—I don’t know—among animals in general, how many animals have four legs? Not all that many. If you look at land animals, the percentage of four-legged animals is much greater—not in the air and not in the sea, right? Much greater. What does that mean? So if a donkey comes before me and I ask whether it has four legs or not—what should I say, probability one-half? Because among all creatures in the world, the number of creatures that have four legs is, say, one-half—I’m just throwing out a number, okay? Nonsense. Because I know the donkey belongs to the land-dwellers. Among land-dwellers, the probability is point eight—eighty percent—not fifty percent. Now you understand that this is basically conditional probability. Why? Because what I’m really saying is: what is the probability that it has four legs given that it is a land-dweller? I have additional information, and that information narrows the group about which the question is being asked, and therefore the probabilities must change. I have more information. Once I have more information, I can focus more precisely on the relevant groups, and in those groups the statistics may not be the statistics of the larger group—they may be skewed statistics, and therefore the probability will be different.
[Speaker D] Rabbi, I didn’t understand. Right—in the end, behind the rule that the burden of proof rests on the one seeking to extract property from another stands the presumption. And that presumption is some kind of probability, isn’t it?
[Rabbi Michael Abraham] No. No presumption stands behind it.
[Speaker D] I didn’t understand that point.
[Rabbi Michael Abraham] What stands behind it is possession. Possession is not a presumption. Possession is a legal rule that says: if I am the current possessor, the burden of proof is on you. But that does not mean that if I am the possessor, then statistically it is really more likely to be mine. That is not true.
[Speaker D] Then why not? If you look at all the objects that come before a court, then you see there is a greater chance that it belongs to the defendant.
[Rabbi Michael Abraham] No, no—if you check the objects in court, who says you’ll discover there’s a majority? Why would you? Most objects aren’t disputed at all.
[Speaker D] The presumption says that.
[Rabbi Michael Abraham] No, the presumption is not statistical—that’s what I’m saying. I’m now trying to construct that presumption, asking: let’s see the data and see whether such a presumption can be built. Check the data and you’ll see you have no such data. Put the probabilities in—how many objects are litigated in court? Very few. So of course among all objects, most objects sit with their owners. But most objects also are not litigated. As for the objects that are litigated, who says that even among them the majority belong to the defendant rather than the plaintiff? Why assume that plaintiffs are more likely to be liars than defendants? There is no reason to assume that.
[Speaker D] Because it’s under his hand?
[Rabbi Michael Abraham] No, and that’s exactly what I’m saying—there is no logic in that, no statistical logic. This is not a statistical claim, unequivocally. Whoever says it is simply mistaken.
[Speaker D] So I didn’t understand why. Why isn’t it statistical logic?
[Rabbi Michael Abraham] Because among the objects that are disputed, how do you know the majority belong to the defendant? Explain to me how the Sages built this proof, how they arrived at the rule of possession. They would have had to check, right, the situation in the world. Let’s check: take all the objects subject to legal dispute. Now we want to see whether in most cases the defendant is right and the plaintiff is the liar. Is there such a survey? Has anyone ever done such a survey? Is it even possible to discover that? First of all, you can’t do such a survey at all, because you don’t know whose object it is—this one says this and that one says that. Maybe you could do a survey among cases where we have witnesses. Then the witnesses tell me who the object really belongs to, and who is lying and who isn’t. But even that would not be a good survey. Why? Because I’m talking about a case where there are no witnesses. So let’s narrow it down to cases where there are no witnesses. If there are no witnesses, then maybe that itself is because the object is not his—not simply a case where there happen to be no witnesses. And therefore, in that subgroup—among the objects over which there is a claim, which is already a subgroup of all objects—within that subgroup there is an even smaller subgroup: objects over which there is a dispute and we have no evidence. How do you know that in that group the majority of the objects really belong to the defendant? You have no way of knowing that, and logic doesn’t say so either.
[Speaker D] Most objects in the world belong to their owners.
[Rabbi Michael Abraham] Correct—in the world. So what? Most animals in the world also don’t have four legs. Now I’m looking at donkeys, and I ask whether this donkey has four legs. The answer is yes. Why should I care that most animals in the world don’t have four legs? In the group of donkeys, they all have four legs. That is exactly the meaning of conditional probability. You don’t look, in the absence of information, at the most general group. You do have information, so let’s look at the group connected to that information. And there the statistics may be different. Again, like with the die. The chance of getting a two on a die is one-sixth—that is the chance in the absence of information. But if I have information that the result is even, then I have narrowed the set of outcomes to three out of the six. So now the chance of getting a two is one-third, not one-sixth. Any information I have shrinks the group I am looking at. Once I shrink the group I’m looking at, the statistics may already be different.
[Speaker D] And here what additional information do you have? In this case of a claim in a religious court, what additional information do you have?
[Rabbi Michael Abraham] I have information that this object is the subject of a legal claim. Most objects in the world are not subject to legal claims, so for most objects they really are with their owners. But for objects that are subject to a legal claim, who says they are distributed the same way? There is no reason in the world to assume that objects subject to legal claims constitute a representative sample. And that’s another concept we’ll need to know: a representative sample. What does that mean? When I take a certain group and do statistics on it in order to discover the general statistics in the whole world, in the whole population. Usually I’ll be mistaken. In order to do it correctly I need to choose a group that is a representative sample. Yes, take election polls, all right? Someone wants to know what the election results will be, so he goes to Kibbutz Ashdot Ya’akov and does statistics there, and he discovers that the Labor Party will sweep one hundred seats out of one hundred and twenty. What is that forecast worth for the general election? Zero. Why? Because Kibbutz Ashdot Ya’akov is a kibbutz, and the distribution of votes there is not representative of the distribution in the general population. It’s a certain subgroup whose distribution is completely different. In order to learn from a small group what happens in the whole group, you need to assume that the small group is a representative sample, that the distribution there is roughly like the distribution in the whole population. And that is the whole art of it—otherwise you wouldn’t need polling institutes. Why do you need polling institutes? The only art of polling institutes is exactly this: how to find a group that I can assume is a representative sample. Because just asking five hundred people on the street what they’re going to vote for—that’s not a representative sample, it’s worth nothing. The cleverness is how to identify a group of five hundred people—by the way, such polls usually go with five hundred respondents—to find five hundred people who will constitute a representative sample of the whole population. By the way, the accuracy of polling institutes is amazing. Contrary to the bad image everyone has of them, that is complete nonsense. They have amazing ability to predict results. Amazing. To this day I still can’t grasp how they manage to ask five hundred people—that’s nothing, what are five hundred people compared to, I don’t know, however many millions of voters there are in this country—and they identify them and give you results with astonishing precision. Meaning, they can miss a seat here or there, so what? That’s a very small error, within the sampling error. It’s nothing. Sometimes it can change the result, because if the race is close, then the forecast can give sixty-one seats to one side and fifty-nine to the other, and they were off by one seat, so it comes out sixty-sixty or flips to sixty-one-fifty-nine. But that is still amazing accuracy. It doesn’t matter that it changed the result, because the results lie within the bounds of the error, the sampling error. The art of such a polling institute is to identify five hundred people who form a representative sample. What does that mean? A small subgroup whose internal distribution parallels the distribution in the population as a whole. That is called a representative sample. And all the examples I gave earlier stem from the same mistake: taking a sample that is not representative and assuming that it is representative. That was the mistake. You took a small subgroup and assumed that the distribution in it is like the general distribution—in other words, you assumed it was a representative sample. It’s like assuming that Kibbutz Ashdot Ya’akov represents the distribution of opinions in the country. I don’t know why I’m picking on Ashdot Ya’akov—Afikim, whatever, all right? You need to understand this. These are very simple things when you think about them, but there are so many mistakes in these matters that one has to get used to systematic thinking on these subjects. You don’t have to be an expert in statistics at all in order to think this way. To conduct a poll, you have to be an expert in statistics. But to think about these things, you don’t need any expertise in statistics; you just need to think systematically, define the questions well, define the concepts, define what—what this datum means, and that’s it. Then multiplication and division—there’s no mathematics here beyond fourth grade. The only question is whether you define the question correctly, look correctly at the groups, whether it’s a representative sample or not, and understand the data you have. That’s all. Everything else really is fourth grade.
[Speaker E] Seemingly support for what you’re saying—that the law of possession, that the burden of proof rests on the one seeking to extract property from another, is not statistical—is what the Talmud says: it’s a major rule, and it even goes against the majority. Therefore it’s not statistical, because the Talmud says it also goes against the majority.
[Rabbi Michael Abraham] In the dispute between Rav and Shmuel whether in monetary matters we follow the majority or not. Right. But in Jewish law, they rule like Shmuel, that in monetary matters we do not follow the majority. And even on that there is dispute—it doesn’t matter. There are opinions that this is only not true with very compelling majorities, not every majority; there are opinions that yes, with every majority. In short, in Jewish law it’s not so simple. But at the conceptual level, yes, this is clear. I don’t need proofs for it. Whoever thinks this is a statistical majority is mistaken; it is simply not correct. It’s not one approach versus another approach, but rather a mistaken approach versus a correct one. As an example, let’s look at that same discussion of the rule that the burden of proof rests on the one seeking to extract property from another, okay? I can ask, for example, two questions that are completely parallel to the medical diagnosis or the legal diagnosis that we saw. Assuming that it is mine, what is the probability that it is with me? How many of my objects are with me? Say an overwhelming majority, ninety-five percent. I lent a few here and there, but ninety-five percent of my objects are with me. The question that interests us legally is the reverse question: assuming it is with me, what is the probability that it is mine? Right? Because the judge sees that it is with me—that’s the fact, that’s the datum before him. He has to decide whether it is mine; after all, that’s what the case is about. And if you take the datum that among my objects, the overwhelming majority are with me, you still cannot infer from that that if an object is with me, then it is most likely mine. Those are two completely different questions. Take an example: Reuven is holding all his own objects and Shimon’s objects as well, okay? Shimon deposited all his objects with him and traveled abroad. Now Reuven is holding both his own objects and Shimon’s objects. Now I ask: assuming the object belongs to Reuven, what is the probability that it is with him? One hundred percent—all his objects are with him. Now I ask the reverse question: assuming the object is with him, what is the probability that it is his? One-half—let’s say they have the same number of objects just for simplicity—one-half, right? Because he also has Shimon’s objects; only half the objects with him are his. You see the difference in the direction of the question? There is a difference between asking whether my object is with me, and whether an object that is with me is mine.
[Speaker E] But that was given the assumption that Shimon’s objects are with Reuven.
[Rabbi Michael Abraham] No, I’m only giving an example to show that the answer to one question is not necessarily equal to the answer to the other.
[Speaker E] No, but if you don’t begin with that assumption, how can you say it’s not the same question?
[Rabbi Michael Abraham] But here I can show you the answers easily, so I chose this example. But in any situation it’s not the same question, and the answer will not be the same number. Sometimes maybe it will be closer, sometimes less so; it depends how many objects are not with their owners in general—again, the prevalence of the phenomenon, okay? But it’s not the same question; these are two different questions. And that’s a very important point. It is so easy to get confused by these formulations, and so easy to correct the confusion—you just have to get used to systematic thinking, orderly thinking. I saw on Wikipedia the Bayes formula; it gives several examples there. I don’t think that entry is built all that well, but it gives this example. These examples, by the way, appear in our news five times a day. All kinds of stupid arguments of this statistical sort. At least five times a day. Whoever listens to all the channels and all that will find more. There’s not a single time these arguments aren’t mistaken. Not once. It’s just unbelievable. Okay, so another example. In a certain country there are two health funds—here, I’m reading this. Okay? Seventy-five percent of the residents belong to the larger of the two, three quarters, and the rest to the smaller. The smaller has a quarter of the residents; the larger has three quarters. Satisfaction surveys found that ninety percent of the members of the smaller health fund are satisfied with it, whereas only eighty percent of the members of the larger health fund are satisfied with it. So the smaller fund publishes ads saying: “If you’re satisfied, you’re probably one of ours.” What do you say?
[Speaker F] Not true, because the sample is much larger in the big one, right? Numerically.
[Rabbi Michael Abraham] Let’s think. Suppose there are one million residents. Seven hundred and fifty thousand are in the big fund, two hundred and fifty thousand are in the small one, okay? Now ninety percent of the members of the small fund are satisfied, meaning ninety percent of two hundred and fifty is two hundred and twenty-five, right? So two hundred and twenty-five thousand are satisfied. Eighty percent of the big one are also satisfied; eighty percent of seven hundred and fifty thousand is six hundred, right? Six hundred thousand. So there are six hundred thousand satisfied with the big one, and two hundred and twenty-five thousand satisfied with the small one. So among the satisfied there are a total of eight hundred and twenty-five thousand satisfied people, of whom six hundred thousand belong to the big one. Three quarters of the satisfied belong to the big one, even though the percentage of satisfied members in the big one is smaller. And once again, we are asking the reverse question: if you are satisfied, do you belong to the big one? And not: if you belong to the big one, what is the probability that you are satisfied? Do you understand? This is simply the question of the direction of conditioning in Bayes’ formula. Okay? Simply two different questions. Here I think it’s also very easy to see. In other places it’s a bit trickier, but it’s exactly the same thing. You just have to get used to thinking systematically, that’s all. And there’s never a time people don’t fall into these mistakes. Never. It’s just incredible. Okay, so another example. Now let’s begin talking about majority rule in a religious court. Why am I talking about majority in a court? Because the verse from which we learn the law of following the majority—and statistics supposedly belongs to the concept of majority, we’ll soon examine that, probably already in the next class—but apparently the concept of majority in Jewish law is basically the expression of statistical considerations. From where do we learn that one should follow the majority? There is a verse: “Incline after the many.” That verse speaks about a majority in a court. When the judges are divided, the ruling is determined according to the opinion of the majority of the judges. Okay? That is “incline after the many.” By the way, from that verse they also derive nullification in the majority and “most of it is as all of it,” but that has nothing to do with statistics. So I’m talking only about following the majority. So this verse basically says that we follow the majority, and from here the Sages learn the general principle that we follow the majority everywhere. For example, the principle dealing with butcher shops in a city, right, the most famous example. There are ten stores selling meat in a city; nine of them sell kosher meat and one sells non-kosher meat. I found a piece of meat in the market. Then according to the strict law I can follow the majority and assume that this piece is kosher, because most of the stores in the city sell kosher meat. This is learned from the verse “incline after the many.” The verse about “incline after the many” regarding the court teaches us the idea of following the majority everywhere, not only in… even in a democratic majority—I may talk about that later—but in all contexts where there is a discussion between majority and minority, apparently we learn from this verse that one should follow the majority. On the face of it, this is a statistical decision. You ask yourself: what is the probability that this meat is kosher? Ninety percent. Nine kosher stores, one non-kosher, so ninety percent. On the face of it, that is a statistical consideration, okay? The Talmud in tractate Chullin discusses the question of where we learn that we should follow the majority. Here it is, Chullin 11.
[Speaker D] So Rabbi, why don’t you say that we don’t look at the population of all meat, but only the meat over which there are claims?
[Rabbi Michael Abraham] The Talmud and the halakhic authorities speak about this where the city gates are locked. No meat enters from outside. All right? No,
[Speaker D] But you need to look at all the meat over which there were claims. When there is a claim in a religious court about one piece of meat, whether it is kosher or not, then it’s fifty-fifty. Like with the plaintiff and the lender and borrower.
[Rabbi Michael Abraham] I found a piece of meat. I ask which store it came from. There is a ninety percent chance it came from a kosher store. I have no additional information. Okay, the Talmud in Chullin says this: “From where are these words that the Rabbis said: go after the majority?” Right? From where did the Sages learn the law that one follows the majority? From where? “For it is written: incline after the many.” It is written “incline after the many.” The Talmud asks: “A majority that is before us, such as nine stores and the Sanhedrin, we are not asking about. What we are asking about is a majority that is not before us, such as a minor boy and a minor girl. From where do we know it?” Now this is a very basic distinction, and we’ll dwell on it; I’m only presenting it here because our time is running short. The first two examples we already saw: a majority in court, that is the Sanhedrin, and nine stores. These are two cases that the Talmud calls “a majority that is before us.” What does that mean? It is a majority founded on a distribution that stands before us. We have three judges before us; two say Reuven is liable, one says Reuven is exempt. So that is a majority that is before us, present before us. We have ten stores in the city; nine of them are kosher, one non-kosher. The majority is before us. That is “a majority that is before us.” The Talmud says: “Incline after the many” is a verse that speaks about a majority in court. A majority in court is a majority that is before us. So I can learn from there also regarding stores, because with stores too it is a majority that is before us. But there is another type of majority, called “a majority that is not before us,” such as a minor boy and a minor girl. From where? From where do we learn that there too we follow the majority? Now I’ll define—let me explain the “majority that is not before us.” A majority that is not before us is basically a majority that is the way the world behaves, laws of nature in a certain sense. What do I mean? “A minor boy and a minor girl,” for example, means a minor boy who must perform levirate marriage with a minor girl. Here, this minor girl is married to his brother, his brother died childless, and now the minor boy must perform levirate marriage with the minor girl. So the Talmud says they raise them together: he performs levirate marriage with her, if he is already nine years old and capable of intercourse, then he performs levirate marriage with her and they wait until they grow up, and then they will be fully married. They ask here: but with a minor boy and a minor girl, it could turn out when they grow up that they are infertile in the halakhic sense, unable to have children. And if they cannot have children, then the law of levirate marriage does not apply to them. The whole point of levirate marriage is to establish offspring for his deceased brother. So the law of levirate marriage does not apply to them, and if it does not apply, then he has had relations with his brother’s wife, which is a forbidden sexual relation. So how can you rely on the assumption that they are not infertile? The Talmud says: because there is a majority. Most people in the world are not infertile in that sense. Therefore I rely on that and can let the minor boy perform levirate marriage with the minor girl, and they will grow up together and everything will be fine. So this, says the Talmud, is “a majority that is not before us.” What does that mean? There are not actually ten repositories of people lying before us, and I say, I don’t know, there are nine infertile and one not infertile, and I ask where this person came from. Rather, there is some general knowledge about the world that ninety-five percent of the world are not infertile in this sense. But that is not something—it is not a distribution standing before us. I’ll define this more… more sharply in the coming classes, but for now I’m just explaining the idea. For example, most women give birth after nine months. In the time of the Sages they held that there were two possible birth dates, either after seven months of pregnancy or after nine months of pregnancy, and every baby is born either after seven months or after nine months. Most babies are born after nine months and not after seven. Okay? So that too is a kind of majority. That majority too is a majority that is not before us. Why? Because there is not before us a collection of women such that I can say these give birth at nine, these at seven—this is ninety percent and this ten percent, so there is such-and-such a majority. Rather, what? I know that generally the way the world operates is that women give birth after nine months and not after seven. I’ll formulate it perhaps more sharply.
[Speaker D] So that’s probability, isn’t it?
[Rabbi Michael Abraham] If I were—and in both cases it is apparently probability—but they are still two different types of majority in halakhic jargon, and I’ll try to explain that further later. But I’ll explain it a bit better, because what I just said is not precise. I could take ten women that I know, nine of whom are fertile and one infertile in that halakhic sense. Okay? Now one of them went out of the house where I gathered them, and I ask whether she is infertile or not. Would that be a majority that is before us or not before us? Before us.
[Speaker E] Before us.
[Rabbi Michael Abraham] Why? Because we are dealing with a woman who separated from a given pool that is before me, and I ask whether she belongs to the majority group or the minority group. Okay? But I know she was in that mixture, she separated from it, and I ask from where she separated. Did she separate from the nine or from the one? Most likely she separated from the nine. That is a majority that is before us, like the piece of meat—the question is whether it separated from the nine stores or from the one store selling non-kosher meat. The assumption is that it separated from the nine. By contrast, just an ordinary woman standing before me, and I ask whether she is infertile or not—I have no information about her, and she doesn’t belong to any group that is present before me for which I have some statistics, and I determine that most likely she is not infertile. On what basis do I determine that? She did not come out of some group whose internal distribution is given. What will you tell me? Yes, she came out of the group, the group of all women in the world. But regarding all women in the world I have no idea how the distribution is structured. What I know is a sample, representative or not representative. I know certain women who gave birth; in that sample, most gave birth at nine months and a minority gave birth at seven. But that sample does not include the woman before me; rather it is another sample. I only assume that it is representative, and therefore that is probably the distribution in the whole world. If the woman belonged to that same sample that I know and separated from it, and now I ask whether she belongs to the majority or the minority, that is a majority that is before us. But if I look at a sample, generalize from it to a general rule for the whole population, and now I ask a question about someone from the general population who does not belong to the representative sample, but because I assume that in the whole world it is
[Speaker D] like this, then it’s a majority that is not before us. A majority that is not before us is built on a sample. But why, why don’t you look at all the women in the world? I didn’t understand. Why don’t you look at all the women in the world? Look at all the women in the world and see how many among them are infertile in this sense.
[Rabbi Michael Abraham] If you can manage to do that—to go through all the women, document them, and total up how many turned out infertile and how many didn’t—good luck. Nobody has done that.
[Speaker D] But that’s only a technical problem, not an essential one.
[Rabbi Michael Abraham] A technical problem, yes—but we still don’t have that datum.
[Speaker D] So the difference between “not before us” and “before us” is something technical?
[Rabbi Michael Abraham] Or rather, a question of how I know the majority?
[Speaker D] So it’s just something technical, right?
[Rabbi Michael Abraham] Fine, all right, but that technical thing is very important. How do I know this majority? Is it on the basis of a representative sample? Say, with a majority that is not before us, it is on the basis of a representative sample. There are one hundred women I know, or one hundred births I witnessed, and among them ninety were at nine months and ten were after seven. So that is a well-defined sample. Now if a child comes before me, I ask—from among the children who were born, these hundred—and I ask whether he was born at nine months or at seven, then I know he belongs to the group and I know the distribution within the group. That is a majority that is before us. It has nothing to do with a representative sample; I am not assuming that this sample represents something else. This sample is considered in its own right. That is a majority that is before us. A majority that is not before us is a situation in which there stands before me a representative sample, I generalize from it to the entire population in the world, and then I make a statement about someone who does not belong to the representative sample, but because I assume that the whole world is like this, that is a majority that is not before us. All right, I’ll stop here. Next time I’ll sharpen this further, and then we can continue. In any case, what the Talmud here is basically saying is that from “incline after the many” we learn a majority that is before us, but a majority that is not before us cannot be learned from there, and the question is from where it is learned. That’s it.
[Speaker E] Thank you very much.
[Rabbi Michael Abraham] All right, good night.
[Speaker E] Rabbi, rabbi,
[Speaker B] Rabbi, it comes out perhaps a bit that when you come to evaluate a certain test, the precision of its reliability, then it also matters for what purpose you’re doing it. Meaning, if your purpose is to diagnose a rare disease, or if your purpose is to make sure the person is healthy, that will give you a different result.
[Rabbi Michael Abraham] The reliability of the test is one objective datum.
[Speaker B] I understand, I understand. But when you come to ask what use the test has, that depends on what purpose—what purpose it serves. And I’m saying maybe really, I’m just suggesting, maybe when the Torah commanded the matter of the bitter waters, or the beheaded calf—“our hands did not shed this blood”—its purpose was not mystical, and it wasn’t really trying to verify that the woman did it, or that the woman—or that those people didn’t spill it, that they bear no responsibility for it. Rather, on the contrary, it came to verify that she is not that kind of woman. And when it’s very, very rare—when adulterers were very, very rare—then it turned out that the test, although it was actually worthless, the waters had no mystical element, had enormous success precisely in verifying that the woman had not committed adultery. If you ask what the probability is that this woman really was an adulterous woman from among people who are—yes. But it could be that this is really the Torah’s purpose, to verify—one could ask of it
[Rabbi Michael Abraham] to stand on one foot and release her, since the overwhelming probability is that she did not commit adultery and everything is fine.
[Speaker B] No, but again I’m saying, the Torah—but when it comes to a person, I’m saying if the Torah’s purpose was not mystical but rather to return a woman to her husband and remove suspicions that can always exist, and if the overwhelming majority in the population are absolutely not adulterous, it’s one in a million who are adulterous, then you say, wow, this is a good test,
[Rabbi Michael Abraham] practically speaking it exhausts you and goes through all these exercises.
[Speaker B] But people suspect. So I’m saying, it’s that kind of interpretation.
[Rabbi Michael Abraham] Which could be simple enough, since the women are modest.
[Speaker B] If people suspect. Yes, but the fact is that they suspect. Yes, the fact is that they suspect. It’s ancient.
[Rabbi Michael Abraham] I don’t—it doesn’t sound plausible to me. Fine, okay, all right. Good, Sabbath peace, goodbye, good night.