Failure of representation in Halacha

Asia – 2012

Rabbi Dr. Michael Avraham

The statistical reliability of tests for rare phenomena

outline:

introduction

Initial questions

Failure of representation

Mathematical explanation

Another example: Munchausen syndrome by proxy

Two important caveats

Legal implication: "Something" in addition to the defendant's confession

Two halachic examples

Statistical explanation

Explanation in terms of representational failure

The Essence of Statistical Failure: On Mathematics and Psychology

Practical conclusions

Summary

introduction

Prof. Daniel Kahneman won the Nobel Prize for his contribution (together with Amos Tversky) to understanding various failures in the statistical thinking of humans in general, and of experts in various fields in particular. A significant part of these failures are based on confusion between relative and absolute frequency, or between conditional and absolute probability.[1]One of the most prominent types of these failures (now included in the matriculation exam material in mathematics) is 'failures of judgment based on representations'.[2], and it deals with the reliability of using statistical tools for rare phenomena.

These fallacies appear in many and varied contexts, and it turns out that they also have a place in halacha. It turns out that the Sages were probably aware of this fallacy, and both the first and the last were careful to beware of it. In this article, I would like to explain the fallacy, and present some of its implications and manifestations, and finally, in a nutshell and in a very general way, draw some conclusions concerning medical examinations and the field of evidence law in the legal world.

Initial questions

Reuven was sent by a doctor to undergo some test. The reliability of the test is 99%, meaning that the test results are correct in 99% of the cases. Regarding Reuven, the test results showed that he has the disease in question. As mentioned, there is some chance of an error in the test. What is the chance that Reuven really has the disease?

Ordinary people, including doctors and judges, tend to think that the chance is very high, and some would say 99%, since this is precisely the proven reliability of the test. But it turns out that this is a mistake. The answer to the question of what is the chance that Reuven is sick depends on another question: what is the prevalence of the disease in the public. In a disease whose prevalence in the public is very low relative to the reliability of the test (i.e. significantly less than 1%), the test result is almost meaningless.

This is regarding medical diagnosis. Now a similar question is asked regarding the law of evidence in a court of law. An expert appears before the judges who testifies to some test that is very reliable. For example, a person is suspected of murder, and blood is found at the scene of the murder. A medical test indicates that it is the blood of the accused. What is the chance that the accused is indeed the murderer? Alternatively, in the trial of Solomon, a person appeared before the court whose parentage (who his mother is) the court needs to determine. King Solomon's court sends him for a genetic test, and an expert informs the court that the reliability of the test is 99%. Can the judges rely on it and determine who the parents of the person before them are? In other words: the test indicates a match for mother A and not mother B. What is the chance that he is indeed the son of A?

In both cases, of course, serious halachic questions arise regarding reliance on uncertain evidence, and questions of the need for testimony in the face of other evidence.[3] Here I would like to address the issue from a different angle, in fact from a scientific angle: Is the evidence statistically truly reliable? That is, assuming that the reliability of the genetic test is indeed 99%, and the test shows that Reuben is indeed Jacob's son, what is the chance that he is indeed his son?

Failure of representation

The answer to these questions lies in what is known as the 'basic rate fallacy' (which is one of the representativeness failures). Let's start with the example of a medical test. As mentioned, we have a medical test that is designed to diagnose a certain disease. The reliability of the test is 99%, meaning that in 1% of the cases it is wrong (turns the real situation: healthy into sick, or sick into healthy). Now we ask: Reuven was tested with this test, and found to be sick. What is the chance that he is really sick? The obvious answer is that the chance is 99%, since this is the level of reliability of the test. But as we will now see, this could be a serious mistake.

Let's say there are 10,000,000 residents in a country, of whom 1,000 are actually sick with the disease being tested (i.e., the prevalence of the disease is 1/10,000). Now we will test all residents of the country with this test. The test will be wrong in 1% of the cases, meaning 1% of the healthy will be diagnosed as sick, and 1% of the sick will be diagnosed as healthy. Therefore, the test results will show that there are approximately 100,000 patients. But, as mentioned, the truth is that there are only 1,000 real patients among all residents of the country. So, when a person is diagnosed with a disease in this test, the chance that he is really sick is...1%. It's hard to believe, but a test whose reliability is 99% gives results in such a situation that are 1% reliable.

This is the case for a disease that is low in prevalence in the population. What would happen if it were a more common disease? For example, if the true number of patients in the population is 1,000,000 (and 9,000,000 residents are healthy, meaning the prevalence of the disease is 1/10). Again, let's assume that there are 1% of the healthy who will be mistakenly diagnosed as sick, and 1% of the sick who will be mistakenly diagnosed as healthy. In such a situation, the test results will show us 990,000 (correctly diagnosed patients) + 90,000 (healthy diagnosed as sick). A total of 1,080,000 people will be diagnosed with the disease, most of whom are indeed truly ill. If a person is found to be ill in such a situation, the chance that they are truly ill is 11/12, which is not 99% but is definitely starting to sound reasonable. In such a situation, if the test showed that someone is ill, they should definitely be concerned. The conclusion is that the reliability of a test for a sick person gets closer and closer to its theoretical reliability, the more common the disease is in the population. In other words: reliable tests for rare diseases are not even worth a penny.

The meaning of this conclusion is that in the previous case, if a person was diagnosed as healthy, this result is reliable, because the proportion of healthy people in the population is high. On the other hand, a person who was diagnosed as sick can ignore this diagnosis, because the overall proportion of sick people in the population is low. In other words: this test is intended to diagnose healthy people, not sick people.

How large does the prevalence need to be for the test to be reasonably reliable? The measure is the ratio between the reliability of the test and the prevalence of the disease. If they are of the same order of magnitude, then the results begin to be significant. This means that the test is a kind of microscope that aims to distinguish some object. The magnification (=precision) of the microscope will determine which objects can be distinguished. It is impossible to find a bacterium with a microscope whose resolution is not of the right order of magnitude. This is similar to shooting a mosquito with a cannon, or shooting a tank with a rifle bullet.

An example of another implication. If we want to adopt genetic tests to decide halachically the question 'who is a Jew?', we will have to use tests whose unreliability (the chance of error) is significantly smaller than the prevalence of Jews in the world population (about a quarter of a percent, 1/250). A genetic test with an reliability of 99.5%, for example, will not be sufficient. The same applies to any other medical or legal test.

Mathematical explanation

This surprising phenomenon can be understood through the complete probability formula:

Where P(A/B) is a conditional probability, that is, the chance of A if B is known. In our case there are only two states i, so the meaning of the expressions in the formula is as follows: (P(B₁ is the chance that a person is sick. (P(B2) is the chance that the person is healthy. P(A) is the chance that the test results for someone indicate that they are sick. The formula says that the chance that a test will show that a person is sick consists of two components: the chance that they are sick times the conditional probability of the test showing it correctly, plus the chance that they are healthy times the conditional probability that the test shows that they are sick (i.e. a false result).

As mentioned, the chance that the test is wrong is 11/3, but this only affects one of the components of this sum: P(A/B2)=0.01. In contrast, the second conditional probability is very high: P(A/B1)=0.99. And if the prevalence of the disease is very low, then the chance that the person is not sick (P(B2) is very high, which significantly changes the final result.

Alternatively, the question of the reliability of the test concerns quantities of the type P(A/B) (what are the test results in relation to reality), while the question we are asking (what is the chance that he is really sick given positive test results) is an inverse question that concerns quantities of the type P(B/A). More precisely, the ratio between the two conditional probabilities that interest us is:

P(B1/A) = P(A/ B1) P(B1)/P(A)

The reliability of the test is: P(A/B1)=0/99, and it is indeed high. But the chance that the person is sick assuming the test results are positive is a different magnitude: P(B₁/A), and it can be very low. In our case it is indeed low, since the ratio between the absolute probabilities is low.

Another example: Munchausen syndrome by proxy[4]

Mrs. Sally Clarke is a British woman whose two babies died unexplained in her home (cot death). She was charged in a British court with the murder of her children, and was convicted and sentenced to prison. The conviction was based on expert testimony from Professor Sir Roy Meadow, who claimed that the chance of cot death is: 1/8,500. Therefore, the chance of two children dying is the square of this small number, which comes out to approximately 1/73,000,000. Professor Meadow claimed that there is a medical syndrome called 'Munchausen syndrome by proxy' (some have attributed it to the 'starving mother' in our country)[5], which means that a person sometimes hurts others to get attention for himself. He argued that since the chance of SIDS is so small, it is clear that this is murder based on the above syndrome (in criminal law, it is not enough to prove that there was an act of murder, but proof of the existence of criminal intent is also required).

Without any supporting evidence, based solely on this statistical consideration, the judge found Sally Clark guilty of murdering her children, and sentenced her to prison.

We note that Prof. Madow testified in hundreds of trials, and in many of them the defendants were found guilty and sentenced to various sentences. Some of them without any other supporting evidence (what we called "something" above).

After some time, a witness, an expert in statistics, came forward and testified in court that the conviction was based on a statistical error. His main argument was that it was wrong to multiply the numbers together, since the events could be statistically dependent. Even if the chance of a child dying in a crib is 1/8,500, this does not mean that the chance of two children dying is the square of that number. Since the causes of crib death are unknown, it is reasonable to assume that there are factors in the home, or in the genes in the family, that could have caused this death. And since these are two brothers who grew up in the same home, it is assumed that the cause of their deaths was the same, and therefore the events are dependent on each other.

Let's clarify this with an example. Reuven bought a lottery ticket, and his numbers were lucky. What is the chance that these exact numbers will be lucky? Very small (say 1 in a million). And what is the chance that Reuven will win the lottery? It is also very small (say 1 in a million). Now we ask: What is the chance that these numbers will also be lucky and Reuven will also win the lottery? Ostensibly, this is a multiplication, and the result is 1 in a million millions. But this is a mistake, because Reuven's winning is a result of the fact that his numbers were lucky. The events are interdependent, and therefore it is not correct to see in this coincidence something that is more surprising than each of them separately.

If so, the expert testimony suggests that the chance is not as small as initially thought. But it is still clear that the chance is very small (1/8,500 is also a very small number). According to this, it would seem that any mother whose son died of cot death could be sentenced to prison.

The main problem with the above-mentioned 'expert' testimony was not the independence of the two deaths, but a completely different problem. There was an ignoring of the representativeness error. This statistical test can be considered as a means of diagnosing Munchausen syndrome. The reliability of the test is 8,499/8,500, and therefore it supposedly captures those with the above-mentioned syndrome with very high reliability. The problem is that the prevalence of this syndrome is extremely low. How many women would murder their sons to get attention? Let's assume for the sake of discussion that this is a number like 1/100,000, which also seems like a large estimate in relation to the true prevalence. Now we can immediately see that a test with a reliability that the chance of error is 1/8,500 is worth nothing. The test is very reliable, but its unreliability is much higher than the prevalence of the phenomenon it is trying to capture. The holes in this statistical network are too large.

Two important caveats

A. This analysis is valid only when the test's error is symmetrical in both directions: it may exclude healthy people as sick and sick people as healthy. If this test has errors in only one direction, that is, it may be wrong only for sick people but healthy people will never be diagnosed as sick, then the reliability of the test for sick people is equal to its theoretical reliability. This can be seen by simple calculation, when these data are applied to the previous numbers. There will be no healthy people who will be diagnosed as sick, and therefore the number of people diagnosed as sick is almost the true number.

B. When we have additional indications of a disease, or a crime, the situation is of course different. If the defendant was identified at the scene of the murder at the time of the murder, and in addition the test shows that the blood at the scene is his blood, we can more easily rely on the test. The reason for this is that the number of potential people suspected of the murder is small, and the prevalence of his DNA among them is high (1/10, not 1/10,000,000). In other words, this circumstantial addition de facto changes the relevant prevalence of the phenomenon.

But if there is no additional evidence, the number of suspects is the entire population of the country, or the world, and if we consider the fact that out of all of them there is only one murderer who murdered Shimon, then the prevalence of the phenomenon being measured is very low. In such a situation, a test with a reliability of 99% is not worth much. Additional evidence reduces the total number of suspects and increases the relevant prevalence, and therefore the effectiveness of the test.

Legal implication: "Something" in addition to the defendant's confession[6]

There is a great deal of debate in various legal systems about the status of self-incrimination. When the accused pleads guilty, some consider such a plea to be the "queen of evidence," while others question it. Some question it because of the possibility that it was obtained through improper means (such as violence and threats by investigators). This is the "fruit of the poisoned tree" doctrine.[7] And some doubt it because of the Rambam's words about the insane (which were widely circulated throughout the world, and also in Israel, following the Miranda ruling in the US, which quotes them). Rambam writes in Sanhedrin 58:56:

The decree of Scripture is that a court of law does not put a person to death or punish him for his confession except on the basis of two witnesses. And the one that Joshua killed Achan and David killed the Amalekite alien by their confession was a temporary decree or a royal decree. However, the Sanhedrin does not put a person to death or punish him for his confession, lest his mind be consumed by this matter, lest he be one of those who are hard-hearted, who wait for death to have swords thrust into their stomachs and throw themselves off the roofs, lest this person come and say something he did not do in order to be put to death. And in general, it is a royal decree.[8].

In Israel, confessions have a very strong status, but the law requires that if they are received outside of court, there must be an addition of "something" to the evidence, beyond the defendant's own confession. Thus, we find in the Attorney General's guidelines, guideline No. 4.3012, dated April 2007, section 1:[9]

It has long been a rule of the Supreme Court that a person should not be convicted solely on the basis of his confession given outside the courtroom, even when this confession was received without external pressure, unless "something additional" is found to strengthen that confession (see 3/49 Andlersky v. the Attorney General, P.D. 2 589; see 290/59, So-and-so v. the Attorney General, P.D. 14 1489).

Why is the addition of something so important? I think this can also be explained through the fallacy of representation.

If we treat a confession as evidence with a high probability, since the chance that a person will incriminate himself when he is not guilty is very low. On the other hand, the number of criminals in the world is also low. Therefore, using a confession test to discover the phenomenon of crime may fall into the representativeness fallacy. The relationship between these two probabilities, that is, between the credibility of the confession and the prevalence of crime, may be such that the confession is not effective. The "something" places the defendant in a smaller group of potential suspects, thereby increasing the relevant prevalence of the phenomenon. If the defendant is seen at the scene of the murder, this already increases the prevalence, thus making the confession more effective evidence, as we saw above.

Similarly, in cases where the reliability of a medical test is of the order of magnitude of the prevalence of the disease, then in medical diagnosis the doctor must also consider additional circumstantial evidence, beyond the results of the test. Such additional evidence will increase the relevant prevalence. If the patient in front of him shows symptoms that are typical of the disease – this means that the relevant prevalence has increased, because among those with these symptoms the disease is certainly much more common. In such a situation, a test with the same level of reliability will be much more effective.

Two halachic examples

The halachic discussion I have chosen to demonstrate the fallacy of representation is two issues in which evidence that comes from a majority is attacked by a riota. We will present two such issues here: the issue of yevamot regarding the majority of women giving birth to a child, and the issue of ketubot regarding the majority of virgin women getting married.

A. The issue of childbearing: Most women have nine children

The Mishnah in Tractate Yevamot 32b states:

If the first child is nine years old, if the second child is seven years old, he will be released, and the child will be kosher, and they will be held liable for the dependent sin.

This is someone who preceded and staged his stage immediately after the death of his brother, and then had a son at a time that raises doubts about whether he was nine years old for the first or seven years old for the second. For the sake of the example, let's assume that he staged it two months after the death of his brother.

The Mishnah says that the son is kosher from the first, because if he is the first, then although the yibbum is not valid and they committed the offense of a brother's wife not in place of a mitzvah, the child is certainly kosher. If he is seven years older than the second, then the yibbum is valid and he is the second's kosher son. So with regard to the son, there is no doubt that he is kosher. However, with regard to them, they bring a contingent guilt due to doubt in the cause of a brother's wife not in place of a mitzvah.

The Gemara there, 37a, makes this difficult:

A doubt of nine years, etc. Rabbi Lia said to Rabbi Nachman, "He followed most women, and most women have nine children!"

The Gemara asks why the child is considered doubtful, since there are many who give birth to nine children, and therefore it must be decided that he is the son of the first, and therefore the obligation to offer the sacrifice should be for a certain sin and not for a dependent guilt.

In conclusion, the Gemara explains this as follows:

…Al, the most Kamina: Most women give birth to nine, and a minority – to seven, and every woman who gives birth to nine – passes a third of her days, and since this one was not recognized as passing a third of her days – it was considered excessive.

At this point, the Gemara suggested that all women who give birth to twins have a visible fetus. And here, we are talking about a fetus that is not visible, otherwise no doubt would arise here. And since women who give birth to twins have a visible fetus, the majority were found to give birth to twins.

Now the Gemara asks:

Not every woman who gives birth to nine - her pregnancy is visible for a third of her days, but if it is not recognized for a third of her days - she is certainly pregnant with seven children, that is, a baby!

If indeed all women who give birth to T. have a visible fetus, then the law should not have been that they are liable for dependent guilt, but rather that they are exempt, because there is certainty that he was born to Z.

Finally, the Gemara corrects that it is not a certainty that the fetus is visible in the woman giving birth to T, but rather that it is a majority:

But a mother: Most of the women who give birth to nine – her fetus is visible for a third of her days, and the island of Madla is not recognized for a third of her days – it is too late for her.

Therefore, in conclusion, the Gemara explains that the majority of women who give birth on the ninth day of the month are considered to be pregnant because among those who give birth on the ninth day of the month, the majority are clearly pregnant. Therefore, the majority of women who give birth on the ninth day of the month are considered to be pregnant, and this is a situation of doubt, and therefore they are liable for dependent guilt.

B. Address issue: Most virgins get married

There are other examples throughout the Shas and in the Rishonim and Acharim in which some consideration neutralizes evidence that came from a majority, and the majority prevailed. Here we will present the example of Ketubot 16 Sua'a, since the course of the Gemara there is exactly the same as the course we saw in the issue of Yevamot. There too, against the initial majority, another majority is cited that neutralizes it.

The Gemara deals with the question of whether the woman before us was married as a virgin or not. The assumption is that no word has reached us that she was married as a virgin. On the other hand, there are a majority of married women who are virgins. Regarding this, the Gemara says there:

Rabina said, because it is said here: Most virgin women marry, and few widows, and every virgin who marries has a voice, and since this one does not have a voice, it is more likely to be rejected.

There is a majority who marry virgins. And the majority is upset because all virgins who marry have a voice.

Now the Gemara argues that if the rule that virgins who marry have a voice is a certainty and not a majority, then not only is the majority wrong, but there is an opposite explanation:

Does not every virgin bear a child have a voice, because with him are witnesses, who is he? You are false witnesses, Nina!

And in conclusion, the Gemara explains that this is a majority and not a certainty:

Rather, Rabina said: Most of the time, a virgin has a voice, and since this one does not have a voice, it is more likely to be rejected.

As mentioned, the move is very similar to the issue of Yevamot.

Statistical explanation

In both issues, it is not clear why the final explanation answers the difficulty. The difficulty was why, after the reservation, a situation was not created in which there was certainty for the other side. The Gemara answered that such certainty would have been created if the reservation were a sweeping rule (all women who give birth to a third child have a visible fetus, or all virgins who get married have a voice), but since this is a majority and not an absolute rule, no reverse certainty is created.

But even if the reservation is only in the realm of an apparent majority, clarification is created for the other side. For example, in the issue of yevamot, it can be concluded that if indeed the majority of women giving birth to nine children have a visible fetus, even if it is a majority and not certainty, then if the fetus here is not visible (and we have seen that this is the case) then there is a majority in the opposite direction, that this fetus was not born to T. If so, why don't we decide here that this fetus was born to Z, and exempt them from the sacrifice altogether? And so with regard to the issue of ketubah, there too we can ask if there is a majority of married virgins who have a voice, then someone who married without a voice is probably not a virgin. So why is it doubtful and uncertain in the opposite direction?

The answer to this is quite simple, and we will see it through the second example. There is a majority of women who get married who are virgins. Therefore, in general, if we ask whether the woman before us was married in a oleh or a betula – the answer will be a virgin. On the other hand, if no vote is cast on her, then there is a majority against her, since the majority of those who get married as virgins are cast on them. Let’s say there are 1000 women in the world who get married. Of these, 80% are virgins, meaning there are 800 virgins, and 200 betula. On the other hand, of the virgins, there is a majority of 80% who get married, meaning there are 640 virgins who get married, and therefore 160 virgins who get married but don’t get married. Now we have a woman who gets married without a vote, and we are debating whether she is a virgin or a betula. To decide, we compare the number of those who get married in oleh (200) to the number of virgins without a vote (160). The decision is clear: she is in oleh. The second majority neutralized the first majority. This is the mechanism of a majority being defeated by a counter-majority.

As an anecdote, I will mention that in Rabbi Shmuel Rozovsky's lessons on Tractate Yevamot, Si' Shatz, he also addressed this difficulty, and he formulated it this way:

And here, in simple terms, the second majority is in the same ratio as the first majority, and through the example of if the first majority is the number of women who give birth to nine, it is in the ratio of four to nine, and for example, out of a hundred women who give birth, eighty to nine, then the second majority is the number of women who give birth to nine, their fetus is visible, and it is less in the ratio of four to nine, that is, out of eighty, they give birth to nine, and out of seven, it is found that out of ten women, there are twenty women who give birth to three, and another sixteen women whose fetus is not visible, and so on. So, this woman is certainly on the one hand a minority and is not among the other women who give birth to three and their fetus is visible, and since there are mothers who provide for her, where does she come from? Then there is a majority of women who give birth to three, since women who give birth to three are fewer than women who give birth to three and their fetus is not visible?

And it is necessary to say that Demeiri here specifically in 23 that the second majority is not in the same ratio as the first majority, but out of the 100 women who give birth to nine, there are 20 women whose fetuses are not noticeable, and those who give birth to seven are not more than those who give birth to three and whose fetuses are not noticeable. Simply, Demeiri does not mean in 23 that the ratio between the minority and the majority in the first and second trimesters is the same as the ratio between the minority and the majority in the first and third trimesters.

If the two guns are of the same power, an opposing majority is created here, as we saw above. If so, the Rashar asks, why do we treat this as a situation of doubt (that the majority was wrong) and not as a certain situation? Ostensibly, there is a decision here in the opposite direction and not a situation of doubt.

Obviously, the answer to this question depends on the ratio between the two numbers. If the majority of virgins who marry with a voice were less significant, for example 60%, then the number of virgins who married without a voice would only be 320. Compared to the number of those who marry in the ol's, which is 200. So, now when a woman who married without a voice comes before us, the decision will still be that she is a virgin. Here the majority will not be affected. With a majority of 75% of virgins who marry with a voice, the two sizes will be equal, and we will be left in a state of doubt.

Therefore, to the essence of the question posed by the Rashar, it must be settled that the halakhah categorically states that when a majority is opposed, we do not take it into account. We do not have the possibility of conducting a statistical survey in every case that comes before the rabbinical court to know the ratio of the majorities, and therefore the assumption is that the situation is equal and there is no way to decide based on majority considerations. From a halakhic perspective, this is a situation of doubt.

It is true that if we conduct a specific survey for a particular case and find out what the rate of each of these guns is, we can decide the question based on the considerations outlined above. But as long as we have not conducted such a survey, the decision is that it is doubtful.

Explanation in terms of representational failure

As mentioned, the reliability of the majority we use depends on the strength of the opposing majority. The result depends on the ratio between the two conflicting guns. This situation can be seen as a parallel situation to those I described at the beginning of the article, and the Rashar problem reflects the failure of representativeness. The ratio between the two guns reflects the prevalence versus the reliability of the test.

We actually want to test the question of whether the woman in front of us is a virgin or a virgin. The prevalence of the phenomenon (virginity) in the population is 80%. What test do we use? A test through the exit of the voice (if there is a voice – she is a virgin, and if not – then she is a virgin). What is the reliability of the test? This is determined by the opposing majority. If the majority of virgins have a voice, this means that the test is very reliable. Its reliability is equal to the strength of the opposing majority. If it is 80% of the virgins who have a voice, then the reliability of the test is 80%. Now we can see that this test is not successful, because the prevalence of the phenomenon it is intended to test is similar to the reliability of the test.

If the voice was really present in a large majority of virgin pregnancies, meaning the test was much more reliable, it could also be relied upon to test a phenomenon that is not so common. It all depends on the ratio between the guns (the reliability of the test versus the prevalence of the phenomenon). Once again, we discover that the resolution of our microscope should be adapted to the size of the observed phenomenon.

We have seen that the halakhah does not refer to the detailed calculation, that is, to the ratio between the majorities, since in different cases the results are different. The halakhic solution is that when there is a failure in the representation of the majority, it is wrong and cannot be relied upon, and the situation is halakhically defined as doubtful.

This is further support for the doubts raised by poskim regarding DNA or other tests in legal-halakhic contexts. The problem is not only purely halakhic but also scientific-statistical. It is true that if we have data on prevalence and reliability, we may be able to reach a different concrete conclusion in this particular case, but it makes sense to establish a blanket and uniform rule for the sake of coherence and simplicity of the halakhic law.

I will note that awareness of this failure may explain several other difficult points in the early ones on the issues in the inscriptions and on the stage, but I will not go into that here.

The Essence of Statistical Failure: On Mathematics and Psychology

Many may feel that addressing oral and written issues in terms of representational failure is unnecessary. The conclusion of the issues is clear even without the use of statistical tools. On the other hand, in the two contexts described at the beginning of my remarks (legal and medical), which are completely equivalent to the Talmudic situations, it seems that many may fail. The example of Munchausen syndrome, which sent many women to prison for no wrongdoing, is a good example of this.

The wonders of our psychological ways, and it is not clear why in some cases we do not fall into this fallacy and in others we do. In some cases it is easy for us to see the answer, and in other cases it is very easy for us to be wrong. It is important to understand that statistical fallacy is a psychological phenomenon, not a statistical one. The structure of our thinking causes us to fail, and it is difficult to know exactly when this happens and when it does not. The Nobel Prize was awarded to Kahneman in the field of economics, not in the field of mathematics. The mathematical discovery is not so impressive, but the psychological discovery (to what extent people, including experts, are liable to fail in their thinking) is the crux of the matter. Both Kahneman and his research partner Amos Tversky are psychologists by training.

As an anecdote, I will add here that Pascal's well-known wager (in favor of observing religious commandments) also suffers from a similar flaw.[10]Pascal was one of the founding fathers of probability, yet he fell into this fallacy. It turns out that even probability experts have a psychology.

Therefore, the fact that the Talmudic examples seem simpler to us should not mislead us. We are prone to falling into this trap, and sometimes the results are disastrous: medically, legally, or humanly.

Practical conclusions

Many are unaware that judges, and even good senior doctors, may err in their statistical judgments. Sometimes we are advised to take a test for a certain disease, when the reliability of the test is not higher than the prevalence of the disease. In such a situation, there is no point in taking the test. Although if it shows that the person is healthy, then most likely he is indeed healthy, but if he is diagnosed with a disease, he should not trust the results of this test (although see the two reservations mentioned above).

As I explained, these things only apply where the possibilities of error are symmetrical, that is, when the error in the test is such that both a patient can come out healthy and a healthy person can come out sick. On the other hand, when the error is one-way, meaning that only patients can come out healthy, but not vice versa, there is no reason to do such tests and consider their results.

In these cases, it is advisable to ask the doctors (and make sure they answer with knowledge) whether the chance of error is one-way or two-way, and whether there is verified information on the prevalence of the disease. It is also advisable to ask how they arrived at the data on the prevalence of the disease, since these data may also be based on incorrect statistical considerations of this type (i.e. on the results of such tests). These things are of course also relevant to the doctors themselves (see the case of Munchausen syndrome).

As stated, the same applies to the law of evidence in the legal/halakhic context. In these contexts, various pieces of evidence are presented before the courts/courts, such as the results of genetic tests, etc. For example, let's think about a person who has medical insurance with a certain insurance company. He is now tested and diagnosed with a disease that is included in the insurance coverage. The case comes before the judge/judge, and he must decide whether the insurance company should pay the insured or not. Generally, if a medical expert comes forward and says that this test is reliable in 99%, the judge/judge will accept his expert's testimony and hold the insurance company liable. However, as stated, when it comes to a disease with a low prevalence (i.e., in the order of magnitude of the reliability of the test) there is no real basis for this. This is an incorrect legal decision (after all, the one who issues the test bears the burden of proof).

The same applies to tests prior to a match. A person is tested and found to be ill with some disease, and now the question arises whether to cancel the match, or perhaps even cancel the child marriage due to a mistake.[11]Here too, statistical judgment (and Torah-Halakhic judgment, of course) must be made with great caution.

When considering the results of a statistical test, it is important to be aware of this (and other) fallacy, as such an error can have disastrous consequences. Experts, especially medical or legal experts, should not be automatically trusted when the problem is a statistical one.

And most importantly, in both the legal and medical contexts, it is recommended to use other independent evidence in addition to statistical considerations. The decision to put a woman in prison, or to cancel a match, or to charge a fee, based on such statistical considerations may turn out to be a serious error.

Summary

The failure of representativeness can be very confusing when considering the weight and reliability of statistical tests for rare phenomena. It has many implications. We have seen some implications regarding medical tests for diseases whose prevalence is of the order of magnitude of the reliability of the test (only if the unreliability is two-way), in most cases they have no meaning, and there is no point in performing them, unless there is assistance from other directions. Also in the field of law, in the law of evidence, it is important to take the failure of representativeness into account and also to rely on direct and independent evidence.

From my investigation, today doctors are taught the field of statistical errors as part of their professional education. For lawyers and judges, this is certainly not part of their professional training. The need for this is perhaps the most important conclusion to be drawn from the picture presented here. As I have shown, this is nothing more than an additional study and conceptualization of issues that are taught in some way or another as part of the training for judges (written and staged issues), but it is important to point out the more general meaning of a majority that has been overturned, and the additional conclusions that arise from it. This will better prepare the judges for their role when they come to consider evidence that is presented to them.

 

1. 1. In this regard, it is recommended to see the book by Varda Lieberman and Amos Tversky, Critical Thinking: Statistical Considerations and Intuitive Judgment, The Open University 1996.

      It can also be seen in the article by Gerd Gigrenzer and his colleagues:

'Helping Doctors and Patients Make Sense of Health Statistics', Gerd Gigerenzer, Wolfgang Gaissmaier, Elke Kurz-Milcke, Lisa M. Schwartz, and Steven Woloshin, PSYCHOLOGICAL SCIENCE IN THE PUBLIC INTEREST, Vol.8 No. 2, pp. 53-96.

      The article is also available online at:

 http://www.psychologicalscience.org/journals/pspi/pspi_8_2_article.pdf .

For a Hebrew article containing some of the material, see: Gil Greengross, 'Medical Statistics – How to Understand Medical Information Well?', on the website Homo sapiens.

2. 2. See Liberman and Tversky's book, Chapter 6, and the two articles mentioned in the previous note.
3. 3. See Rabbi Wesner's response to the Border Police Rabbi, "Halakhic Identification Based on DNA Testing," Areas 21 (5761) 121. Also, see in detail, A. Westreich, "Medicine and Natural Sciences in the Rulings of the Rabbinical Courts", Sentences 20 (5756) pp. 425-492. See also: D. Primer, "Determining Paternity by Testing Blood Types [in the A, B, O System] in Israeli and Jewish Law", Asia 5 (5786), p. 185; M. Halperin, H. Brautber, D. Nelken, "Determination of Paternity by the Central Tissue Coordination System", Areas 4 (5783), p. 431.
4. 4. See report and analysis in Tal Galili's article, 'How (non)statistical thinking sends a woman to prison – the story of Sally Clark', on the website The Hitchhiker's Guide to Statistics:

 http://www.biostatistics.co.il/?p=20

5. 5. For more information on this controversial syndrome, see its entry on Wikipedia. Also, see a short article by Prof. Zvi Zamishlani, 'What is Munchausen syndrome anyway?', on the website Makoto,

 http://www.mako.co.il/news-columns/Article-e06d4aab8d97221004.htm .

      Also, see Professor Esther Herzog's article, 'The Syndrome That Never Was', in the 'Opinions' section of the website Ynet, dated 19.7.2009.

      Regarding the sequence of events in the case of Sally Clark, see the website established in her honor: http://www.sallyclark.org.uk/.

6. 6. See Wikipedia on 'Confession'. Regarding confession in Jewish law, see Michael Vigoda's article, "Confession in Jewish Law," on the 'Da'at' website:

      http://www.daat.ac.il/mishpat-ivri/havat/48-2.htm, and in the sources cited there.

7. 7. See A. Kirshenbaum, "Self-Conviction in Jewish Law," Jerusalem 1976, p. 523, and its environs. Kirshenbaum recommends adopting this doctrine in Jewish law as well.
8. 8. Indeed, compare his words in the Book of Revelation, 22:2, and the words are ancient.

[9] See also Judge Dalia Dorner's article, "The Queen of Evidence v. Tarek Nojidat – On the Danger of False Confessions and How to Deal with It," The defense attorney 95, February 2005, and in the sources cited there. And on the web.

9. 9. See about this in my book God plays dice, Yedioth-Sefrim, Tel Aviv 2011, pp. 104-112. There I explained the fallacy in a slightly different way: the expectancy criterion is not effective for making decisions if the chance of getting the expectancy is low. This can also be seen in another way as a type of representational fallacy, and so on.
10. Regarding the cancellation of a kiddushin on the grounds of a mistaken purchase of defects and illness, see Babylonian inscriptions 37b, 6.Rambam, Ishut, 25:2, andToshu'a Abba Zechariah 17:4. And also regarding defects in a person, see our Rabbi Simcha of Shapira, whose words were quoted in the Responsa of Maharam Rothenberg (Krimona) 6:10, and inArm light Chai 30th 576; Responsorial Psalm Hot Yair C. Rakha; B.S. C. Kand S.K.B.; Innovations Beit Halevi C. C.; Herzog, Rulings and writings, Volume 7, Haba'zah 65; Responsorial Psalm Father's house Satisfied part Abraham's help On the authority of Abba Zechariah 27; Harshah Auerbach, his words were quoted inThe soul of Abraham Haba'za Si' Lat Ska'a.. and see more in the Responsa Ein Yitzhak Chavhez Cha"a Si' 24. And also the Responsorial Psalm Cha"a Si' 1; B.A. Abba Az there; Beit Meir Name of the SCA; Responsorial Psalm Jacob's Return Cha Si Ka; Responsorial Psalm Shlomo's sheets H.A. C. H.; Interpretations of Ibra pp. 11 ff.; Hazo"a Abba Zechariah 67:33; Responsorial Psalm Minchat Yitzhak 17:28. And see also the article by Rabbi D. Bass, Areas, 24, 5764, pp. 194 ff.