Another Look at Expected Value as a Criterion for Decision-Making (Column 255)
With God’s help
In column 252 I discussed Parfit’s mistake in proposing expected value as a criterion for calculating the utility of a step within a framework of collective action. I explained there that there are situations in which expected value is not the relevant criterion for decision-making. When the standard deviation is large (the outcomes are very widely dispersed), or when the chance of obtaining the average is negligible, there is no point in evaluating the value of the game (the likely payoff) by its expected value. In such cases we should assess the likely payoff in the single drawing, not the expected value. Following that column, I received a perplexing probability question, one that has been occupying and confusing a few people all day, and I therefore decided to write this column.
The Relation Between the Likely Payoff and the Expected Value
For brevity, I will illustrate the basic claim about the difference between the likely payoff and the expected value by means of the simple example I gave in that column. Suppose we toss a biased coin. The chance of getting heads is one in a million, but if the result is heads I win a quadrillion dollars. If the result is tails (whose probability is very high, almost 1), I lose one thousand dollars. The expected value of the payoff in such a drawing is 109 – 1000 dollars (the payoff multiplied by the probability, for the two outcomes). The thousand is completely negligible, and so one may say that the expected value is 109 dollars. What is expected value? It is the average profit per drawing if we repeat this drawing infinitely many times. That is, if we repeat the drawing again and again we will get many outcomes (profits). If we add them all up and divide by the number of drawings, then as the number of drawings increases, the average profit per drawing (note well: for one, not for all of them together) will approach the sum of 109 dollars.[1]
You are now offered the chance to play this game, that is, to buy a ticket for a one-time drawing. How much would you be willing to pay for that ticket? Notice that here you are not making infinitely many attempts, but only one. There will be only one outcome, one way or the other. And yet it is commonly thought (and this is what Parfit assumed in the above-mentioned column as well) that the expected value more or less expresses the value of participating in the drawing. Thus, for example, in the case of a fair coin it would be reasonable to pay a fortune to participate in such a drawing, because the expected value of the profit is enormous. As stated, even in the drawing that uses a biased coin as I described, the expected value is not materially different (it is still enormous). Therefore, seemingly, we would expect people here too to pay an enormous sum.
But that is not correct. A reasonable person will not be willing to pay anything for the drawing with the biased coin. Why? Think about the outcomes of such a drawing. It is almost certain that the result will be that I lose one thousand dollars. The chance that I will win the fortune I described in a single drawing is utterly negligible. If that happens I will of course come out immensely rich, but there is no realistic chance of it. Therefore it is not reasonable to invest anything in such a drawing. If someone really likes risk, there may be room to invest some amount that is very small relative to his means, but a reasonable person will not enter this game at any price. One may say that the reasonable entry threshold for this game is that I should be paid (!) about one thousand dollars for participating (that would cover the near-certain loss awaiting me).
The basis of the difference is that in a drawing with a fair coin, the expected value of the profit is also roughly the likely payoff (or: fairly likely). But in the game I described (with the biased coin), the expected value of the profit (= the average profit per drawing) is not the likely payoff. The likely payoff is determined by the distribution of the outcomes for receiving different sums (more precisely, by the part of the distribution that describes the outcomes expected with high probability), and here the chance that I will lose one thousand dollars is 1 (minus something utterly negligible). So I am facing a certain loss here. Remember that I play only once, and on that one occasion I will certainly lose one thousand dollars. So why enter the drawing? Clearly, if I were offered the opportunity to continue for infinitely many drawings, it would be worthwhile even at a price of a billion dollars per drawing (for someone with "deep pockets" who can hold out until he wins, of course), because there what matters is the expected value.[2] But in one drawing, or in a small number of drawings (here even a million drawings is a very small number), what matters is the likely payoff, and it is much lower.
What is the likely payoff? Unlike expected value, which is well defined mathematically and for which there is a systematic way to calculate it, the likely payoff is not a mathematical quantity (because there are many possibilities as to what will come out in the single drawing), and therefore it depends on the player’s situation, his appetite for risk, and the distribution of the possible outcomes for each drawing. Each person will decide what the likely payoff is from his own perspective, but it is reasonable to place it around the order of minus one thousand dollars (that is the payoff expected in this drawing with probability almost 1).
The Card Lottery
The question I was asked is the following: suppose you are offered the following lottery game. In front of you is a deck of a million cards numbered consecutively. You choose one of them at random, and win an amount equal to the number written on the card. For example, if card 123,385 comes up, then you win 123,385 ₪. Now you are offered the same drawing on a different deck: each card pays one shekel more than its number, and the last one pays one shekel (instead of one million). The expected value of the winnings in the two decks is exactly the same, of course. What is it? We calculate the average by multiplying each winning amount by the probability of winning it and summing over all possibilities. The probability of winning any amount is one in a million (that is the probability that any particular card will come up in the drawing). So in effect we have the sum of all the amounts from one to one million, divided by one million. The result is 500,000.5 ₪. As noted, the expected value is the average profit per drawing if one makes infinitely many drawings. But what is the likely payoff? Here we get a different result in the two cases. For every card except the last, the payoff in the second drawing is one shekel higher than in the first. Only for the last card does the payoff drop dramatically from one million to one. But the chance of drawing the last card is negligible (one in a million), and therefore it is clear that one of the other cards will come up. If so, the likely payoff for a single drawing (or for a sufficiently small number of drawings) is one shekel higher in the second deck. Therefore it is certainly preferable to participate in the second drawing, since we will earn one shekel more with probability 1, despite the utterly negligible chance that we will lose a great deal, but that should not be taken into account. It simply will not happen.
Think about the risk we take when we get into a car for a nonessential trip. If we die, it is certainly not worth it, and yet we drive. The chance that we will die is negligible (it simply will not happen), and therefore even for a gain that is not very significant we take that risk. Contrary to the familiar and mistaken saying, "It won’t happen to me" is a simple probabilistic truth and entirely rational.[3]
The Difficulty
Now comes the difficulty. If you look at the two decks of cards I described, you will see that in fact they are two completely identical decks. In the first deck, card number 1 pays 1, card 2 pays 2, and so on. In the second deck, card number 1 pays 2, card number 2 pays 3, and so on up to card number 999,999 (one million minus one), which pays one million, while card number one million pays 1. So let us call card number one million in the second deck card 1, and shift the names of all the other cards forward by one. We have obtained exactly the first deck. So how can the same drawing on the same deck yield a different likely payoff? In other words, in each of the two decks there is one card that pays each amount between 1 and one million, and in both drawings one chooses one of the cards from the deck at random. So how can the result be different?
Notice that the expected value of the winnings (the profit per drawing when one makes infinitely many drawings) is of course identical in both cases, but as we saw above the likely payoff is certainly not identical. The same drawing in the second deck
will yield a profit one shekel higher. As long as the number of drawings is small (in particular, in a single drawing), it is better to enter it. How can this be?
The person who asked me this question told me that he had even run an experiment with a random number generator (software that chooses a number at random) between 1 and one million, and indeed found that the drawing on the second deck produced a higher profit (so long as he performed only a not-large number of drawings). That is, the likely payoff in the drawing on the second deck differs from the likely payoff in the drawing on the first deck. How can that be when it is exactly the same situation? Do the numbers on the cards change anything? They are just names. Why does it matter that in the second deck it is the millionth card that gives me the payoff of one, while in the first deck the first card gives it? After all, the drawing is random.
Preliminary Analysis
When the questioner asked me and reported his results, I told him that I had no doubt that something in his drawing procedure was not entirely random. In some way he was preserving a dependence on the identity of the cards (that is, on the serial number of the card as defined in the first deck). Quite quickly I realized that he was apparently conducting the drawing as follows: he generates a number at random (by means of the random-generator program), and then asks the person opposite him which deck he would choose. Choosing the second deck always yielded a higher profit, because the payoff from a card in the second deck is always one shekel greater than from the same card in the first deck (except when the last card, number one million, comes up in the drawing, but as stated, that does not happen).
But this is of course a rigged game. After all, once some card n has come up (which the chooser does not see), then in every case except one, choosing the second deck will yield one shekel more. So it is obvious that he should choose the second deck. We are dealing here with a single drawing (or a small number of drawings), and therefore the criterion is the likely payoff, not the expected value, and the likely payoff is one shekel greater than with the first deck.
What Should He Have Done?
I told him that if he wants to compare the two drawings, he should conduct them differently. He should place two decks before the participant, conduct one drawing and choose a card from the first deck, and then conduct another, separate drawing to choose a card from the second. He should then ask the participant which of the two outcomes he chooses: the profit from the first drawing or the second? Here there is no difference at all between the two drawings (for in both cases you choose, completely at random, one card from two identical decks), and therefore there is no doubt that even a random-generator program will show no difference between them.
Explanation
The conclusion is that the question whether the second drawing is preferable to the first depends on the way the drawing is conducted. If you draw lots for the serial number of a card and then choose a deck, it is clear that the second drawing is preferable. But if you conduct two independent drawings, it is clear that neither has any advantage over the other.
The difference between the two cases is that in the first case you preserve the serial number of the cards (that is, the identity of each card as defined in the original deck). This is not a random and independent drawing between the two cases. Therefore it is not correct to compare the results. You are drawing the very same card with the same identity (the same serial number), and therefore from the definition of the process it is clear that the payoff is greater for the card from the second deck. Despite the appearance that this is a random drawing, the correlation (dependence) between the order of the cards in the decks makes the comparison problematic.
Notice that what I said here remains true even if we thoroughly shuffle the cards in each deck before the drawing. That will change nothing, because a number is written on every card, and when you randomly choose a card from the first deck, you are not choosing its place in the shuffled deck (you chose the number written on it, not its position in the shuffled deck). In effect, you chose a number between 1 and one million at random, and now you ask what the preferable payoff is from that card if we refer to the first deck or the second (we are not conducting another drawing on the second). The result is that the amount in the second is larger, just as above. The position of the card you chose in the deck does not matter at all. What matters is its identity (the number written on it). Therefore the fact that I chose it at random is of no significance whatsoever when comparing the payoffs in the two decks.
Conclusions
First, this example reinforces the claim that the likely payoff in a small number of drawings (or in a single drawing) does not necessarily depend on the expected value of the payoff. The criterion for a small number of drawings is the likely payoff, not the expected value. When the probability of obtaining the outcome represented by the expected value is reasonable, there is a connection between the two.
Beyond that, one can also see here just how confusing probability and statistics are as a field. Translating the statistical calculation into a real-world case (or into a drawing that is actually carried out) is delicate and depends on various parameters. What seems random to us turns out not to be entirely so. There are, of course, many other confusing elements in probability and statistics, and it is no wonder that at the Hebrew University’s Center for Rationality they make their main living from this.
[1] If the amounts accumulate, we enter the question of the random walk, but I will not go into that here.
[2] Not entirely precise. Because there are losses, and what we really have here is a random walk of cumulative profits, the calculation is more complicated. See Nadav Shnerb’s comment on the above-mentioned column.
[3] Admittedly, in collective situations the consideration of "no, it won’t happen" is not correct (at least morally). For example, going into shelters when missiles are being fired. The chance that I will be hit is utterly negligible, and therefore there is no logical reason to go into a shelter. But if everyone makes that calculation, someone will be hit. Therefore here the categorical imperative tells each of us to go into the shelter anyway (assuming that every person who is hit is a blow to society as a whole, in terms of morale and strategy vis-à-vis the enemy).
Discussion
As R. Moshe Shapiro said in a lecture (given on the 5th of Iyar): the latest current event that interests me is the Exodus from Egypt 🙂
In all calculations of "expected profit," one has to pay attention to another point: the true value of money is not directly proportional to the amount of money. One hundred billion shekels are not really worth 1000 times as much as one hundred million shekels. For an ordinary person, it is almost the same amount, and certainly the first sum does not justify reducing the chances of winning by a factor of 1000 compared to the second sum. Not even by a factor of 2. This is the simple explanation for the St. Petersburg paradox: from a certain amount onward, doubling the prize no longer has any meaning.
Another example: suppose a fair coin is tossed. If it comes up heads, I lose 100,000 shekels. If it comes up tails, I gain 120,000 shekels. Even though the expected profit is positive, and the chance of winning is high, for a person who does not have much money this is not worthwhile, because the risk is too great. For most people, the harm from losing 100,000 is much greater than the benefit of gaining 120,000. Economists see the phenomenon of "loss aversion" as an example of irrational behavior, but that is because they look only at the numbers and do not understand that the real benefit or harm is not proportional to the numbers.
Every single day people take out insurance on all sorts of strange things. Yet the expected profit from buying insurance is zero. According to your approach, buying insurance is an irrational act—although I seem to recall that you too insure your car. But when one takes into account the utility weighted for each shekel, one can see that the calculation becomes rational (it hurts more to lose a car than to lose a small monthly insurance premium). In the same way, it also hurts more to lose 1,000 dollars in a lottery than to forgo a potential profit of a billion dollars (because money has diminishing marginal value the more of it I have).
Regarding the first experiment with the unbiased coin, there is a very rich literature on this (as I wrote in the comment on St. Petersburg). The claim is that in such cases one should maximize not the amount won but the log of the amount won, and if so then in the present case (a one-in-a-million chance of winning 15^10 dollars) one should be willing to pay 0.00003 dollars to participate in the lottery, a sum that seems perfectly reasonable to me.
A comment to Oren: the fact is that people both buy insurance (they are willing to pay more for freedom from risk than the expected value of the loss—it has to be so, otherwise insurance companies would go bankrupt) and buy lottery tickets (they are willing to pay more for a chance than the expected value of the profit. Again, it has to be so, otherwise the national lottery would go bankrupt). So apparently there is a strong irrational element in us.
[One might note from Makkot 3a that the claim-right of his doubt and the claim-right of her doubt do not add up to the amount of the ketubah, even though anyone who would buy each of their rights would certainly receive the ketubah (unless we say that this is because of the value of waiting for the money).]
That is not an explanation of the paradox, because the value of the lottery stops rising long before you get to an amount that is irrelevant for us. In your second comment as well, you are speaking about the utility function, which is a personal matter depending on the circumstances. That is obvious. My claim is that in any case, the value is not determined by the expectation.
Not true. Buying insurance is a rational step under certain circumstances. Each person according to his own aversion to losses and risks. My policy is that with insurance one should in fact go according to the expectation (that is, not buy insurance), except where you cannot handle it. I am not moved by "pains" as you described. In my view, a rational person should not take them into account but should overcome them. I am moved by situations in which I would get into objective trouble (as distinct from "pains").
I commented in the note about ignoring the cumulative effect and the issue of the random walk. But I did not understand what "one should maximize" means. Should, in order that what? Each person decides what he wants and how much he is willing to invest. After all, these considerations depend on circumstances, inclinations, the individual, and so on. There is no quantity here that is mathematically defined and can be calculated.
As I explained in my reply to Oren above, I do not agree that there is an irrational element in insurance. There are people who buy insurance irrationally, but it is not true that insurance as such is something irrational.
As for the ketubah, that is how I understood it—that it is about the cost of waiting for the money (and the trouble of obtaining the sum. I think that is also what happens in the market for selling promissory notes and various debts).
Amusing and thought-provoking.
In your view, then, is there no point in taking out home insurance, for example? Since there you lose tens of shekels every month for years for an unknown expected profit that may never materialize (and even if it does, it is doubtful whether it will exceed the amounts of the monthly payments)
Where did you see that in my words? If anything, I wrote the opposite (that expectation is not what determines it).
"Should" is defined in the following sense: in a competition over time between someone who follows the strategy of maximizing the expectation of the logarithm of the winnings, and someone who adopts any other strategy, the former will win with probability one (with probability one).
The Kelly strategy
https://en.wikipedia.org/wiki/Kelly_criterion
Recently I discovered some hole in this argument, but I do not think it is relevant to the matter under discussion here.
But here we are dealing with a decision about one lottery (or a small number of lotteries). That is not mathematically defined, and therefore in my opinion in such a context there is no meaning to "should." Perhaps there is meaning to "reasonable," and that is what I was talking about.
As for what you wrote, even if we assume that this is an optimal policy for infinitely many lotteries, I still do not understand. If we have infinitely many lotteries, why not go by the expectation? After all, what I will get over infinitely many lotteries is by definition the expectation of the profit, not of the log. The log can perhaps determine how many lotteries count as a sufficiently large number for the expectation to be relevant (it is a measure of the standard deviation of the random walk).
Moreover, if the ticket price is entirely up to me, then of course the best thing is to pay 0. That will yield optimal profit. But that is not the question before us. The question is what price I should agree to, not what it is worthwhile to pay. Are you saying that if I am offered a ticket at a price higher than the one you mentioned, I should not accept the offer? That sounds completely unreasonable to me. By the very definition of expectation, it is worthwhile to accept the offer at any price whatsoever, as long as I have infinitely many lotteries before me.
In short, I did not understand what it means that this strategy will bring maximal utility.
Here is another suggested point for thought:
Before you are two envelopes, each containing a sum of money. All you know is that one envelope contains twice as much money as the other. You may choose whichever one you like and take the sum inside it.
You took envelope no. 1, opened it, and discovered 1000 shekels. Now you are offered the chance to switch envelopes. Will you do so?
The "school" solution says that you should calculate the expected gain from switching. Well then, with probability 1/2 the switch will cause you to lose 500 shekels, and with probability 1/2 to gain 1000 shekels. That means your expected gain is 250 shekels. Worth it.
But here you begin to wonder. After all, if you had taken the second envelope, then no matter what amount was in it, it would also have been worthwhile for you to switch. If the second envelope contained X, then in the case of a loss you would lose X/2 and in the case of a gain you would receive 2X, so the expected gain is always X/4.
This paradox shows that when the space of possibilities is not well defined, expectation calculations can be very misleading. I think this is also true regarding an infinite expected gain.
The envelope problem is well known. I do not think the problem lies in defining the event space. Everything there is well defined. In my opinion the school solution is correct. One really should switch envelopes, and it is indeed always the right policy. You will not always profit from it, but that does not mean the expectation calculation is wrong. It is correct. Therefore the paradox you described does not seem like a paradox to me.
Rabbi, but according to what you answered Nadav (that one really should always switch envelopes), then before you opened the envelope—you should switch to the second one (because suppose the one you opened contains X, then the second one contains either X/2 or 2X and the expectation is positive), and then switch back again to the first, and so on forever… Isn’t that a paradox?
I did not understand. After all, the first one is open. After you switched and opened the second one, you already know what is in each of them. At that point switching is no longer relevant. The game is over.
Yes, but I meant that based on what the rabbi said, even before I open the first one—it would be worthwhile for me to switch to the second (let us assume that is allowed), and then before I open it—it would already be worthwhile for me to switch back to the first, and so on.
You are right. Perhaps that is the paradox Nadav described. On the one hand, opening the envelope causes me to prefer switching, regardless of the envelope’s contents. And on the other hand, if it does not depend on the envelope’s contents, then why does it matter that I opened it? And if that is also the strategy with a closed envelope, then one should keep switching forever. I will think about it.
Usually it is customary to think that everything depends on the probabilities of the amounts chosen to be inside the envelopes. (That is probably what Nadav meant when he wrote that the event space is not well defined.) But with a more careful formulation one can see that the problem is not really solved.
See here:
https://gadial.net/2008/07/25/envelope_paradox/
The link is broken
Not for me. It is an article on the site "Not Accurate" about the envelopes paradox. I will post it again:
https://gadial.net/2008/07/25/envelope_paradox/
I think I have a solution to the paradox—after all, in mathematics there are not supposed to be real paradoxes :).
If my policy is to switch in a certain case and only in that case—say, in the case where I open it and get 1000, only then do I switch—then indeed switching in that case is worthwhile, as follows from the expectation calculation.
But if my policy is to always switch, then I gain nothing—against the case of profit when I switch from 100 to 200 there is the case of loss when switching from 200 to 100; since my policy is always to switch, I have to count all those cases in the expectation calculation.
That is, the mistake in the expectation calculation of the school solution is that it does not count in the expectation all the relevant cases. In the case of "Nadav Shnerb" above, it does not count the case where you would not have opened the 1000 but rather the 2000; one must count that case, because your policy is always to switch.
I think that is roughly what Alexandrovich suggested, no?
But at first glance it seems to me that you are right.
I understood that he remained with the question unresolved, but perhaps I did not fully understand him.
Still, something of the infinity-related paradoxicality remains, because in Alexandrovich’s case a switching policy over a finite set of numbers is worthwhile, and yet switching in all cases is not.
The categorical imperative mentioned in note 3 is a good enough reason to go out and vote.
If too many people say, "My single vote has no effect"—it is possible that the democratic regime will not survive, or that the election results will be dictated by the camp whose people do not say that.
Come on, Rabbi, what about a little current affairs?!