The Relationship Between Intuition and Recursive Thinking: Puzzles, Intelligence, and Intelligence Tests (Column 100)

With God's help

This is a celebratory column, in which we all mark the hundredth installment of my abusing the readers. I will therefore use it to dwell on something essential to the great majority of my columns on the site.

From time to time an article appears on the internet containing challenging puzzles for users/readers. Occasionally there are gems there, but usually these are fairly banal things. That of course does not prevent the various sites from presenting them as the puzzle of the century that Einstein said only a tenth of a percent of the population would be able to solve. These articles are further evidence of the somewhat low intelligence of some of the writers and editors at those sites, something that can also be seen in other articles.

My children told me that anyone who clicks into such an article has already failed the test (that is, has turned out to be an idiot). A few days ago I failed such a test, that is, I clicked on an article reporting on puzzles of this sort, and there are some lessons from it that I wanted to share with you here.

The World's Shortest IQ Test

The story was as follows. A few days ago I saw yet another such article, except that this time it dealt with the findings of a Princeton psychologist who claims to have developed a test of three short questions that can tell us, in a nutshell, whether we have an especially high IQ, are unmistakable geniuses, or not. The world's shortest IQ test, proclaims the article's headline. The test, The Cognitive Reflection Test, or CRT for short, was developed by the Princeton psychologist Shane Frederick in 2005, and the article reports that only about 17% of students at the elite American Ivy League universities (Harvard and Yale) managed to solve it successfully (let us assume for the sake of the discussion that this is indeed a good measure of especially high IQ. I would not put my head on the line for that, though admittedly it is not a Harvard student's head).

On the face of it, this sounded promising, academic, and amazingly challenging. After all, Princeton, Harvard, and Yale all in one go—that is not YNET or WALLA. I could not restrain myself and failed by entering the article and tackling the questions. I looked at the three questions and it took me around 5-10 seconds to answer all of them. In fact, most of the time went to reading the questions. I confess without shame that I did not feel my well-known super-intelligence was responsible for this (I am not much good at solving puzzles), since in my estimation an average tenth-grade student would answer all three of these questions correctly. This made me wonder about the admissions threshold of the Ivy League universities, or perhaps of the profession of psychology and psychometrics, and perhaps of the journalists' association. In fact, anyone who does not answer these three questions within a minute at most, in my opinion, does not deserve to be admitted to the preparatory program of the gender-studies faculty at Birzeit's satellite college.

When one looks into the details (see its Wikipedia entry) one sees that these grandiose declarations are the product of the WALLA writer's fantasies. And yet the report about Harvard and Yale students disturbed me a bit, and I searched the web a little (17% is a numerical datum, and it is unlikely that the writer simply invented it, though I cannot rule that out). I found a rather minor report here about the success of about half the MIT students, and here about a similar percentage at Carnegie Mellon and Harvard (see also here on the average success rate at Harvard, Princeton, and MIT). I must say that this too is rather disappointing, though not to quite the same degree. After all, we are still talking about only about half the students at those elite institutions who do not pass a test suited to an average tenth-grader.

System 1 and System 2

A look at more reliable sources shows that the CRT is not really an intelligence test. It is a test meant to determine whether we are good at suppressing System 1 (intuition, gut feeling) and allowing our recursive and controlled thinking (System 2) to govern it. This distinction was discussed at length by Daniel Kahneman, who analyzes their roles and functions, as well as the advantages and disadvantages of Systems 1 and 2. There are things that we do exceptionally well precisely when we are not focused on them and leave them to instinct, that is, when we entrust them to our "automatic pilot." But there are fields in which this is inadvisable. Thus, for example, in column 38 I gave several examples in which people have a clear initial intuition (the law of small numbers), but a more orderly statistical calculation gives us entirely different answers (the law of large numbers).

It turns out that the three questions in the CRT are of the kind in which System 1 tends to trip us up. Here, precisely those who are prepared to use System 2 will succeed (but a particularly sophisticated System 2 is not necessarily required here. See below).

Context Dependence

In my view, the quality of the answers we give to these questions is a product of the context in which they are presented to us. I assume that if we were to ask a person on the street one of these questions and demand a quick answer on the spot, most of them would indeed get confused. By contrast, if it were presented to them as a mathematical question (for example, on a mathematics test), average tenth-grade students would answer all three of these questions successfully within a fairly short time. Similarly, if one enters a site that warns one that these are confusing questions that require one to stop and think in order not to err, one will manage to answer them quite easily (though perhaps, out of panic and awe at their lofty grandeur, one will get confused).

The reason is that these questions indeed contain something that instinctively pulls us in the wrong direction. Some analytical control is required so as not to blurt out the instinctive answer immediately, but to stop, think, and arrive at the correct answer. Therefore, if the questioner causes us to focus and do a calculation, we will answer them quite easily. Once one understands that a calculation is required and that one should not merely throw out an answer, the calculation is very simple indeed. The question is who among us will stop to calculate before answering. Who will go straight to System 1 and who will stop and use System 2.

What Exactly Is the Problem?

It seems to me that in almost every post on this site you can find some kind of CRT. I bring a position or opinion expressed on the web, in a journalistic article, or somewhere else, which on its face many will see as a reasonable, argued, and well-founded position, but after a not overly complicated critique turns out to be nonsense. In most cases one does not need very deep philosophical knowledge or skill to become convinced of this, and anyone who pauses a little to think will reach that conclusion on their own. The trouble is that people generally do not do so. When the analysis is presented to them they may agree, but then the question arises why they themselves did not think of it in the first place. After all, they did not learn anything new from the analysis. All the tools and all the knowledge were in their hands from the outset, and yet they did not make use of them.

I do not mean to claim that my positions are pure truth (they are, but I do not mean to claim that here). Clearly human beings are made of mortal stuff, including your faithful servant, and can err, and indeed make no small use of that ability. My claim is that most people make far too much use of it. The reason is that people are accustomed to reacting through immediate System 1, and therefore all too often arrive at conclusions and positions that do not withstand the tests of logic and reasonableness. It is important to understand that this happens not only when we answer immediately without thinking, but also when we feel that we have checked and relied upon reasons, arguments, and calculations. The reason is that many people do not have the habit of stopping and subjecting things to the critical test of System 2. Even if they have reasons, which ostensibly suggests that they made use of System 2, sometimes this is only an illusion. If and when they are prepared to wonder whether there may nevertheless be a mistake here, whether there may be unfounded assumptions or unjustified logical leaps, they will discover that their System 2 gives a different answer.

Well, we are all human, are we not? When people on the street are asked a mathematical question, it is entirely acceptable and reasonable that they answer from the gut and make a mistake. No great tragedy. But from a public commentator or a person expressing a considered position we would nevertheless expect that they stop and activate System 2 before reaching a conclusion. There we would expect fewer errors. I am not at all sure that this is really the case. As stated, even when people formulate a position in an orderly and systematic way, very often System 1 still stands at the base without any supervision by System 2.

In that sense, it seems to me that there is no small value in this silly test. If people get used to stopping, defining the concepts and the claims properly, and doing the calculation before they reach a conclusion, formulate a position, or simply give an answer, the quality of the positions and the answers would improve dramatically. As stated, in most cases this is not a calculation that requires the abilities of Einstein or Gauss. The problem is not computational ability and skill in thought, but only the willingness to examine one's position again, to be ready to conclude that one was mistaken, and to make one more controlled use of the aforementioned abilities.

Most of my columns here are devoted to this task. I would like to make a modest contribution to our ability to stop the instinct of System 1 and activate System 2 before reaching a conclusion.

Is That Intelligence?

Some would say that this itself is intelligence, or that it is at least a component in the definition of intelligence. All told, such people make better decisions. So perhaps it is right to say that the CRT is indeed an intelligence test. Personally, I do not think so. Even without entering into a semantic polemic, I will simply say that it is important to distinguish between the willingness to stop and make use of System 2 and the quality of System 2 itself (which is what I define as intelligence). Let me sharpen this a bit more.

In column 35 I tried to define the concept of intelligence. Among other things, I pointed out there that a computer, or entities lacking judgment, do not possess intelligence. They can "solve problems," but they do not do so by means of judgment; rather, by means of software that someone has put into them. In this sense water "solves" the Navier-Stokes equations (very complicated equations that mathematicians and physicists can hardly solve in any realistic case), and birds solve differential equations. But as I explained there, these are not processes of "solving." It is mechanical calculation, and it has nothing whatever to do with intelligence. Intelligence requires the exercise of judgment, not merely mechanical calculation.

These remarks apparently contradict what I wrote here. Here I argued that our thinking ability itself (the software and the hardware) is intelligence, whereas the willingness to make use of it is something else. And there I said that our mechanical abilities are not intelligence. But this is only an illusion. The judgment we exercise is part of our thinking capacity and not of the willingness to make use of it. My claim there was that our thinking capacity is not mechanical logical calculation, but includes within it a component of judgment and decision-making. Without that component one cannot relate to the capacities of thought and calculation as intelligence. That is what I dealt with there. By contrast, the ability to make use of all this is something entirely different, and that is the subject with which I am dealing here.

What emerges is that there are four components in informed and successful decision-making: 1. The best hardware possible. This is usually not in our hands. Einstein and Gauss apparently had much better hardware than most of us. 2. The software, that is, the information and the skill. This already is in our hands to a certain extent. Those with excellent hardware will be able to go farther, but any hardware can be improved through acquired study and skill. These two together, the hardware and the software, are our computer. 3. In addition to these two there is judgment, which is the ability to decide among the possibilities that arise from the calculation and to choose the relevant computational technique. This is a uniquely human characteristic, and as I explained in column 38 it is a necessary condition for the definition of intelligence. Intelligence is the sum of these three components (and not only of the first two, as those working in artificial intelligence assume. See column 38), and this is in fact our System 2. In addition to System 2, which includes these three components, there is also a further component: 4. The refusal to adopt the conclusions of System 1 automatically, and the willingness to stop and make use of all the abilities of System 2 in order to arrive at the correct position or answer. That is the component with which I dealt here.

All this explains why System 2 is so important, and why it might seem that System 1 is nothing but a collection of misleading impulses that our goal should be to ignore and get rid of.

When Is There Room for Intuition?

A reader familiar with my holy writ may wonder about these remarks of mine. In my books I extol intuition and lay much blame at the door of those who refuse to recognize it, make use of it, and appreciate its conclusions. Whereas here, apparently, I am saying the exact opposite. I wrote here that people who rely on their intuition (System 1) and do not make use of analytic-recursive thinking (System 2) arrive at errors. Our ability to reach informed positions depends on a willingness to ignore our initial intuition. My claim is that there is no contradiction here. To explain this I will present two relevant criteria.

First, adopting foundational assumptions differs in its essence from mechanisms of analysis and inference. Analysis and inference use systematic logical and statistical tools, whereas foundational assumptions originate in intuition. Using intuition within inference is a recipe for trouble, as I described above. But foundational assumptions necessarily stem from intuition. What matters is to keep the domains separate and to use each system in its own domain.

But even that is not absolute. Often orderly analysis leads us to conclusions that seem, on their face, problematic. We examine the results of System 2 by means of System 1 (intuition), and they seem incorrect. What arises here is a confrontation between intuition (which sees a problem here) and analysis and recursive thinking (which led us to those conclusions). Is it necessarily right in such situations to give up intuition? It is important to understand that almost every paradox is built in this way. There is a logical calculation that leads to a conclusion, but the result seems, on its face, absurd. Must we infer from what was said above that intuition should be rejected in favor of systematic inference? This is a confrontation between System 1 and System 2, and here I sided with the superiority of System 2, certainly when analysis and inference are at issue.

There is an additional criterion here. If my intuition leads me to conclusion X, and then I stop, wonder whether there may be a mistake here, define the concepts properly, perform the systematic calculation, and then arrive at the conclusion that this is not correct, in such a case it is more appropriate to accept the superiority of System 2 and give up the intuition. But if even after I have gone down both paths, tested my intuition by means of calculation, and it still remains in place—I have not been convinced that the intuition erred—then there is certainly room to focus on the calculation and see what is mistaken in it. Intuition is a very important tool in criticizing systematic and analytic thinking (System 2). In such situations, it is precisely System 1 that serves as a critique of the operation of System 2.

As I understand it, one can say that in the final analysis the supreme criterion is precisely intuition. If my conclusions have been tested along both paths and I still remain where I was, I will generally adopt what my intuition tells me. But it is very important that before I reach that conclusion I also do the calculation in System 2. This does not mean that I will always accept its conclusions, but it is very important always to compare myself against it.

To understand this, take as an example the three questions in the CRT mentioned above. If you answer the questions immediately, and afterward do the calculation and see that you erred, there is no doubt that you will understand that your initial intuition was mistaken. In such a situation the decision is clear in favor of System 2, because System 1 itself relinquishes its position. In such a case it is clear that we must adopt the conclusions of System 2. But if even after the calculation it still appears that there is a computational error in System 2 (this is the situation that arises in paradoxes), and in fact my intuition remains in place even after the System 2 calculation, then there is certainly room not to give up System 1. Intuition carries important weight, and it would be a mistake to belittle it.

The Synthetic Stage

In my books Shtei Agalot and Emet Ve-lo Yatziv (see especially the introduction to the second book), I presented a three-stage model of the development of our civilization, and in parallel, of the maturation of a child. In the initial, dogmatic stage (childhood), he accepts what he is told, or his initial intuitions, as self-evident. He does not bother to subject them to empirical, scientific, or logical testing. He does not activate System 2. In the second stage, adolescence, he seeks certainty and puts everything to the logical test. Here System 2 prevails, and System 1 is abandoned entirely (everything is logic and proofs). The third stage comes when the youth matures and reaches the insight that there is nothing that can be proven with absolute certainty (for everything depends on foundational assumptions, and those themselves, by their nature, have no proof).

At this point three possibilities stand before our adolescent: a. Skepticism (postmodernism) — if there is no certainty, there is no truth. Everyone has their own truth. b. Fundamentalism — adopting alternative (non-rational) paths and reaching certain conclusions through them. c. Synthesis — true, we have no way of reaching certainty, but certainty is not required. We accept things if they seem reasonable to us, even without certainty. What is the difference between the third, synthetic stage and the first, dogmatic stage? There too we accepted things without proof simply because they seemed right to us. The answer is that at the mature and synthetic stage we are not dogmatic. We accept things, but we are aware that they are not certain. We are prepared to subject them to the test of critical thought, and to give up our conclusions if they prove unreasonable to us. For this purpose humanity developed a system of systematic rules of thought that govern our scientific thought and our rational everyday thinking. These are not rules of deductive logic, but rather "softer" rules that critique our scientific and everyday thinking (as distinct from mathematical and logical thinking). Francis Bacon presented a system of such rules that came to be called inductive logic, or the logic of science. These are non-deductive systems, and therefore by their nature they include an intuitive component. As I showed in the aforementioned books, without intuition there is no critical thought either. Without System 1 there is no System 2.

In my book Emet Ve-lo Yatziv I presented a system of rules of "soft," non-deductive logic, drawn from the logical hermeneutic principles (a fortiori reasoning, the two binyan av rules [arguments from a paradigm case], all the refutations of all these, and their combinations at every level of complexity). In the first book in the Talmudic Logic series we showed that this system allows us to conduct our non-mathematical thinking (the accumulation of information and the way we handle it) in a controlled manner, even if not with certainty or pure logic. This is also the way science works. A scientific conclusion is not the fruit of logical analysis, and therefore it cannot be free of intuitions. And yet science is a systematic and controlled mode of thought. It certainly uses System 2, but not only it. Mature synthetic thinking makes use of both these systems, with each one critiquing the other. Ultimately, it seems to me that intuition is decisive. But only after it has stood the critical test of System 2, as above. Conversely, it also returns and critiques System 2. The art of thinking is knowing how and when to combine these two systems, each of which is very important to our functioning, our decision-making, and the formation of our positions.

The Columns on the Site versus My Books

In sum, the CRT, although not a significant challenge for anyone who has made it through tenth grade, nevertheless reflects something of great importance to our conduct. It does not test intelligence, but it does test the ability not to use System 1 immediately, but rather to integrate System 2 into it before reaching a conclusion. As we saw here, that ability is not intelligence, but it is very important for decision-making and for forming a rational, controlled, mature synthetic position. By way of a somewhat imprecise generalization, one could say that my main aim in the columns on the site is the opposite of my aim in my books. There I wanted to dwell on the legitimacy and the importance of System 1, and this against the analytic conception that focuses on System 2. In the columns on the site my goal is the opposite: to sharpen the important role of System 2, analytic thinking. In my books I showed that it is not the whole story, but here my aim is to point out that it is not right to abandon it. In the end, the goal is the proper integration of the two.