Will Artificial Intelligence Be Able to Replace Us? (Column 592)

With God’s help

Disclaimer: This post was translated from Hebrew using AI (ChatGPT 5 Thinking), so there may be inaccuracies or nuances lost. If something seems unclear, please refer to the Hebrew original or contact us for clarification.

In the previous two columns I dealt with artificial intelligence and its philosophical and halakhic implications. The entire discussion proceeded under the assumption that artificial intelligence will, at some point, reach a state in which it can do everything human beings can do, and even more. I argued that in such a state it could rule on halakhah, make moral decisions (as in an autonomous vehicle), and more. On the other hand, I argued that even in such a state it does not have the moral status of a person, and it also lacks understanding and creativity. It engages in imitating the syntax (form) of our thinking and language, but not their semantics, namely understanding and meaning. Although there is a correspondence between the two (as Wittgenstein argued, and as the adequacy theorem in logic says), it is a mistake to identify them.

Up to this point I was careful to leave open the factual question of whether AI indeed can, or will be able in the future to, do everything a human being does. I noted the very generic and formulaic phrasing of many of the answers given by AI programs to the questions they are asked, but that could be solved over time. In addition, I pointed out several examples about which I am doubtful of its ability, such as scientific generalizations (discovering the theory of relativity or quantum theory), and more. Following questions by Tirgitz in the comments to the previous columns (see here and here), I thought it appropriate to devote some space to these questions as well.

In this column I will try to return and examine a bit the very common assumption among AI researchers today: Is it really only a matter of time until AI programs can replace us in everything? Do we, as human beings, have no ability (as distinct from understanding and meaning—which we have and the machine does not) beyond sophisticated software? I wish to deal here with the syntactic question itself: Is the similarity between person and machine at the level of syntax indeed correct? If that is the case, then fully sophisticated machines (if there ever will be such) could replace us in every field in which we operate, and do it even better. Here a more refined and complex Turing test enters the picture, one that contemporary AI is very far from. If we place an AI at the knowledge state of the end of the nineteenth century (an experiment that is in principle possible today), will it arrive at the theory of relativity and quantum theory? Perhaps it will even succeed in unifying them—something physicists still do not know how to do today (known as the problem of quantum gravity).

Before I proceed to discuss this question, let us clear value questions off the table.

Weakness of the Will and Value Judgments

It is interesting to examine whether a thinking machine could have a state of weakness of the will (see in Columns 172 – 173)—that is, can it think one thing and act otherwise? It would seem not. The experience of weakness and the dissonance between what I think and what I will actually do does not exist in a machine, since for it what it “thinks” is exactly what it does in practice. I do not see how one could program it to think one thing and do another, since it does not really think. In a machine there is only the practical dimension and not the mental one. What it “thinks” is expressed in what it does. It does not have two such planes at all, since it has no mental interiority as we do.

But thus far this is only an advantage. Our sins generally stem from a weak will, namely a gap between what we think and what we do. Our drives sway us; we construct imaginary pictures for ourselves (see the story of the Rooster-Prince, in Column 354, among others) that justify our steps, even though deep down we understand what is right. Machines have no such problems. If they think something, that is what they will do. Therefore, the question is not whether they will act correctly but whether they will think correctly. If they “think” correctly—that is, if they reach the correct moral conclusions—then that is what they will do.

If we look at moral reasoning as a kind of computation, there is no reason to rule out the possibility that a machine could perform that computation. To be sure, in some cases there is no single correct answer (humans disagree), and then it is unclear what would count as the right answer in the refined Turing test we would administer. But at least we can test it on questions about which there is consensus (these are most questions). In cases where there is disagreement, perhaps we can accept each of those moral positions as legitimately correct. Therefore, if indeed we are dealing with a kind of computation, then in principle it is possible to bring a machine to think like a person and to reach the moral conclusions and decisions that a person would reach. The question is whether this is really true. Is it indeed computation? In other words, is there an algorithm, based on several rigid principles, that yields the correct moral answer?

Think about forming a position on the question of socialism versus capitalism. Sometimes the debate proceeds as if this were a factual dispute—which system will produce better economic outcomes. But at its root this is a value dispute, not a factual one: do we prefer the value of freedom or the value of equality? Of course, some prefer this and some prefer that, and therefore it is a contested issue. But how does each side form its position? Does one simply compute it? If that were the case, then such disagreements would not really be meaningful. The computations differ only because each person has a different mental makeup. If so, this is not a moral decision but merely the product of a psychological structure. By contrast, if we see such a debate as a truly value-laden one, we are tacitly assuming that a person arrives at his position by exercising judgment. This is not computation but a value decision, and the disagreement arises because people have made different value choices. If that is the correct picture, this is a free decision of choosing a value, and it is hard to see how a machine could imitate such a process. Even if we feed it all the data about a given person and all the economic and social data, it will not necessarily reach the result that person will reach, because there is a decision here and not just deterministic computation from data (see on this in Columns 35 and 175). To be sure, all this only says that the machine will not be able to imitate a person perfectly, yet its decisions might still be correct either way, for whether it arrives at socialist or capitalist conclusions, it remains within the legitimate realm as humans are. However, with a machine it is possible that it will arrive at a third kind of conclusions altogether that are manifestly immoral. Therefore, in any case there is here an important point that distinguishes person from machine.

Of course, one can feed the machine the values of a particular person (assuming all of them can be formulated and explicitly stated), and then it will at least imitate that person (it will be socialist or capitalist, depending on whom it imitates)[1]. That is, it will not arrive at a conclusion entirely outside the moral framework. But another question arises here. Think of a new situation we have not faced before, and we need to make a moral decision about it. A machine trained to think morally on the basis of familiar cases whose correct moral answers were fed into it may not know how to answer a new case whose answer cannot be deduced from the previous cases. And what about a person in such a case? That depends on your view of the person and his moral reasoning. If a person has an embedded system of moral principles (I mean moral intuition, not necessarily rigid principles from which the derivation is a logical deduction), then he can employ them and derive a conclusion even for the new case. This is, of course, assuming that these embedded principles are not the product of accumulated experience and training of our neural system, but something ingrained in us in some way, or our own decision (in any case, not arbitrary; otherwise it has no weight and is not “true” in any sense). If that system were the product of accumulated experience, then a person too would err in a new case that cannot be learned from cases we have encountered in the past. It should be remembered that we train the machine on the basis of previous cases (that is how AI works), since we cannot feed it the moral principles themselves (as in classical software), because we do not have an explicit and complete formulation of them. In such a case, it may be that AI will not make correct moral decisions in some cases (the new ones). Now, if we assume that human beings will indeed make correct decisions in such cases (due to the moral system embedded in them, as above), the conclusion is that the machine cannot replace us in all moral decision situations.

I will note that with respect to autonomous vehicles the problem does not really arise, because most of the issues there are entirely computational. The goal there is to make decisions at every step that will save as many lives as possible, and therefore the question is only technical: what is the best way to do that. A sophisticated machine should do this better than we do. Moreover, as I explained in Column 590, it is clear that, in general, AI will act better than humans. Even if you find rare new cases in the sense described here, for which the machine will make an incorrect value decision (for example, the trolley problem is not computational but value-laden: is it better to kill one person or allow five to die passively),[2] still, if the consideration is overall benefit and gain, using machines is preferable. But if I want to discuss the status and capabilities of a machine to make correct decisions (i.e., no less correct than a person’s) in every given case, that is a different question, and the answer to it is not entirely clear.

Incidentally, I will add that machines also do not have moral responsibility, and for human beings this is a very important aspect of the moral decisions they make. But this too is not a real problem, since the machine imitates the decisions of human beings in such a situation, and these decisions are made out of the responsibility they feel. In this sense the machine imitates the responsibility as well, since it is factored into the decisions themselves. Of course, if the machine erred we will not punish it, but that is not our discussion here. We might punish the programmer or the user, but even that seems implausible, since he did not intend the problem that occurred. It is the result of an error or of imperfect software. There is no criminal intent here, and probably no responsibility either (unless there was negligence by the programmer or the regulator).

Thus far regarding machines’ ability to make correct moral decisions. As noted, the answer is unclear, at least regarding new cases. It is quite clear that in new cases the machine will not arrive at a “correct” answer (by definition, being new, the answer cannot be deduced from previous cases), but it may be that human beings are also computationally built (this is the standard assumption among AI researchers), and then they too will not arrive at the correct answers. If that is so, nothing can be inferred about the inferiority of machines compared to humans. And in general it will be difficult to conduct a Turing test for the value decisions of a machine, since right and wrong here are not unambiguous. Let us now move to examine their judgment regarding facts, where the situation is supposed to be simpler. In principle there should be no disagreements there among people, and there is a simpler way to administer the refined Turing test on this. But, as we shall see, even there the situation is far from simple.

On Empiricism and Thinking Machines

Seemingly, when we examine facts and try to understand or explain them and derive conclusions from them, this is a purely computational process. Even if it is complex, at the end of the day we depart from the facts before us and arrive at conclusions that are warranted or derived from them. If so, a sophisticated thinking machine could seemingly do this procedure. Yet this is not simple at all. First, I have explained more than once that the move from facts to an explanation (the theory) is neither univocal nor apparently algorithmic. The same set of facts can be explained in several ways, and the choice among them seems to involve a creative component. Whether a machine will be able to imitate this process depends on the extent to which it is computational.

This is precisely the mistake of naïve empiricism (see on it in Column 586), which thinks that science works by collecting facts, analyzing them, and deriving the warranted conclusions. Such an empiricist believes that the laws of nature and the conclusions of science are the product of pure observation and nothing more. There is no synthetic intellectual component here (and therefore sometimes he also thinks this is certain). But that is a mistake, since there is no scientific inference that is purely deductive. The path from facts to theory is winding and very resistant to formulation. Things go so far that, in the philosophy of science, a distinction is drawn between the “context of discovery” and the “context of justification.” The claim is that we have no way to trace and criticize the arrival at the formulated theory. All we can do is run experiments to examine whether its predictions stand up to empirical test. Formulating and articulating a scientific theory is a complex process, and it is very doubtful to what extent it is computational. Therefore it is not clear that a thinking machine, however sophisticated, will be able to do this in all cases.

Only now, as I write these lines, I read an interview about Sam Altman, the CEO of OpenAI, which developed ChatGPT (please ignore the exaggerated superlatives regarding his importance and cosmic stature; that is mostly journalistic). Within the piece there is a paragraph devoted to whether an AI program will succeed in making scientific leaps:

Altman’s most important goal—the “grand target” that will herald the arrival of Artificial General Intelligence—is a scientific breakthrough. GPT-4 can already synthesize scientific ideas, but Altman wants AI that will stand on the shoulders of giants and see into the depths of nature. AI tools have created new scientific knowledge, but they are algorithms with narrow aims, not machines of general reason. AlphaFold, for example, opened a new window onto proteins by predicting many of their structures—an impressive achievement given their importance for medicine. Altman is betting that future machines of general reason will be able to create new scientific insights.

Altman imagines a system that will generate its own hypotheses and test them in simulation. He emphasized that humans must remain “in full control” of real-world lab experiments, although to the best of my knowledge there are no laws that guarantee this. He longs for the day when AI will be able to “go understand the rest of physics.” For that to happen, he says, we will need something new built on OpenAI’s existing language models. It seems that no one at OpenAI knows what needs to be added to GPT-4 to produce something that can surpass human thinking at its highest levels…

But generating hypotheses is not sufficient to imitate human scientific progress, since there are innumerable hypotheses one could test. Our principal ability is to sift the relevant hypotheses, or to hit upon them in an initial intuition (below I will speak about the “context of discovery”), and then to test what we have found (the “context of justification”). Beyond this, there is also a problem in the justification context he describes. What does it mean to test a hypothesis in simulation? If you know the fundamental laws and want to know what they will do in a complex situation, or to generate some generalization of them (for example, to generate the laws of biology from the laws of chemistry and physics, assuming reductionism is true), simulation is the right tool for the job. You simply build a computerized system that behaves according to the fundamental laws and see what large-scale picture emerges in the complex situation that interests you. But discovering the fundamental laws themselves cannot be done by simulation, since the simulation itself must run according to those laws. What exactly does he want to instruct the simulation to do in order to test the result? Fundamental laws of nature are not discovered in simulations but in observations of nature itself (which, by definition, operates according to the true laws).

You can already see here a hint of a special human ability that may not be “computational,” and therefore poses a real challenge for AI programs: breaking out of a circularity. In the following paragraphs I will try to clarify a bit the difficulty AI faces when it comes to replacing us, namely to clarify a bit the uniqueness of human thinking as it manifests in this domain.

The Core of the Scientific Process

When a scientist approaches a complex scientific issue that requires a new scientific theory, he finds himself in a state of “paradigm crisis,” in Thomas Kuhn’s terms. In such a state the existing theory encounters several difficulties for which it has no solution, and then it is clear to all that a new theory is required to replace the existing paradigm (as with relativity and quantum theory versus classical mechanics). In such a situation, the scientific community generally solves the problem collectively. Even if there is a special genius who contributes a decisive contribution, almost always he stands on the shoulders of giants who preceded him (and generally one does not ignore prior theories but refines and articulates them). But we will set this aside for the sake of the discussion and consider the steps required for such a course. I will rely here on things I have written and said elsewhere in the past (mainly in my series of lectures on learning from experience).

In Francis Bacon’s naïve conception, science first gathers facts. Afterwards, it analyzes them and tries to create from them a generalization (by induction) that will encompass them all within one broad framework. In the final stage, one tries to generate from the overall account a new theory (this is a process of abduction; see in Columns 399, 537, and more) that will explain them and additional facts as well. These are three stages that constitute one link in building a scientific theory. After completing them, we return to gather more facts, to correct the generalizations and the theory, and so on. It turns out that each of the three stages in this process requires creativity, and it is very doubtful to what extent it is “computational.” Let us now see this stage by stage.

A. The Vicious Circle of Collecting Facts

At the first stage, as noted, one must collect the relevant facts. But this in itself is not a trivial step, since in the absence of a theory one cannot know which facts are relevant. There are several clear examples of this in the philosophical literature, the most common being that of Ignaz Semmelweis and puerperal fever, which is presented in the work of the philosopher of science Carl Hempel.

Semmelweis was the head of one of two maternity wards in a large hospital in Vienna. In his ward there was a much higher mortality rate among women than in the other ward. They searched for the reason, but had no direction. They tried to collect the relevant facts, but as long as they did not understand what caused the mortality, they could not know which facts were relevant. As a starting point, there could be innumerable facts that differentiate the wards: ceiling height, the average age of the doctors, the median age of the nurses, the head nurse’s surname, wall color, the ward’s floor area, how long it has existed, the size of the chief steward’s office, the length of the test tubes, the company that made the ceiling chandeliers, and so on. As long as you do not know what causes the mortality, you cannot search for facts. Had they known that the cause was different organisms causing infections, they could have searched for reasons why hygiene differed between the wards. That is indeed what they eventually discovered, but that too happened almost by chance. If you have no initial hunch for a theory that will explain your facts, you have no way to isolate the relevant facts.

Think of Newton as he set out to formulate the law of gravitation. Why should he have treated the tides, the fall of objects to Earth, and the orbits of the planets as a set of facts that belong to the same domain and are governed by the same laws? Why not the color of the bird that flew by this morning, or the frequency of its call? Only after you know that there is a force of attraction can you understand that perhaps all these facts might be relevant. So how did Newton decide to focus on precisely this set of facts? It seems he had an initial intuition, even before he knew the theory he would find, which guided him as to what could be the relevant facts and what surely did not belong. Without this he would have wandered endlessly among innumerable facts and would not have succeeded in isolating the relevant ones.

Incidentally, in the lecture series mentioned above I explained that the very division of science into disciplines and different fields (physics, chemistry, biology, psychology, and of course all the branches of each) itself defies simple understanding for the same reason. There is an assumption here that one set of facts belongs to the laws of physics and another to the laws of chemistry, and so on. But why did we group the totality of facts precisely under these categories and not choose a different partition? This itself is a creative-intuitive process.

To summarize: collecting the facts requires some acquaintance with the theory. But the theory is built on the basis of the facts. This is a circle for which it is very hard to see a “computational” way out. It requires an initial intuition that we possess before we have formulated the theory and collected the facts. Afterwards we collect facts that intuitively seem relevant, build a theory (induction and abduction), and then we return to the theory and correct it, and so on. Sometimes it works and sometimes it does not, but there is no other way.

This ability I have called “intuition” appears, on its face, to be a non-computational ability. It is inspiration with an essential creative dimension. There will be those who will claim that this is an illusion and that, in fact, it is computation, even if covert. Our intuition is actually performing a gigantic computation very quickly, but in principle it can be imitated by software and algorithms. It is very hard to believe this. There are so many possibilities, and we ourselves clearly do not go through all of them. Just think of the total number of facts in the Semmelweis case. There are millions or billions of such facts. The set of relevant facts is the set of all subsets of those facts, which is 2 raised to the number of facts. These are fantastical numbers of possibilities; within each possibility one must try to build a generalization and a theory, and then return to the facts, and so on. It does not seem practicable. We certainly do not do it that way, and a mechanical imitation that systematically arrives at the same results seems somewhat implausible.

B. Creativity in Choosing the Right Induction

Let us now assume, for the sake of discussion, that we have succeeded in isolating and collecting the relevant facts, and we have the data concerning them. We now seek the correct generalization for them. In Column 426 I discussed the problematics of generating a generalization from any set of facts. It turns out that any set of facts allows innumerable theories, and thus there is, here too, a non-computational dimension. Admittedly, here the problem is somewhat easier since we use the principle of Occam’s razor—namely, we choose the simplest generalization. This is not an univocal procedure either, since there may be several generalizations of the same level of simplicity, and in fact there are additional criteria (for example, avoiding over-fitting and under-fitting; see on this in Columns 243, 426, and more), and therefore this too is not entirely “computational.” But overall, this allows us to sift several candidate generalizations from among many others. In fact, this is the basis of most basic machine-learning algorithms; that is, we can teach the machine to do this.

But note that here too we are, after all, imitating a procedure we perform. That is, had we not had our brain built as it is, it is doubtful to what extent we could have created machines that do this. If we are imitating a person’s creative procedure, does it mean the machine itself has creativity? Not necessarily. It may be that we have creativity, and after we uncovered the creative way of thinking we can build machines to imitate it. Thus, for example, we hit upon the idea of seeking the simplest generalization (Occam’s razor), and now we train the machine to look for the simple solution and even define for it what simplicity is (for example, a straight line is the simplest graph among graphs, and thus we employ the least squares algorithm to find the straight line that best fits the data). Still, even if a machine can only imitate us here, it seems that, at the end of the day, we are dealing with a “computational” procedure. Therefore, at least in the practical sense, there appears to be a mechanical substitute for human thinking.

C. Between Induction and Abduction

There is a common mistake that the move from the empirical set of facts to a theory is an inductive procedure. But this is a mistake. Induction generates a general technical rule of which the facts are particular cases. The path to a theory requires an additional procedure called abduction (see on it in Column 537). Thus, for example, the law of gravitation gives us a formula that describes the acceleration of any body under the influence of another body. But this description is not an explanation. The explanation for the facts is given by a theory: we propose that any two bodies with mass exert on each other a force described by this formula, and the force produces their acceleration. So too with any other scientific theory. The theory generally includes theoretical entities we cannot observe directly (force fields and the wave function in quantum theory, the electron and other elementary particles), but the theory points out that they participate in producing the phenomena we wish to explain.

The move from the formula obtained by induction to the theory is sheer creation. We have no way to describe the move from those to this. So much so that, as I mentioned, the philosophy of science speaks of a “context of discovery,” which is an almost mystical matter beyond our ability to trace and critique, versus a “context of justification,” which is the empirical testing of the proposed theory in the lab. Will a machine be able to perform abduction, that is, to move from a set of facts to a theory? Will a machine placed at the knowledge state of the early twentieth century be able to propose quantum theory or the theory of relativity as a possible solution or explanation? It is very hard for me to believe so, although it is not impossible. Again, the issue depends on whether this is a “computational” process, even if very complicated, or something essentially creative that cannot be mechanized.

In fact, the question is whether Einstein’s head is a sophisticated computer, or whether there is something there beyond phenomenal computational ability. It is quite clear that Einstein himself certainly did not go through all theoretical possibilities and then select one. Our intuition helped him sift the reasonable ones and focus on them until he arrived at the formulation of the “correct” theory. But if so, there is still a “computational” procedure here that could be imitated by an AI machine, even if it would need to go through all possibilities. Alternatively, perhaps we can teach it to imitate Einstein’s creativity and skip the need to traverse all options (this is what is done today in chess programs, which already beat all human players). Still, there is something in creating a scientific theory that seems far more creative than a brilliant tactic in chess. In the end, in chess it seems fairly clear that a machine with immense capability always has the option to go through all possibilities and win without a drop of creative thought or genius. Therefore, there it may be a shortcut in practice. The question is whether every human thought has an algorithmic computational counterpart or not.[3]

It is important to understand that even if a computer can imitate Einstein, it does not necessarily follow that it can do every other human move. It may still be that there are even more ingenious human moves that would require resources and time beyond what it has, and it will never be able to replace the person completely—at least in the technical sense of time and computational resources (that is, there may be problems that are in principle “computational,” but require enormous resources, and only a human will know how to shortcut the path while the computer will not).

What Is Intuition: Between Thinking and Cognition

In several places I have argued that at the base of these human abilities stands a capacity we call “intuition.” In these terms, the question this column addresses can be phrased thus: will a thinking machine be able to develop intuition? This of course depends on what intuition is and how it develops in us.

It is common to think that our intuition is nothing but the fruit of accumulated experience. After we have encountered many facts and solved many problems, a neural network develops in us that can solve new problems. On this view, the development of our intuition is nothing but training a neural network. If so, it is clear and expected that thinking machines will be able to replace us in everything. We too are nothing but such a thinking machine.

According to this view, intuition is merely a tool for data analysis. Observation brings data to us, and our intellect processes them and produces understanding, or a theory. Intuition is part of that capacity for analysis and processing. It is important to understand that in this sense intuition belongs to the category of thinking and not to the category of observation. I will clarify this a bit. Cognition or observation are human faculties grounded in our interaction with the world. We observe the world and draw data from it through our senses, and this is what is called observation or cognition. Thinking, by contrast, is processing that takes place wholly within us (without interaction with the world). The person receives data from observation (cognition), and then his thinking processes them and infers conclusions. Intuition, as it is commonly described, is part of thinking. It is trained by past observations and inferences, through various feedback it receives from many directions, but in the end a cognitive capacity is built that belongs entirely to thinking.

I do not accept this thesis. In several places I have argued that intuition is thinking-cognition, a kind of observation not performed by the senses but by the intellect. My claim is that understanding the general law of nature from the facts we observed cannot be explained by thinking alone, for thinking as such could take us in innumerable directions and there is no probability that we would arrive at the right one. The natural way out is that intuition is a kind of contemplation or observation of the world. We, as it were, “see” the correct theory through the facts we observed with the senses. In Columns 494 – 496 and in Column 502 that continues them, I argued that this is the only reasonable solution to Kant’s synthetic-a-priori problem, and in Columns 155 – 160 I showed that this is a necessary assumption for philosophy to exist at all, as distinct from science.[4]

It is important to understand that this intellectual “seeing” contains a creative dimension. It is not passive observation (another sense), since it has a quasi-thinking component as well, but it also has a cognitive component. Without it there is no intuition. By definition, this cannot be a computational ability, for if it were then the route from empirical data to the theory or their intuitive understanding would be purely intellectual. The data alone would suffice to derive the conclusion mechanically. Intuition does something beyond computation based on the data set; otherwise it would truly be pure thinking. As noted, it has a cognitive dimension.

In light of this, Tirgitz raised a very important question in the comments to the previous columns (see here and here). If intuition indeed has an observational dimension, it does not seem that a thinking machine will be able to imitate it. A machine can process the data it receives and arrive at some conclusion, but it has no ability to contemplate the Ideas. Even if we build it senses like ours, it will be able to accumulate more data, but not to process them and connect them to abstract ideas and concepts. In theory one might imitate even this “sense” that we have, but if it is not a regular sixth sense but something that also includes a thinking-creative dimension, it cannot happen. If there will be a machine that has this ability too, we will have to plant it within it. It will not arise as a result of training a neural network on different examples, since training only shapes the network and creates a better computational machine. A non-computational step cannot be performed by a machine.

In other words, a thinking machine may not be able to create philosophy, since it lacks the ability to contemplate the Ideas. It will, of course, be able to be a professor of philosophy, and that is indeed what the articles it produces look like (see in the previous columns the generic and formulaic nature of its answers).

These points also connect to Wittgenstein’s question of “following a rule,” as presented in Reiss’s article discussed in Column 591. There too the claim is that following a rule always presupposes a kind of intuition that cannot be conveyed algorithmically. To be sure, we can teach software to follow rules, but this is done by training that brings it to be built like us in its understanding of the rules. It is not truly conveying and clarifying to it the very meaning of the rule. It is unclear whether we can clarify for it a new rule, i.e., a rule that was not inserted into it through its structure or that is not built on a simple combination of rules already inserted.

Summary

You can see that I have no decisive argument for or against the prediction that machines will be able to replace us entirely. I have shown what this question depends on and what its implications are, but I do not know the answer. The essential question depends on how we understand the human capacity called “intuition.” Thus, for example, one who believes that philosophy is defined as distinct from the sciences (a field of intellectual observations of Ideas) is, in effect, assuming that machines will not be able to replace us, because they cannot do philosophy—only collect data and analyze them. One who thinks that in the scientific paradigm shifts from facts to theory (and particularly in intellectual leaps like those seen in quantum theory and relativity) there is a creative dimension, will also likely hold that machines will not be able to do this in most cases.

In general, we have seen that in principle software may be able to imitate any computational procedure (though we have no clear definition of exactly what such a procedure is). But non-computational procedures, if there are such at all, are unique to humans. Beyond this, we have seen that even in computational procedures, humans do not always act computationally. Sometimes they shortcut the path by means of intuition. But that is only a shortcut and not essential creativity. Although I noted that even here it is not clear that machines will always be able to replace us, since that shortcut itself is the fruit of some creativity, and it is unclear whether a machine could do that. It is true that if we are dealing with a computational procedure, the machine will always be able to carry out the computation by traversing all possibilities, but that depends on computational resources and time, and it is not clear whether future machines will be able to handle every problem regardless of the resources required to solve it. But this is not a very interesting difference at the philosophical level (though it is very interesting practically). The philosophical question is whether there is a fundamental, qualitative difference between person and machine, beyond quantitative differences in either direction.

[1] I have already noted that AI programs developed in Silicon Valley tend to lean left, since the environment that feeds them holds views that lean left. A few months ago I saw someone ask ChatGPT for its opinion on the judicial reform, and he received the expected answers from software trained in a “left-leaning” environment.

[2] Incidentally, there is room to claim that if it is the software that kills the one person, there is no moral problem because it is a machine and not a person (there is no prohibition of murder on a machine). Therefore, with machines one should consider only expected-value considerations in quantitative terms of human lives. To be sure, there is also room to see the programmer or the vehicle’s user as responsible (see in the aforementioned column the difference, in IDF jargon, between an unmanned aerial vehicle and a remotely manned one).

[3] This is not a problem from the well-known NP-Complete family, since there we are clearly dealing with computational problems, but the computational power and time required to solve them are enormous (exponential and beyond). That is what is called non-computationally tractable problems in computer science. As far as I understand, computer-science researchers do not know how to define computability in the essential sense I am speaking of here (this is one of the challenges in a systematic treatment of the Church–Turing thesis mentioned in the previous column). I am speaking of something that cannot be “computed” at all (perhaps not even in truly infinite time, insofar as such a category can be defined).

[4] See also in Column 268, note 10, where I noted that intuition is not the product of experience.