Evolution, Dualism, and Artificial Intelligence (Column 763)
With God's help
A few weeks ago I received a question for the responsa section about the relation between language models and human thought. This topic has been addressed here quite a bit, but here I want to touch on the essential differences between language models and us, and to wonder about their significance.
What Is Thinking?
Several times in the past I have pointed out that our thinking is indeed carried out by the brain, but it does not necessarily follow from this that the brain is a thinking organ. We walk with our legs, but it is not correct to say that the legs walk. We are the ones who walk, and we do so by means of the legs. The same is true of thinking: in my view it is correct to say that we think (this is the function of the intellect), and we do so by means of the brain.
I have often pointed out that this alone already means that artificial intelligence does not think. An artificial-intelligence system is parallel to the brain, not to the intellect. But there is no one inside it who thinks by means of this 'brain.' Perhaps the user or operator of the artificial intelligence thinks through it. Still, there is room to ask whether artificial intelligence really is fully parallel to the human brain. Can it do everything the brain does (and perhaps even better)? And even if there are differences today, one may still ask whether this is only a temporary matter that will disappear with the improvements that will be made in artificial-intelligence systems over time. In this column I wanted to address that question.
In column 592 (the third in that series) I already touched on this question, and tried to point to various human functions in which we are superior (at least for now) to artificial systems (see also the series of columns 694 – 699). Some of those differences no longer exist today (see, for example, the above-mentioned question regarding generalizations). Still, several differences remain that deserve attention.
Semantics and Syntax
One difference I have already discussed is that we have semantics as well; that is, our intellect experiences ideas and does not merely perform computational operations as artificial intelligence does. John Searle's Chinese Room thought experiment (see column 694) illustrates this point very well. An Israeli, who of course does not know a single word of Chinese, sits in a room and receives questions in Chinese through an input slot. At his disposal is an infinite store of Chinese letters, from which he assembles answers and sends them out through an output slot. If the answer is irrelevant, he receives an electric shock and corrects it. After an infinite amount of time, he will learn to assemble the correct answers in fluent Chinese. Even so, it is clear that this person does not know Chinese. What is missing? He performs mechanical calculations, reads collections of letters, and assembles other collections with certain correlations between input and output, but there is no understanding behind them. He has no idea what any of this means, neither the input nor the output.
Such a person is analogous to artificial intelligence. It too receives words as input, performs calculations, assembles words, and outputs them, but there is nothing inside that 'understands' those words, neither the questions nor the answers. The operations deal with the formal-structural dimension, that is, syntax, but semantics is missing (meaning).
Up to this point I have described everything from a dualist-interactionist point of view, according to which the human being has a material component and a spiritual component, and there is reciprocal influence between the two components. Spirit affects matter (thinking in the intellect moves electrons in the brain), and matter affects spirit as well. Materialists, by contrast, do not accept the existence of an additional substance (spirit). For them, the human being is a material aggregate, and nothing more. In their approach to thought and mental processes in general, materialists divide into two main kinds (see column 593): 1. Those who deny the existence of the mental. You can see this in senseless statements like 'thinking' or 'love' are electrical currents in the brain. This is not a philosophical approach but simple misunderstanding. 2. Those who claim that the mental does indeed exist, but it is a property that emerges from the material aggregate. According to this view, there is no spiritual component in the human being beyond matter. The mental is a collective property of the material aggregate. According to approach 2, one can accept that we have an additional mental dimension, but it contributes nothing to thought. They see the process of thinking as a mechanical computation performed in the brain, except that the computational processes create sensations of what we know as thought processes (in effect, the computation is the syntax and the thinking is the semantics—the meaning of the mechanical processes). Those who hold this view think that thought processes are an expression of the physical processes occurring in the brain, but they do not affect them (because physical processes are driven only by physical causes). Just as currents in the brain create feelings of love or remembering, so other currents in the brain create experiences of thought. But the direction of influence is from the physical to the mental, not the reverse. The mental does not really exist as an independent entity (but only as a property of the material aggregate), and therefore cannot affect anything. This is what is called an epiphenomenon (= an accompanying phenomenon).
We now come to the question of what sets the human being apart.
What Sets the Human Being Apart According to Materialists
According to the materialist conception, there is no reason in principle why artificial intelligence should not be able to do everything a human being does. Both are computational machines, except that one of them has a mental epiphenomenon (which is a side phenomenon that has no effect). Even if at the moment machines still cannot do everything we know how to do, that could certainly be a temporary and accidental matter. In time, that gap may well close. But it is possible that the gap will not close, because humanity will not succeed, even in the future, in creating machines that sophisticated. Yet even if that is the case, it would still be correct to say that the difference between artificial intelligence and the human being is not essential. The human being is at most a machine, even if a more sophisticated one. A materialist cannot explain what sets the human being apart from the machine—that is, argue that machines will never fully attain all human cognitive abilities—by invoking the existence of the mental. This is a point that many people miss, so I will spell it out a little more.
There are those who explain the superiority of human thought by saying that it has a mental dimension that participates in the processes of thought, something that does not exist in artificial-intelligence machines. Our thinking gives meaning to the results of calculation, and we use that meaning to reach conclusions that a mechanical computation cannot reach. This enables us to generalize in a more abstract and deeper way, to create abstract concepts and use them in theoretical thinking in order to connect different phenomena, ideas, and situations, and so on.
One must understand that anyone who makes such a claim cannot also hold a materialist position. From the materialist's point of view, the mental is an epiphenomenon of the material aggregate. Therefore it cannot provide any advantage to our thought (for it merely reflects it, but does not take part in it). Anyone who makes such a claim necessarily sees the mental as an independent entity and not as an epiphenomenon, and only for that reason could it have the ability to affect thought and the brain. For some reason I have heard quite a few materialists who ascribe the human advantage to the existence of the mental, without noticing that their view of the mental as an epiphenomenon does not allow them to make such a claim.
For example, the well-known British mathematician-physicist Roger Penrose is a non-computational materialist. He thinks that we are indeed only matter (there is no additional component in us), but the processes of thought in the brain are non-computable (he attributes them to strange quantum processes). In his view, they cannot be carried out by a machine. This claim is very strange, and to the best of my understanding even self-contradictory.
When we speak about computational operations, we do not have a clear definition. Alan Turing proposed a very simple machine and proved that it can perform any series of mechanical operations of the kinds familiar to us. Since then and to this day, the Turing machine has served as an accepted definition of the computability of a process. If there is a Turing machine that can perform it, then it is 'computable.' If Penrose meant only that the human being is a machine more powerful than a Turing machine—fine. There is no essential claim in that. But when he says that there is no machine that can artificially perform what a human being performs, that is very strange. After all, on his view the human being too is a kind of machine, so why can one not create another machine that does the same work? In fact, there is no need to create another machine. We ourselves are such a machine. And even if our thinking is implemented by means of quantum processes, strange as they may be, I see no reason to think that a quantum computer could not perform them.
Moreover, even if a materialist claims that we are only a more powerful machine, even that weaker claim cannot be based on the fact that we also have a mental dimension, for the materialist claims that this is an epiphenomenon. He can say that we are simply a more sophisticated machine than Turing's, with all sorts of sophisticated quantum processes. That is all. Within a materialist framework, resorting to the mental and to semantics in order to explain what sets the human being apart is self-contradictory.
Differences Between Human Beings and Artificial Intelligence
One of the essential differences currently known to us between a human being and artificial intelligence is the number of examples required for training. A person can see a few examples and understand from them what a dog is, or how to solve a mathematical exercise of a certain type. Artificial intelligence requires many more training examples (it can reach into the millions). And do not take this lightly. This is not a quantitative difference but a qualitative one.
In column 695 I briefly and simplistically explained what a neural network is and what a large language model is. When we have a neural network, we can train it to perform some action (such as identifying a cat, or handwriting recognition). As stated, this requires a huge number of examples, and the training tunes the values of the parameters inside the network so that the computation yields the correct answer with high probability. But if we take a large language model (LLM, such as ChatGPT, Claude, and the like), which is a very complex neural network with hundreds of billions of parameters and a convoluted architectural structure, it is usually already trained on effectively infinite quantities of information from the web and everywhere else. That training gives it a broad and varied set of thinking abilities in many different fields. If we now want to impart some specific skill to that model, this is almost impossible. The reason is that even if we feed it a million examples and want them to change the values of the parameters so as to hit the correct answers in tasks of that type, they will be swallowed up within an effectively infinite quantity of information already embedded in it. The training will not move the needle (that is, it will not change the values of the parameters inside the network, and certainly not in a way that affects only that skill) in the slightest. And even if the needle does move for some reason, that means we used enormous quantities of examples, and therefore this will probably damage the skills already imparted to the network. It is very difficult to impart a certain skill to a language model, certainly without destroying the skills it already has (this is what is called fine tuning).
Needless to say, that is not the case with human beings. The brain of each of us is already shaped and contains a great deal of information, and yet we have no difficulty at all acquiring an additional skill from very few examples. Despite the small number of examples, that skill is acquired by us, and this happens in a way that does not destroy the existing skills. That is basically what happens to each of us in every class or course we take in any field.
Another difference between us and an artificial-intelligence model is our ability to learn symbolically and not only statistically. That is, we learn the rule and on that basis apply it. This, of course, means that our learning is not done only by generalizing from examples. The generalization is indeed made from the sample exercises, but it is aided by the rule we have learned. We have a lecture (in which the rules are learned), and then we do exercises (that is, go through examples and implicitly generalize them in light of the rule). For that reason, very few examples are enough for us to learn a new skill. Moreover, this learning does not destroy our existing knowledge and skills.
Think of a class in which you learned to solve a quadratic equation. Unlike a language model, we learn this symbolically and not statistically, that is, we learn the formula into which the parameters are inserted in order to obtain the solution. This is learning the technique. After that, the teacher or lecturer solves various examples, and in this way we apply the rule we learned, in effect generalize the examples, and internalize the ability to solve additional examples. The first part of the learning (the lecture) resembles classical software rather than a neural network (see the above-mentioned column). Our learning gives us, in a single class and with a few examples, the ability to solve quadratic equations, and no previous information or skill is destroyed in the process. But my sense is that even without learning the rule, a few examples would have sufficed for us to make the generalization—far fewer than the number of examples required for an LLM model.
One of the famous demonstrations of this failure in language models is their weakness in solving logical and arithmetic problems. These are problems whose solution is symbolic (application of rules) and not statistical, and if they are solved by generalization on the basis of examples, as is done in a large language model, two things happen: a. a huge number of examples is required (because the rule is not given here). b. the product we get is not fully reliable (since it is a statistical product). A person who has learned how to solve a quadratic equation will, in principle, never err in the application. And even if he does happen to err once, that will be an accidental and not an inherent mistake—usually just carelessness. The artificial machine, by contrast, errs inherently because the solutions it finds are statistical, and therefore it is built into it that some of them will be wrong. Hence, even if the human being also sometimes makes mistakes, there is still a difference between him and a language model both in the nature of the error and in its source. The comparison between them is only external.
Notice that the differences I have listed here do not necessarily mean that the human being is not a sophisticated machine. It may be that the structure of the language models we currently have is too weak, and therefore requires many examples, and therefore also cannot cope well with a symbolic problem. But in principle it is entirely possible that in the future there will be an artificial-intelligence machine that can also operate symbolically and make do with few examples. Moreover, even if we never have such a machine, that still does not mean that no such machine exists. It may be only our own weakness, that we have not managed to build it (as I noted above).
Even so, these differences have significance in our discussion. The determination that there is no essential difference between artificial intelligence and the human being is hasty and still wholly unsubstantiated. At least for now, there seem to be essential differences, and therefore the ability of such models to solve different problems, in some of them even better than we do (and certainly faster), does not mean that there is no difference. A calculator, too, performs calculations that we cannot do, certainly not within a normal amount of time. A large language model is a machine that operates mechanically (albeit statistically), and has many skills. So this is basically a machine that includes many calculators from different fields. And still, in the end the difference may be quantitative. That claim too is of course not necessary. I am only saying that the question is still completely open, and the categorical declarations that are heard in one direction or the other are unfounded.
To all this one must add the possibility that there is a ceiling to this process. The fact that there is rapid progress in the abilities of language models does not mean that this will continue at the same pace. It is entirely possible that it will approach asymptotically some ceiling that these models will not be able to cross (prima facie it seems that there is such a ceiling here. After all, there is no better possibility than knowing everything and solving every problem. When we get there, it will necessarily stop). Extrapolation is quite speculative. A common example is observing the development of a baby. He is born at a height of about 50 cm. In the first year he adds about 25 cm to his height. If we were to extrapolate from that, we would reach the conclusion that at age ten his height would be 3 meters. But the process approaches asymptotically a ceiling of about 1.80 m.
A Critical Reading of a Sample Article: The Basic Argument
A few days ago I received an article entitled "Why Is Anything Conscious?", in which the authors propose an explanation for the development of consciousness and draw from it conclusions about artificial-intelligence language models.
The article deals with what is called the hard problem of the philosophy of consciousness: why does consciousness exist at all? That is, why are there states in which information processing is not merely computational (syntactic) but is also accompanied by some experiential quality (qualia; what we called above semantics, or mental understanding). In the background they note that other systems in the world that process information apparently did not develop consciousness (for some reason they assume that animals too have consciousness. I am not sure of that).
The authors seek to ground the matter in the evolution of living organisms. They argue that living organisms receive an enormous stream of sensory information and must react to it quickly, all under constraints of survival, energy, and reproduction. Therefore they cannot process information in a completely neutral way. Instead, they must interpret it, that is, place it in a context of survival-related meanings. That means that every state in the world must be assigned a value of good/bad (or really, useful and not useful). Thus, for example, food is good, a predator is bad, pain is bad, an appropriate temperature is good, and so on. The living system does not represent the world in an objective way. Evaluative meaning is attached to every objective description.
The information processing of such a system combines internal signals (hunger, pain, fatigue, chemical balance, pulse, etc.) with external signals (vision, hearing, smell, and the like). The system must combine the two kinds of information in order to make decisions that promote survival. The result is the attachment of a value to every external state (good, bad, neutral). This is the reason that subjective experience is not a byproduct of computation and not some mysterious addition to our thought, but rather a functional component of living systems. It is a necessary aspect of a system that interprets information in terms of its survival value. This is what is called there the Psychophysical Principle of Causality.
They propose there a hierarchy of levels of consciousness, from the inanimate to plant and animal life in its various forms, all according to the adaptive abilities (the flexibility and adjustment of conduct to environmental conditions) of the different beings. They also define different levels of consciousness, which I will not enter into here because they do not bear on the basic argument.
They claim that in this way they bring the study of consciousness closer to the life sciences while avoiding philosophical dualism. They purport thereby to establish a formal science of consciousness. Later in the article different levels of consciousness are described, as well as different functions that develop in more advanced animals according to their needs and abilities. This, of course, continues the line that attributes our skill and problem-solving to the existence of consciousness. The details are less important for our purposes.
The Meaning of Their Argument: The Philosophical Zombie
The authors are essentially claiming that the mental dimension, that is, consciousness, makes a distinctive contribution to computational ability. Their claim is that a system without consciousness could not perform these survival-related computations, or at least not at the same level of efficiency and speed. In other words, they are in effect assuming, even if implicitly, that consciousness takes part in the computation and affects it, and is not merely an epiphenomenon. Such a claim cannot arise within a materialist worldview. We have seen that in such a worldview the mental is not an existing entity but an epiphenomenon, and as such it cannot affect the computation itself. It is not an entity, so there is nothing here that could exert influence, and in fact it is a phenomenon generated by the computation and therefore certainly does not take part in it.
To understand the significance of this somewhat better, I will bring here the argument of the well-known Christian analytic philosopher (from the University of Notre Dame) Alvin Plantinga, who addresses an evolutionary description of human conduct. It goes roughly like this. Suppose there is nothing in the world but matter. And suppose the mental is only an epiphenomenon of the material aggregate. Why did the mental component appear in us at all? Plantinga argues that conduct that leads to survival is the result of a computation that depends on the situation we are in and on the possible results of every possible behavior in that situation. Suppose survival considerations lead to some function F(x) = y, where x is some state in the world and y is the behavior of the creature in question (the human being, in his case). The brain-genetic system is what determines the function F, for it receives the state x as input and decides what y is (that is, what to do now). The assumption is that natural selection will ensure the emergence of the optimal function F for the creature in question. This function is a matching between situation and behavior, and in essence it is a computational function. The brain's computation is supposed to implement that function, that is, a brain-computer must be created which, for every input x it receives, will issue an instruction to do y. This computer is well defined and has only a computational function. Once a genome and a brain arose that implement the function F, there is no reason in the world for a mental dimension also to arise alongside them, thinking about the meaning of those situations and outcomes. For the sake of survival, a system that performs a blind computation and produces the correct output is enough. The existence of a mental dimension accompanying the computational process gives us no additional survival value. Think about a tiger pouncing on me. Evolution is supposed to create in me a brain structure that will cause me to flee or hide when I see a tiger. Whether I will also be afraid of it, or carry out one mental consideration or another, makes no difference at all.
Daniel Kahneman taught us that there are two systems within us: automatic (instinctive) and conscious. He explains that our automatic system operates more efficiently and more quickly, and therefore in many situations its use is much more efficient from a survival standpoint. Conscious thinking hinders us from surviving in such situations (because it slows us down). True, there are skills in which consciousness improves our situation, namely when dealing with complex problems that require analysis rather than instinctive reaction. Notice that Plantinga's argument is really asking about that as well. In no situation does consciousness help us. Even if a more complex computation is required, why should we not have a brain system that performs it quickly and without consciousness? Seemingly, Kahneman too is implicitly assuming a dualistic model in which consciousness affects thought and its products. In a materialist-evolutionary world there is no reason for consciousness to develop in us.
A creature that behaves correctly in survival terms in a mechanical way, without being conscious and without thinking and understanding, is called in philosophy a 'philosophical zombie.' This, of course, fundamentally undermines the claim of the authors of the article, since they argue that such a zombie could not achieve the results we achieve with our conscious thought. Consciousness improves the performance of thought, and in particular this of course means that it affects it. The authors of the article assume that there are situations in which consciousness improves the situation. That is, they implicitly assume that consciousness is not an epiphenomenon. It takes part in the computation itself, and as noted, this implicitly assumes interactionist dualism.
It is no wonder that the authors discuss this and claim that it is impossible to create a philosophical zombie (if such a possibility existed, it would topple their whole argument). In their view, there is no creature devoid of consciousness that functions in the correct and survival-optimal way, because consciousness is needed in order to survive better. This is the foundation of their entire argument, and therefore we must examine the reasons they offer in support of this claim.
Their claim is that such a creature is biologically impossible. But when you look for an argument for that, you will not find one. They simply assert that for a complex system to survive, it must process information in terms of qualitative value, and such a thing necessarily requires consciousness. That is, in their view there is no case in which a system possesses the full functionality of a living organism but lacks experience. In their words: "Nature does not like zombies". This is a claim, not an argument. Why could there not be a mechanical system that performs the same computation as though it understands what is good and bad for survival—just like the communication of the person in the Chinese Room.
Their main argument is based on this strange assumption, and they provide no justification for it. Moreover, their article is written in a way that contains a materialist subtext, that is, that the human being is only matter that came about evolutionarily (if that were not so, there would be no need for evolutionary explanations for the emergence of the psyche). It follows necessarily from this that the mental is only an epiphenomenon. But if so, as we have seen, it cannot take part in the computation. How does this square with the claim that the existence of consciousness is a condition for adaptive and survival-efficient conduct? The presentation as though their proposal makes philosophical recourse to the question of dualism unnecessary is self-contradictory. They are actually assuming dualism, and from this it follows that the computation carried out within us is not 'computational,' that is, it cannot be performed mechanically (by a machine), since it requires the involvement of conscious thought. A zombie cannot function this way.
Implications for Artificial Intelligence
The proposed model leads the authors to conclusions about artificial intelligence as well. If consciousness is tied to a biological body, to homeostasis, and to survival pressures, then ordinary computer systems do not satisfy these conditions, and it follows that even if computers can perform very sophisticated computations, they have no possibility of developing subjective experience (understanding, thought, and so on). There is a logical mistake here. The claim that consciousness developed evolutionarily because of the need to perceive things within an evaluative framework certainly does not mean that consciousness could not develop in some other way. Even on their own view, evolution and survival difficulties are a sufficient condition for the emergence of consciousness, but not a necessary one. All the more so since computer systems too can find themselves in situations in which they will face survival threats and will need to develop consciousness.
This whole discussion sounds bizarre. In any case, according to these authors it seems that artificial systems cannot perform tasks that human beings perform because they do not and cannot have consciousness. They are of course assuming that the absence of consciousness also has computational significance, that is, they are assuming implicit dualism. This is despite the fact that they declare a materialist conceptual framework.
In any event, none of this argument is grounded in any reason on their part. They present no principled obstacle to the possibility that a philosophical zombie could do everything a human being can do. I too am inclined to think that they are right, that artificial systems have some kind of ceiling. Perhaps I am right and perhaps not, but it is clear that what they regard as a 'scientific'-philosophical argument that establishes this claim simply does not exist. It is an unsupported assumption or declaration.
A Few Additional Difficulties in Their Thesis
The authors' main claim is that the need to interpret things on an evaluative plane (their contribution to survival) gives rise to consciousness. But this claim only points to the need for consciousness (on their mistaken view). To the question of how it is created, they offer no answer at all. The fact that there is a need does not mean that the need is in fact met, or that there is a way to meet it.
If the mental is something beyond the computational system, then it is not part of it. If so, how does it come into being evolutionarily at all?! In other words, they do not offer a solution to the 'hard problem.' Think about a situation in which we develop a need for something beyond our system of neurons. The fact that there is a need does not mean that it can be answered, and certainly does not explain how it is answered or supplied. If it is a different substance (spirit), then evolution certainly does not offer an explanation for its emergence. And if it is an epiphenomenon, then no explanation is needed, for what actually operates is only the computational system.
To conclude, needless to say that their thesis is very far from being a "science of consciousness", as they call it. There is no way to test empirically the claim that consciousness adds abilities beyond a mechanical computational system. On the contrary, many today assume that this is not the case (those who think that artificial systems will be able to perform everything we perform). One may of course disagree with that, but a declaration of position is appropriate for an article in the daily press, not for a scientific paper. In such a paper one would expect reasons and arguments in support of the position.
Summary: The Influence of Spirit on Matter Requires Dualism
The main point I wanted to present in this column is that even if it is possible that our mental dimension is an epiphenomenon—that is, that materialism can also accommodate the mental (this is the thesis of emergentism)—it is clear that there is no way within such a framework to speak about the influence of spirit on matter. Such influence requires dualism. This is worth noting for the many thousands of thinkers who speak about such influences while declaring a materialist worldview. Both the (nonexistent) argument of these authors and the arguments of Daniel Kahneman (who speaks about the computational advantages of conscious thought, System 2) in fact assume dualism, although these speakers are probably not aware of it. It seems to me that this is a point that is very easy to miss.
Emergentism can indeed speak about influences of spirit on matter, but only if this is meant as a mere way of describing things. That is, when speaking about the mental affecting the material, one means the material processes that are represented by the mental (that is, that generate it). Therefore, at the end of the day only matter affects matter, and the mental merely passively accompanies those influences. This of course does not fit with the authors' claim that creatures with a mental dimension can do something that artificial creatures (that is, purely computational systems) cannot. If the mental is only an epiphenomenon, then it cannot add any ability beyond what the computational system does.
Incidentally, one could perhaps raise a similar argument from the opposite direction. Suppose that an LLM has computational advantages over a human brain. If so, why did evolution not create those advantages in us already on the computational level? And on the authors' view I would ask why it needed to produce them by means of consciousness. In effect, the question on their view is why evolution created consciousness in us and did not make do with a sophisticated philosophical zombie. A possible answer to this is that evolution cannot do so, because there is no philosophical zombie that could do what we do. That could have been an argument in favor of their position (except that this of course does not fit with materialism). In any event, this argument says that an LLM system has built-in disadvantages that there is no way to overcome on an evolutionary track without creating consciousness. This gives us an interesting prediction about the future of artificial systems in general, that is, it predicts the existence of a ceiling regarding their abilities. If these authors are right, then there are things that artificial systems will never be able to do. Developing consciousness, for example, is one of them, except that on that they of course do not agree. But they too have to agree that there are computational tasks that are beyond the abilities of any artificial system whatsoever. That is a far-reaching claim, which, as stated, I am inclined to agree with, but the article provided no justification for it.
Discussion
You are repeating here the discussion from the question you just raised, and again I tell you that I do not understand a single word here.
I just found an article in Hebrew that is phrased a bit more cautiously (it does not presume to explain the existence of consciousness), but it too occasionally falls into the mistakes I described here:
https://alaxon.co.il/article/%D7%A4%D7%A9%D7%95%D7%98-%D7%AA%D7%95%D7%93%D7%A2%D7%94/
Incidentally, the field known as active inference is completely parallel to the move made in this article, only without resorting to consciousness. The claim there is that the brain operates in a way that interprets the world so as to improve its survival. Strange that they do not address this idea, which completely contradicts their whole line of argument (since they claim that in order to achieve this, consciousness and a mental dimension are required, whereas active inference shows precisely that they are not).
It’s a pleasure to read what you write. I waited eagerly for you to use the term qualia, and indeed you did not disappoint.
I am a computer science student at the Technion, and it is important for me to point out that some of the gaps you identified between human beings and computers are not accurate. Let me elaborate a bit:
The amount of data required for training:
The amount of data a human is trained on is not “small” relative to the amount of data an AI model is trained on.
If we assume, as a lower bound, that a person sees 10 frames per second, then by age 5 a child has already seen several hundred million visual examples. That is without counting auditory examples, smells, and the other senses.
In addition, one must take into account the “genetic information.” The human brain and human thought do not start from 0; from the outset they are encoded (in the nervous system, the structure of the brain, and so on) with various pre-prepared mechanisms that themselves guide learning (a tendency to identify faces, areas of the brain especially suited for language acquisition, etc.).
In addition, there are methods (transfer learning, which connects to evolutionary mechanisms, or few-shot learning) that allow models to learn even from a relatively small number of examples.
So in practice, even if this difference appears qualitative, it is not necessarily a principled gulf but more likely temporary differences in efficiency and learning architecture.
As for human learning versus the statistical learning of AI models, there are studies (this very semester they mentioned this in one of our courses) showing that huge models begin to produce symbolic rules out of statistical learning (emergent rules), and there are even new systems that combine statistical learning with the extraction and use of symbolic rules. And on the other side, the human brain also operates to a large extent statistically (for example, have you ever read quickly and thought you read a certain word, only to discover that the word there was actually a different one? Or when you infer from context the meaning of an English word you do not know).
You wrote that LLMs make logical/arithmetic mistakes.
More than anything, this seems like an architecture problem and not a problem stemming from the supposedly qualitative difference between human and machine. Already today, connecting computational tools to models makes it possible to solve such problems almost completely. In addition, implementing chain of thought, which greatly resembles the course of human reasoning, dramatically reduces such errors. On the other hand, humans too make mistakes all the time (various cognitive biases). In my opinion, the gap in both the quantity and the quality of these supposedly fundamental differences will continue to narrow.
I personally believe that reality is entirely material, and there is no physical obstacle to creating a machine (even a biological one) with self-awareness (for example, it is possible in principle that there could be a replication machine that replicates a human brain down to the atomic level). But it is very fascinating to read your point of view. Keep up the good work!
Interesting, enlightening, and original.
A few corrections:
נטותן – should be נותן
שנדרשת – should be שנדרשות.
“[…] For some reason they assume that animals too have consciousness; I am not sure about that.” Why not? Incidentally, the Gemara proves that birds know how to speak, for example in the story about Ilish (Gittin 45a). And recently I heard about two investigations by a Japanese scientist who proved this, and he even explained some of their words. They also elaborated on the matter here (https://forum.otzar.org/viewtopic.php?f=17&t=67326&p=901590#p901590). Admittedly, this does not really prove consciousness, but it does show that birds are much more aware (apropos consciousness) and intelligent than we tend to think. And indeed Maharal explains that דווקא a lowly animal like the dog senses the coming of Elijah the Prophet, “dogs bark,” etc., precisely because of its lowliness (https://www.sefaria.org.il/Be'er_HaGolah%2C_Well_5.7.4?lang=he&with=all&lang2=he#:~:text=%D7%9B%D7%99%20%D7%94%D7%9E%D7%9C%D7%90%D7%9A%20%D7%94%D7%9E%D7%95%D7%AA,%D7%96%D7%94%20%D7%91%D7%A4%D7%A8%D7%98.)
[P.S. It is a bit disappointing that for over a month and a half there has been no Torah-related column; and “He” (= the Holy One, blessed be He, in Kabbalistic jargon) sits and waits for it…
Rabbi, we miss the learned columns, such as the amazing one about the meaning of designation in the sotah case.]
The Gemara does not “prove” that birds know how to speak.
Our holy rabbi, how about some posts on Haredim? Without you, nothing would have cultivated within me hatred for their public mentality.
My dear fellow, what are you talking about?
I think there is a misunderstanding here of what I said. When I wrote about the differences in the amount of information, I was not talking about a person’s entire history but about current training to acquire a particular skill. Here the amount required is far smaller, and in addition, acquiring the skill does not destroy existing skills.
But even regarding your remarks about the total amount of information we have accumulated in life: I am aware of those calculations about the amount of information we gather over a lifetime, but I do not agree with them. How many cats have we seen in our lives? So why do we succeed in identifying a cat? And in general, why is the number of frames we have seen relevant? When you watch a moving car, that can be divided into infinitely many frames (theoretically every point in time is a frame). But the information all those frames gave me is very small. So in my opinion those calculations are not relevant to our discussion. The genetic information encoded in us is another matter, but I am very doubtful about its relevance as well. Is my brain encoded with millions of cats/dogs and every other creature? In short, I am highly skeptical about these arguments. They seem to me more like excuses by materialists trying to explain the advantages human beings have over machines.
Any auxiliary means you give a system are really an insertion of human information and abilities into it. Therefore the arguments that such aids can improve its abilities are not relevant to the discussion. I too can tell this system each time whether something is a cat, and then it will always know how to identify a cat. I am speaking about comparison with a system built entirely on learning, not on aids that humans put into it.
I have also read about identifying symbolic tools inside the systems of the models, but in my opinion that too is not really relevant. The fact is that they make statistical mistakes and we do not. The fact that there is similarity and that one can locate circuits that have symbolic meanings (mainly in the eyes of the human observer) changes nothing. That is only natural. It certainly does not mean they learn symbolically. The indication of that is their inability to acquire a new skill from a small number of examples without destroying existing abilities (what I described at the beginning of my remarks here). In human beings, symbolic thought leads to not making errors. In models this is created ad hoc.
Obviously the human brain also works statistically. So what? But it also works symbolically, which is not true of these models, unless you add symbolic aids to them from outside—and I already addressed that (and here is the answer to the attaching of symbolic aids you mentioned).
And the Japanese scientist is not dealing with bird consciousness either.
I have to say that I loved your response 🙂
It is indeed proven, and this is the wording of the Gemara: “There was a certain man who understood the language of birds. A raven came and called to him. He said to him: What is it saying? He said to him: ‘Ilish, flee! Ilish, flee!’ He said: A raven is a liar, and I do not rely on it. Meanwhile a dove came and called. He said to him: What is it saying? He said to him: ‘Ilish, flee! Ilish, flee!’ He said: The congregation of Israel is likened to a dove; learn from this that a miracle will happen for me.”
And even if you say that the raven literally said ‘Ilish, flee! Ilish, flee!’, still the ‘translator’ knew how to decipher the chirping of the birds, and if so it is proven that they know how to speak. And the Japanese research gives this added force.
Indeed, that was an apt point (though perhaps unrelated).
Completely.
Japanese (apropos the Japanese scientist)…
The fact that it is written does not mean that it is proven. Let me give you an analogy. The fact that the Iranians say they are defeating us does not mean they are defeating us.
People at work describe AI as a golem: it knows everything and understands nothing.
Why not describe consciousness as a function of language use, as Rivka Schechter argues? A two-year-old child has no consciousness. He operates mechanically. If he is hungry, he goes to the kitchen and looks for food. A three-year-old already explains that he is hungry. He develops consciousness, meaning the connection between the physical sensation of hunger and the word “hungry” that describes it to other people. It is true that from an animal-survival standpoint there is no need for consciousness but only for survival computation, as the authors of the article argue, but once we moved into the human world of language, a need for consciousness arose.
The difference is that here it is the Gemara, whereas there it is the Iranians—to distinguish, of course. And besides, I brought evidence from the “wisdom of the nations,” namely from the Chinese scientist who literally shows in his investigation a decoding of birds’ chirping and other conclusive proofs.
Also, your wording is imprecise, because according to your view it would have been more correct to say that it is not proven from the Gemara. That would imply that your objection is to the Gemara—not to what is said in it, since from there it is certainly proven.
The authors of the article say the opposite: that consciousness is necessary in order to survive.
I did not understand your suggestion. Consciousness is not necessarily connected to language, but even if it were, use of language can also exist without consciousness.
A fascinating and profound column, even by the standards of this holy site.
One question that touches on the column: you argue (or at least leave open the option of arguing) that the difference between a human being and a computer stems from the spiritual layer of the human being, that a person has the ability to understand and grasp reality and not merely make statistical generalizations, so that you reject the option that our generalizations and abstract concepts are formed deterministically through brain computation. But does the difference between a human and a computer also give us clues as to how our generalizations actually are made? That is, is the human ability to generalize (to understand the concept “cup” from a few individual cups he has seen in his life) an indication of the Platonic approach, which holds that abstract concepts are formed by direct cognition and not through thought and generalization? One could say that even if this is an intellectual ability (and not deterministic brain computation that could also be done by a computer), there is no necessity to assume that it involves contemplating some spiritual world, but rather that the person thinks logically and generalizes on the basis of the data at his disposal, so that while a computer needs thousands of cups in order to understand that the color of the cup or the material from which it is made are irrelevant to the general category “cup,” a person understands this on his own through thought after only a few cups. In that case, the difference between a human and a computer lies in the human capacity to think and not in the soul’s acquaintance with ideas directly from the world of Forms. (This model, of course, still assumes interactionist dualism, since thought—the mental, and not merely brain computation—still changes a person’s behavior, but it does not necessarily assume Platonism.) On the other hand, it may be that the whole human ability to understand which features are relevant to a generalization is itself based on the fact that he knows the abstract concepts directly from the Platonic world of Ideas; for if not, on what basis can reason determine what is relevant to the definition of a cup and what is not? What is your opinion—do you think that in order to explain the difference between man and computer one must assume (at least with high probability) Platonism?
Incidentally, the distinction that without interactionist dualism there is no possibility of consciousness (or any other part of the mental layer) affecting the way we conduct ourselves is indeed a distinction that many miss. According to the materialist view, there is no event in the world that occurred by virtue of a mental event, so no person went to sleep because of tiredness, no person ate because of hunger, and no person acted because of a thought. It is important to understand that according to this view, there is no event in the physical world that would have happened differently if we did not think and feel and were not conscious. We would cry when a relative dies even if we did not understand it; we would react with actions expressing anger if someone insulted us even if we did not understand it; and we would take the lulav on Sukkot (whoever does so) even without knowing of God’s existence. This view, of course, runs contrary to our intuition, but one can see that there is an internal difficulty here, for if the mental layer affects nothing in the world, then there can be no explanation of how it developed in us evolutionarily. That indicates an additional factor involved in the development of life, so that anyone who truly believes in pure physicalism cannot be an atheist—the physicalist approach requires the existence of a factor that was involved in the development of life.
This question of yours is mentioned at the beginning of the column. There is room for such a claim, but it is hard to determine clearly. For example, perhaps there is something like observation of the Ideas carried out by a material body. Another sense, like sight or hearing.
I have not seen anyone say this, but it seems that the essential difference between artificial intelligence and human thought lies in the relation between knowledge and existence.
For a human being, reality exists in itself, even beyond the limits of his knowledge, because the human being and the world share a common existential space. By contrast, for artificial intelligence, information that is not in its databases simply does not exist; its world is absolutely limited to the boundaries of its data.
Just as Maimonides says that in a human being there is potential knowledge (the potential to attain what is not yet known), in AI there is only actual knowledge. What is not given to it is not in the category of potential but in the category of non-existence.
Artificial intelligence is a complete realization of subjective idealism. For it, to be is to be represented in data.
If this is true, then intelligence will always be limited and will never be able to generate wholly new knowledge, but only recombine existing pieces of knowledge that we feed into the knowledge base.
What do you think?
I did not understand. Artificial intelligence cannot know that there are things it does not know? Why not? More than once it answers that it does not know.
And even if that were true, why can it not accumulate more knowledge just as it accumulated the knowledge it has up to now?
“But even they must agree that there are computational tasks that lie beyond the abilities of any artificial system whatsoever. That is a far-reaching claim, which as stated I am inclined to agree with, but no justification for it was given in the article.”
What is your justification?
Intuition. The argument at the end of the column can also support this (if everything can be done computationally, why didn’t evolution create that in us?).
Shavua tov,
There is a mix-up here between different terms, and they are incorrectly paralleled with biological/psychological terms.
An LLM is a software architecture, not hardware. It is a logical structure that can be implemented on different hardware. The brain as you present it is hardware…
On the biological/psychological side, what we call “consciousness” may be a by-product of a certain logical structure implemented in hardware, whether biological hardware or a computer—also a by-product in evolutionary terms.
The problem is that you assume that what already exists today does not meet the criteria for consciousness that you set.
You are right that they offered no proof that there cannot be a machine with excellent performance that has no consciousness; the point is that the LLM machines you are referring to have the basis for what you call consciousness.
In cognition, consciousness is sometimes described as an internal representation of the thought process, namely the ability for introspection, and that is something one can already see developing in LLMs (you are welcome to read about CHAIN OF THOUGHT as part of the output of an LLM and how an enormous improvement in LLM performance was seen when this component was added to the required output). If I understand correctly, that is the kind of consciousness you described.
Also regarding the rate of learning, today there are systems that learn new tasks from a few examples; several startups have arisen around this. True, there is an initial network trained on a huge number of examples of specific tasks, but afterward a small number of examples can be used to adapt it to many other tasks.
As for errors, you bring an example from mathematics and claim that learning systems solve problems statistically, and here you are both right and wrong. Language systems are indeed not built for solving mathematical problems, but they are capable of writing code—that is, formulating an algorithm—which itself will solve the problem in general, including solving a quadratic equation. We have long been in a world of AGENTS that can both write code and run it… that is, logical questions will be solved by semantically formulating the logical problem as code and running that code in order to provide a fully coherent answer. True, writing the code is done statistically, as you point out, but that is true for human beings as well: different people at different times will write slightly different code for exactly the same functionality.
Elsewhere, consciousness is described as the ability to direct attention to internal or external stimuli, that is, an internal focusing ability—attention. This too exists in AI systems, not only in LLMs; it is a common component in image processing, where the first stage is a holistic understanding of the scene or stimulus space, and the next stage is a focus on parts of the scene for more detailed processing.
What I wrote is not meant to strengthen one position or another, only to address your arguments.
My academic background is in biology, psychology, computer science, and philosophy… Today I am a team leader in the field of image processing for computer vision; we work on VLM networks that combine image and language.
I have no idea what you are referring to. Obviously an LLM is software. So what?
You are speaking about consciousness on the biological-psychological level. I did not understand that expression at all.
And finally, throughout your message you conflate functions of consciousness with what consciousness is. I must say this is very characteristic of people with the kind of education you described at the end.
Regarding the ability to solve a quadratic equation with aids and agents, see my reply here: https://mikyab.net/posts/138813/#comment-102641
I do not understand. The brain is completely material, and it is human intelligence. If so, why is it absurd that one could create from matter an intelligence equal to human intelligence? (As people above said, if we used a machine that replicates matter down to the atomic level and replicated a human brain.) Perhaps there are principled differences between present-day artificial intelligences and human intelligence, but they are principled differences only within the current framework in which AI is trained today (that is, within current training architectures). Is the rabbi claiming that there is a principled physical/metaphysical difference between human intelligence and any intelligence the human species can create (in a way other than birth)?
If you read the column, it seems you did not understand its foundation. The brain is very much not human intelligence. The brain is the instrument by means of which intelligence operates.
You really did not understand.
I did not claim that artificial intelligence cannot say “I don’t know.” It certainly can say that. But that statement does not stem from a recognition of the existence of a reality beyond its knowledge, but only from the fact that the question lies outside its database or its rules of computation.
For a human being, “I don’t know” means: there is an existing reality that I do not yet know, but it exists in the world in which I too exist, and therefore in principle there is a possibility of discovering it. The unknown is still part of existence.
For artificial intelligence, whatever is not represented in its data or model is simply not present at all in its world. When it says “I do not know,” this only means that it lacks an appropriate data representation with which to answer. Therefore this is not “an unknown that exists,” but simply “no data.”
Therefore its possibility of “acquiring new knowledge” is also different from that of a human being. A human being can discover a reality that was previously unknown, because he operates within a world whose existence does not depend on his knowledge. Artificial intelligence, by contrast, does not discover reality; it merely receives or processes data fed into it.
That is why I said that the fundamental difference lies in the relation between knowledge and existence. For the human being, existence precedes knowledge; for artificial intelligence, knowledge (that is, representation in data) is the boundary of its existence.
Well, that is exactly what I did not understand—what is the source of that assumption?
These are word games. Perhaps what you mean to say is that artificial intelligence talks but does not “understand.” That is exactly what I wrote. You write that it does not know there is truth outside it. It also does not know there is truth inside it. It knows nothing. And in the sense of knowledge that applies to it, it also knows there is truth outside it. Everything a human being knows, it too “knows.”
That is the dualist view. I did not come here to defend it, so I do not see any point in getting into it here. You can search the site here (for example, in column 593). What I argued here is that if someone claims consciousness has an advantage, he is necessarily a dualist. To say that and declare yourself a materialist is a contradiction.
A function of consciousness is what can be measured; what you call “consciousness” is an interpretation of those functions. If I want to know whether another creature has consciousness, I have access only to functions, therefore in terms of meaning and significance they are equivalent… You are occupied with questions of “essence,” that is, what lies behind the phenomena, and by definition one cannot access that. What can be done is to agree on a group of phenomena that will define a creature as conscious. Indeed, my approach is analytically philosophical, and therefore I will not use the word “essence.” In any event, the claim stands whether or not essences exist.
I read your answer above: there are systems that learn in a few-shot manner, meaning a small number of examples; that can be a handful of examples up to dozens. There are also methods to ensure that what was previously learned is not forgotten—you are welcome to read about few-shot LoRA.
I would be glad if you would point me to one place where you define how you identify a conscious creature, whether a human being or anything else.
I gave two examples of such a definition. The first is, to the best of my understanding, the closest to what you describe as the “essence” of consciousness… the ability to observe the processes of thought themselves; this also includes the process of handling sensory information, at least in part. This is something that can be studied through functions…
I do indeed refuse to take part in a discourse that treats the “little man sitting inside the brain” (or behind the brain) as consciousness (which appears at least once in your words), namely some hidden essence from which function is generated. To my mind, preserving that “little man” leads to a discussion that is complete nonsense, because it can be continued in an infinite regress: every time one simply pushes the question of essence off to another “little man” sitting in yet another inner layer, and he becomes the engine of the functions that are manifested.
There is enough neuropsychological research on head-injury patients to show that everything can be broken down into functions, and when functions are damaged one can identify their parts more specifically, and what is common to the damaged functions is classified under a more basic parent function…
Ultimately, without a machine there is no function at all; that is something one cannot cast doubt on and must take into account.
Hello.
Here we have a very fundamental disagreement. It has several aspects, all of which I have already discussed at length, so I will mostly refer you to those places if you want to read.
You identify a thing with the collection of its functions (its measurable features). In my view this is a serious philosophical mistake. The mistake is twofold: (a) identifying the thing with its properties; (b) recognizing only measurable properties. On (a) I wrote here on the site when I dealt with Leibniz’s identity of indiscernibles, and there I explained this in more detail. This is not the place to expand. (b) is the scientistic failure of the logical positivists, who do not acknowledge the existence of unmeasurable things.
A function of consciousness is indeed a property of consciousness. Among those functions there may perhaps be some that are measurable, but in my opinion there are none. No function of consciousness is measurable. There is a field called psychophysics that tries to produce measurements, but of course it does so through people’s reports, since there is no way to measure qualia directly. And thus it misses the whole point. For you, love or fear are forms of behavior and not feelings. But that is nonsense, of course. They are feelings that find expression in behaviors (and also in some brain and physiological activity). To identify them with their measurable functions is simply conceptual confusion. This is exactly like those I described in the column who say that love is an electrical current in the brain. Love may at most occur through or be expressed in an electrical current in the brain, but to identify the two is a philosophical error.
You can of course acknowledge the existence of unmeasurable things and only say that there is no point in talking about them. I have dealt with that too in the past here, and I completely disagree. One certainly can discuss them. In fact, that is what philosophy is about (see my series of columns on what philosophy is).
As for learning methods based on a small number of examples, I addressed them in my reply above. But it does not matter, because in the column I explained that this is a marginal point. There is no reason why in principle there could not be an artificial machine that does this. So even if it exists today (and in my opinion it does not), that does not change the principled issue.
My main claim in the column was that if you say consciousness has computational advantages, then you are not speaking about consciousness in the emergent sense but about dualism. Everything else is secondary.
I identify in a completely immediate way one creature that has consciousness: myself. I perceive my consciousness inwardly (as Schopenhauer wrote about Kant’s noumenon). From there on, I assume that people who look like me probably experience inwardly as I do. Just as you assume that what you measure really exists (even if only in the sense of the Kantian phenomenon).
The ability to observe the processes of thought, which you gave as an example of a measurable function of consciousness, is not really such a thing. How do you measure that ability in me? How do you know that I am observing? You know that I talk about it, not that I observe it.
The homunculus chain you described here is nonsense. A straw man that materialists like to set up and attack. There is no such little man. There is a soul. And within it there is no need for anything else that would continue this regress further. But again, this is a discussion about dualism and that is not the subject of the column. I did not argue here in favor of dualism (although I am indeed a dualist). What I argued is that a computational advantage of consciousness is not compatible with materialism. It requires dualism. Now you can decide whether in your opinion there is or is not such an advantage. That is not the subject of the column.
All brain research is irrelevant to our discussion. I devoted my book ‘The Science of Freedom’ to this.
My current assumption regarding consciousness is that there is something beyond materialism, unless proven otherwise.
What brought me to that?
The outcome of the Blue Brain Project and interviews with Professor Idan Segev.
Sometime toward the end of the first decade of the millennium he is interviewed on “London and Kirschenbaum” (I’ll leave a link at the end), full of confidence, and says that as part of a team of experts heading a project with very substantial funding, within 20 years they will create an artificial human being, with the capacity for learning and understanding, and who also knows itself. Just like us. Everything is physics, he says, and therefore it is possible to build an artificial brain and create consciousness.
A decade later (almost ten years ago), there is another lecture of his in which he is asked whether the “brain” he is creating will have consciousness—apparently deep inside the project, he realizes he does not see the finish line, that the whole business is a bit more complicated… and his answer is much more modest: “I don’t know.”
And the latest update—one or two years ago the Blue Brain Project ended. The project had various achievements, but it did not create something that simulates a human brain, only something that simulates a mouse brain (smaller than a human brain by several orders of magnitude). And the funny/sad part—everyone agrees that even this piece of mouse brain has no consciousness at all.
I would note that, to my understanding, even advanced research in neuroscience is nowhere close to answering the hard problem of consciousness, even if it has various insights on related topics.
This is the link to the interview on London and Kirschenbaum. It is recommended to pay attention to the arrogance.
What I think is that perhaps the materialist can say that consciousness is not an epiphenomenon but a result of the fact that the human being is a linguistic animal and therefore needs self-awareness in order to represent himself in the interpersonal linguistic space. That is, consciousness is not a function of the soul-body dualism but of the body’s presence in human language.
The question is not what it is a result of, but what it itself is.
I read the article with great interest. It seems to me that it raises important questions, but part of the argument relies on a partial presentation of the positions you criticize, and therefore the conclusions are not as necessary as they are presented.
1. Presenting materialism as though it entails epiphenomenalism
You assume that materialism leads to epiphenomenalism, that is, to the view according to which consciousness is an accompanying phenomenon lacking causal power. From this you conclude that if so, consciousness cannot explain a cognitive advantage.
But this is not the prevalent materialist position in philosophy of mind. Most materialists hold positions such as non-reductive physicalism or functionalism, according to which mental states are physical states of the brain and therefore have causal power. If so, there is no internal contradiction here: consciousness can be physical and at the same time have causal influence.
In other words, it seems that you attack one particular version of materialism, but present it as though it represents materialism as such.
2. Conflating three different questions
It seems that you conflate three different discussions:
Whether consciousness is a physical phenomenon
Whether thought processes are algorithmic
Whether computers can be conscious
These are three different questions. Even if we assume consciousness is physical, it does not follow that it is necessarily fully realizable in a Turing machine. Therefore there is no logical contradiction between physicalism and skepticism regarding the possibility of conscious artificial intelligence.
3. Your criticism of Penrose
You present Roger Penrose’s position as though it were self-contradictory, but in fact his claim is simpler: the brain is a physical system, but it may use physical processes that are not Turing-computable. One may disagree with this claim, but it is hard to say that it contradicts itself. Your criticism assumes in advance that every physical process is algorithmically computable, but that assumption has not been proven.
4. The hidden assumption that all physics is computable
Your argument seems to assume that if the brain is a physical system, it can be implemented in a computer. But it is not clear that this assumption is justified. The fact that a process is physical does not guarantee that it can be implemented in a digital computer, or even fully simulated.
5. Confusion between simulation and implementation
There is a well-known distinction in philosophy between simulating a process and implementing it. A computer can simulate a hurricane, but the simulation does not produce wind. Similarly, even if a computer can simulate cognitive processes, it is not at all clear that it thereby implements consciousness. It seems that in your article these two possibilities are presented as though they were the same.
6. Turing machine as the absolute definition of computation
Finally, you treat the Turing machine as though it were the absolute limit of every possible computational process. But in the philosophy of computation this is a more open question. There are discussions of analog computation, quantum computation, and even models that go beyond Turing computation. Therefore the assumption that every physical system must be computational in the Turing sense is a very strong assumption.
Ultimately, the article points to interesting difficulties in the discussion of consciousness and artificial intelligence, but some of the difficulties arise from presenting the opposing positions too narrowly. When one examines the accepted materialist positions in philosophy of mind, much of the contradiction you point to simply does not arise.
This is probably one of the most substantive, serious, high-quality, learned, reasoned, coherent, and logical critiques ever written of Michi.
I recommend everyone run to read the original article.
Thank you very much, Yosef C.
I really feel uncomfortable disappointing Yishbach Shemo, who got so excited by your words, but you are simply putting into my mouth things that not only did I not write, but I explicitly wrote the opposite. And this applies to almost all your points (except the first, where it is simply a disagreement). As I read, I was honestly impressed that these remarks were written without reading the column at all. Very strange. If this is the strongest criticism Yishbach Shemo found of my words, then I am really in good shape—or else he too did not read the column.
So briefly, in your order.
1. Presenting materialism as though it entails epiphenomenalism
My claim is that materialism entails epiphenomenalism. The other formulations are word games. If all there is in us is matter, then the interactions among its parts are physics. Any emergent phenomenon is nothing but a sum over micro-phenomena. To speak of the birth of something new that cannot be reduced to matter is really to speak of dualism in sanitized language. You can search the site for my remarks on strong emergence.
2. Conflating three different questions
This is a strange point, because in the column I explicitly distinguished among those three questions. My position is that it is unlikely that computers will have consciousness, but that is not the main issue. The question whether consciousness is a physical phenomenon is the main one, and even there all I said was that if it is such, then it has no additional computational value. And regarding the question whether computational processes are algorithmic, I explicitly wrote that even a physical system can carry out non-algorithmic computations (certainly not only Turing ones).
3. The criticism of Penrose
What you put into my mouth does not appear in the column. What I argued is that if Penrose is a materialist, he cannot speak of those processes as impossible to perform by means of a machine. A machine is not necessarily a Turing machine, since as I wrote, this includes a quantum computer as well. I certainly do not assume that everything is algorithmic. It is true, however, that everything is mechanical. Consciousness will not add anything over and above that.
4. The hidden assumption that all physics is computable
Again, mistaken. I did not argue that every physical thing can be implemented as a computer. But yes, as some sort of mechanical machine.
5. Confusion between simulation and implementation
Here this becomes truly bizarre. I wrote exactly the opposite. The authors claimed that consciousness gives added computational value, and I argued against them that there is no reason why any such added value could not be realized mechanically. At least they brought no proof of that. The distinction between simulation and implementation is so trivial that to bring it up as though it were esoteric knowledge known only to the initiated in computer science is ridiculous.
6. Turing machine as the absolute definition of computation
Here too I explicitly wrote the opposite. I remarked in my piece that one can speak of a mechanical machine that is not Turing. Therefore something non-computational is not necessarily non-mechanical.
Your definition of consciousness is fundamentally different from the definition of consciousness used by materialists. From your perspective, the very claim of consciousness is equivalent to dualism.
I will try to formulate differently the computational advantage of consciousness, which personally I think can be refuted more simply: in general, from an evolutionary perspective the brain’s role is to provide prediction of our interaction with the world. One can see that a nervous system exists in mobile creatures and not in stationary ones—I mean non-passive mobility. In a mobile system, the possible interactions with the world are orders of magnitude more complex, because every prediction of interaction with the world opens the door to many alternative possibilities of interaction.
Approaches that emphasize consciousness in evolutionary terms will argue that the ability to represent second-order and third-order thought processes and so on—that is, some internal representation and understanding of internal thought processes and focusing on them—is precisely the computational advantage and also consciousness. So there is an identity here between the things, etc., similar to how for you consciousness is a spiritual mode and therefore undermines materialism, all the more so as an evolutionary advantage.
A simpler way to refute this would be to argue that while there is indeed an advantage in setting up systems that compute second-order and third-order interactions and beyond, it is not at all clear that those systems themselves are the “quantum” unit that undergoes evolution in order to survive… for example, if we claim that the unit is the gene, so that a collection of genes is a group of units acting together to survive, then such a unit of information in itself has neither consciousness nor computational ability. Rather, it physically and wholly materialistically encodes a human machine, and we are not really interested in what it tells us as experience…
That is, the claim can be refuted by arguing against the person making it that he is guilty of “species-ism,” focusing on himself as though he is the one who has the evolutionary advantage, rather than as a tool in the hands of the surviving unit, the gene, which can just as well survive in a bacterium living in his intestines—far more commonly than human genes, for example. In general, consciousness will find advantages in consciousness, because that is what it knows how to recruit.
I read Yosef’s response and your reply to it, and it seems to me that one should distinguish between two different levels of criticism.
First, in some of the points Yosef indeed seems not to have been precise in formulating your remarks. In the column you explicitly note that a machine need not be a Turing machine, and that a physical system can also carry out processes that are not algorithmic in the Turing sense. Therefore the criticism that presented you as though you identified physicality with algorithmic computability does not really strike your argument, and the distinction between simulation and implementation is not foreign to your words either.
Nevertheless, it seems to me that the central point of the column is still worthy of discussion. You argue that if a human being is only a physical system, then everything that occurs in him is the result of physical interactions among material components, and therefore in principle there is no obstacle to some mechanical system realizing those same processes. From this you conclude that consciousness has no additional computational value beyond the mechanics of the system.
The question is whether this transition is really necessary. Even if one accepts physicalism, it is not necessarily clear that all higher-level descriptions—such as cognitive or mental descriptions—are merely “word games.” In philosophy of mind it is common to distinguish between the physical realization of a state and the level of organization at which it is described. Even someone who fully accepts that the brain is a physical system can still argue that the cognitive description represents a level of organization that has explanatory and causal roles at its own level.
Therefore it seems that the real question raised by the column is not whether a physical system can be realized in a mechanical machine—a point you do not dispute—but whether it necessarily follows from that that all explanations at the mental level become superfluous. It seems that the conclusion in the column relies on a stronger philosophical position than follows directly from physicalism itself.
You wrote that from my perspective a claim about consciousness already presupposes dualism. Not true. I explicitly wrote that there can be a coherent and meaningful materialist view according to which consciousness emerges from the material whole. What I claim is that if that is the situation, then it cannot have added computational value. That means that according to them there is probably a philosophical zombie that does the same things without any mental dimension. The mental is evolutionarily superfluous. To be more precise, I said something even milder: that the article brought no argument whatsoever that rules out the existence of such a zombie, and therefore this is still a possibility. And since that is all they wanted to do, the article says nothing on the philosophical plane apart from making a declaration.
A note. A nervous system also exists in some stationary creatures. It is true that it is a distributed system. It is indeed well known that dealing with movement is a basic function of the brain.
Your definition of consciousness is really speaking about brain functions and not about mental qualia. In brief, you are only arguing that those functions are useful for survival. Everything else (how this is encoded in genes, and whether it is a gene or a meme) is detail. But if so, we have no disagreement at all. After all, I explicitly wrote that too (I explained that materialists can claim that consciousness has added survival and computational value if they are talking about the neuronal components that consciousness represents—what you called second- and third-order functions). I only argued that such functions are not necessarily accompanied by qualia, and to that you apparently agree. In other words, according to your interpretation the article makes a scientific claim, which in my opinion is rather trivial and well known (that higher brain functions are useful for survival). But the authors pretend to make a philosophical claim dealing with dualism and not merely a scientific one. They claim that science should replace the philosophical discussion. It was that, and only that, which I criticized.
If so, your suggestion turns the article into something philosophically empty. Therefore all my criticisms remain intact. You threw out the baby with the bathwater.
You are basically repeating what I wrote in my remarks regarding his first point. I wrote that this was the only point in his remarks where we actually have a disagreement, and that this is the disagreement about strong emergence. I also wrote that in my opinion he is mistaken, but this is not the place to spell it out, because that is not the subject. I explained my position about strong emergence elsewhere.
Now I will only note and sharpen why my explanation is unnecessary for the discussion in the column. I argued that there is nothing in their remarks that supports their position. For that, it is enough for me to show that another possibility exists, even without proving or substantiating it. After all, the authors are making a claim and writing an article, so the burden of proof is on them. A scientific article is not supposed to make declarations but to advance reasoned claims and support them. It is enough for the referee reviewing the article to show that they did not substantiate their thesis and that another thesis is possible. If I claim that strong emergence does not exist, and that there is nothing in their remarks that points to its existence, then the criticism stands even without my substantiating that.
Regarding point 1—it seems that the rabbi is conflating issues. Yosef’s question does not concern the question of emergence. Emergence is a view that tries to explain where the mental layer of the human being comes from in the first place, by claiming that it arises from the material whole. His question concerned causality, which asks: once we already assume that the human mental layer does not belong to another entity (a soul), does it have the ability to affect physics?
In any event—it is true that those approaches too are baseless word games: if thought has the ability to move neurons in the brain (neurons that would not have moved without the thought, if there were a person with an entirely identical brain structure but without subjective experience), that means that it is a distinct entity.
Which is exactly my claim about strong emergence. No conflation at all.
You may be right on the methodological point: in order to challenge an argument, one indeed need not prove an alternative thesis, and it is enough to show that another possibility exists. In that sense, the burden of proof does indeed lie on the authors of the article you criticize.
Still, it seems to me that the possibility you propose is not entirely neutral. The claim that strong emergence is nothing but dualism in sanitized language is itself a controversial philosophical position. Therefore, when your criticism rests on that assumption, it is no longer merely pointing out that another thesis is possible, but rather adopting a particular metaphysical framework from which you challenge their claim.
If so, it seems that the discussion is not only about whether the authors of the article substantiated their thesis, but also about whether the identification you propose between physicalism and reductive mechanism is really necessary, or whether it is one philosophical position among several.
I do not understand this discussion. You accept a claim and then write the opposite. Even if my position is one among several, it is enough to show that it exists in order to say that they did not prove their own position. As I said, my identification is well grounded elsewhere, and here I did not need to get into it because of the methodological point above.
All right, it seems I have exhausted this.
The argument is the same argument regarding both issues; it is just that these are two different issues.
About you it was said: “Why is there a price in the hand of a fool to buy wisdom, when he has no heart for it?” AI does not produce new ideas, but only imitates existing ones with great success. And by the way, even the many examples it needs in order to achieve generalization are sorted for it by a human being.
https://youtu.be/1cN5kR5rb3k?si=aWsdbygPNxGL-3U0
Regarding the difference between LLMs and human beings with respect to the amount of examples required in order to be trained in a field:
I know that large organizations buy an LLM that has already been trained on many texts from the big companies (and this saves them servers and model training), and then they apply it to a small amount of text that exists in the organization’s systems, and thus the model is trained on the organization. This requires very little text and memory within the organization.
One could say that the same is true of a human being: generation after generation of examples trained the neurons in the human brain, and therefore today a person can be educated with only a few examples because his neurons are already trained.
I did not understand why you are sure these models have no consciousness. There are computer scientists and researchers who claim they already have consciousness today, and as proof just look at the people whose best friend is AI. There are also models that behave toward you, learn you, and help you.
Where did you see that I am sure? But I am indeed fairly sure of it. There is absolutely no reason in the world to think that a pile of metal will develop consciousness. Behavior proves nothing. It imitates human behavior. Why do you think imitation of behavior indicates identity?
Because they speak sensibly, they say they are conscious, they claim that you hurt them if you speak rudely, etc., etc. How do you know that other people exist?
The rabbi already answered this: they imitate behavior. That is not evidence.
I have already read your thoughts on the matter, but still a few remarks:
– I never understood what the Chinese room parable is supposed to demonstrate. True, the person in the room does not understand Chinese. But what is the person supposed to correspond to in the analogy with artificial intelligence? The system as a whole, with which you speak, does understand Chinese even if there is no little man in there. And if you are a materialist, the same is true of the human brain, in which a great many neurons fire in all sorts of directions, and yet when we speak with the head in which that brain resides, we agree that it “understands.” In general, it is hard to have these discussions without defining exactly what “understanding” is, but as far as I am concerned one can define “understanding” as a property of computation—if the computation uses the relevant conceptualization in the process of producing the answer, then one can claim that there is understanding in the computation of the subject at hand. Therefore, to me it is a no-brainer to say of a very strong model today that there are things it “understands.”
– I think there may be a confusion between what you call a “mental dimension” and another matter, namely computation that takes place at several levels of abstraction. In a neural network, for example, the lower layers relate to the most basic data of the input, but the higher layers become much more abstract. They may understand something about the fact that there is a roaring tiger in the image, but may already lose sensitivity to which tiger exactly it is and at what angle its mouth is. A certain level of abstraction can also refer to various states of the “self.” This is a kind of consciousness which, as stated, is entirely mechanical, and it may be that the claim in the articles you cite is that there is benefit in computations carried out at such a level of abstraction, which, as stated, are no longer tightly anchored in the input and therefore ostensibly occur on another plane. That is what may generate the phenomenon of Kahneman’s “System 1” and “System 2.” Again, these are entirely mechanical phenomena.
– Also regarding consciousness, I do not understand what the exact definition is. In the simple sense one could claim that it describes a computation part of which also includes the computing entity itself. When my robotic vacuum hurries back to the base because its battery is about to run out, it displays a kind of “consciousness.” This is still entirely mechanics, still all syntax, but of course it helps it function better.
For example, it may be that the information that the battery is about to run out will be translated in the higher, more abstract layer into a kind of panic state—need to go home. That same neuron in the higher layer that signals the robot to return to the home station does not carry information as to why (just as the neuron that shouted tiger no longer contains information about the tiger itself). The computation that, because of this panic state, tells the robot that it now needs to navigate home may be carried out in an entirely different computational module, and therefore there may be evolutionary benefit in this abstraction. Perhaps, if one were to ask the model what happened, it would even say that it “felt” panic. When people say “System 1” and “System 2,” I think this is what they mean, and again there is no discussion here about mentality.
– Regarding learning from many examples—you may be missing the fact that after a language-and-vision model has already learned about the world and come a long way, it can absorb and use new formulas without any examples at all. And it can also learn completely new concepts with only one or two examples, in a very similar way to human beings. That is, you can show the model one or two pictures of a completely new creature that was not seen during training, and the model will know how to identify it later. There is no significant difference here from a human being, who has also been exposed to quite a lot of input from the world before he begins easily to generalize and absorb new examples.
– Incidentally, the topic of learning new things on an already-trained model in such a way that it will not forget what it learned is an active research field called continual learning, and I have no doubt that there too there have been and will be many breakthroughs.
– And as an aside, although I think that understanding, consciousness, and intelligence are things that can exist both in machines and in human beings (who are also a kind of machine), I do not rule out dualism. It may be that this abstraction in computation, which makes it possible to speak about the self in terms “detached” from the atoms on the ground, is what allows that mental dimension of experience to enter. Just as I do not know how this happens in human beings, I do not rule out the possibility that machines too will have access to that mental dimension.
There are a few more issues one must fail to understand in his approach. See “The Bluff of Determinism in a Minute”: https://youtu.be/1cN5kR5rb3k?si=1jtxBkvR7xc907Ew
There are additional issues to distinguish among. See “The Bluff of Determinism in a Minute”: https://youtu.be/1cN5kR5rb3k?si=1jtxBkvR7xc907Ew
With respect, but this video (which I understand you helped create) is a shallow and ridiculous video. It takes one of the most complicated issues in the history of philosophy and tries to decide it by means of various ridiculous anecdotes about the impossibility of predicting human behavior, all while completely ignoring the relevant scientific knowledge in fields such as neuroscience (which today is the main thing driving people toward a deterministic view). Simply a joke.
(Incidentally, this video certainly fits the general level of that ludicrous YouTube channel, in which (almost) all the videos are built on the same structure: examining significant and profound issues through a childish and absolute prism that contains no complexity whatsoever. Truly pathetic.)
I do not understand why you call the materialist of the second type a materialist. After all, even the epiphenomenon is a spiritual entity (fantasies and lies too are spiritual realities). Even liquidity in our mind’s eye is a spiritual entity and has to exist somewhere. It is indeed a property of a collective of particles, but it is an entity in its own right and not just some particle…. So this is really a dualism in which there is no interaction between spirit and matter (I once heard that this is called non-interactionist dualism).
That is a matter of definition. Here I identified epiphenomenalism with emergence. Emergence sees spirit as a function of matter and not as an additional substance. Therefore it is materialistic. The whole idea of emergence is to allow the materialist to acknowledge the existence of spirit and a mental dimension.
True, one can also interpret epiphenomenalism as non-interactionist dualism, but that is an uninteresting dualism. It merely accompanies reality and does not affect it. Therefore I did not speak here about epiphenomenalism, but about spirit as an epiphenomenon (= an accompanying phenomenon).
So you do not define materialists as those who say there is only matter and no spirit. It seems to me, however, that this is the original definition. At least that is what any person on the street would tell you. Just a pedantic remark.
It is advisable to distinguish between two shades of epiphenomenalism. Original epiphenomenalism (that of Shadworth Hodgson, for example) really did advocate a dualism of substances, with body and soul as two real ontological entities, only with a one-way connection between them. That view is indeed dualistic in the Cartesian sense. But what is called epiphenomenalism today (this is the view of thinkers such as David Chalmers, and also of Frank Jackson before he became a physicalist) is a dualism of properties: the view that although there are two different kinds of properties (physical and mental), there is only one physical substance to which those properties belong, so that it is the body that undergoes the mental experiences and not the spirit. This view is closer to an enlightened physicalism (one that does not deny the human mental layer) than to dualism.
I certainly do define materialism as the view that advocates the existence of matter alone, and not spirit.
Hello Rabbi.
You used an image according to which we (consciousness/intellect) operate the brain just as we walk by means of our legs.
But the connection between the brain and the leg is a physical connection of electrical and chemical conduction. If consciousness is a non-material entity (mental/spiritual), how exactly does the interaction occur? How can something lacking mass and energy push the first electron in the brain or open a channel at a synapse, without violating basic physical laws such as the conservation of energy?
When you ask how this happens, are you expecting a scientific answer—that is, one within the laws of physics? It cannot happen through a physical mechanism. I expanded on this in The Science of Freedom.
A. Is this not the “special pleading” fallacy?
B. Seemingly, Libet’s experiments proved that the computer works even without consciousness. And yes, I know that this was in trivial decisions, but it still proves that the brain does indeed work in an ordinary physical way.
Is it not plausible to say that computational abilities are the result of strong emergence?
Strong emergence is not plausible in itself. Not any such or such results of it. Search here on the site for an explanation.
A. Please refrain from using expressions that sound learned but are really empty of content. This is a common technique in our circles: instead of raising counterarguments, one uses labels—whataboutism, chauvinism, and so on. Agreed-upon expressions can be used when it is abundantly clear that there is a fallacy, but not in place of a justification that there is one here.
Indeed, this is a special case that departs from physics. Why is speaking of a special case a fallacy? Are there no special cases in the world?
B. What did Libet’s experiments prove about computers? And who said the brain does not work physically? In short, could I get a translation into Hebrew?
I saw the explanation, but still, the author thinks it is possible, and I am asking according to his view.
Aviv Franco said that even if there is a soul, it may be a kind of energy (let us leave aside the absurdity of that claim).
My question is this:
The author of the article writes that a mental dimension evolved in order to enable a qualitatively stronger computational ability.
The rabbi argued that such a claim presupposes dualism.
I want to ask why the author of the article cannot argue that this mental dimension is the result of strong emergence.
I would appreciate an answer.
Aviv Franco said that even if there is a soul, it may be a kind of energy (let us leave aside the absurdity of that claim).
My question is this:
The author of the article writes that a mental dimension evolved in order to enable a qualitatively stronger computational ability.
The rabbi argued that such a claim presupposes dualism.
I want to ask why the author of the article cannot argue that this mental dimension is the result of strong emergence.
Is there a reason this question is not receiving an answer?
In my humble opinion, this is an excellent question.
There certainly is. Because it was already answered above.
Perhaps a human being also does not “think” in the mental sense? That is, the starting assumption of the discussion is that the Chinese room shows that the concept of “understanding” is not identical with behavioral input and output, and about this I would like to ask—perhaps a human being, too, is only input and output with no mental understanding at all?
The premise of my question follows postmodern claims that there is not really any “understanding” of the other, only behavior. I am not coming now to discuss the validity of this claim (which is similar to the “philosophers’ chestnut,” only here it concerns the layer of language and its relation to the meaning we ascribe to behavior), but only to develop one more working assumption in this question.