Q&A: Machine Learning
Machine Learning
Question
With God’s help,
Rabbi, is machine learning—like neural-network unsupervised learning—not a difficulty for the anti-Platonist claim that concepts cannot be understood without reduction to prior analytic concepts or without noticing patterns in a purely empirical way?
For example, it can distinguish from images of cats and dogs what is a cat and what is a dog, without a prior definition.
Answer
That is not learning but training. The machine does not “understand”; it behaves as you want it to. It does not “identify” anything; it merely acts in accordance with the correct identification. So it has no connection to Plato’s question.
Beyond that, the machine receives inputs of relevant images, so it is in fact fed by prior information.
Discussion on Answer
It still uses inputs, even through its programming. This is not creation ex nihilo.
As I understand it, your claim that this is a defined calculation and therefore unrelated to philosophical discussions is identical to my claim that there is no understanding or identification here.
Understood
I didn’t completely understand the commenter’s point, so I’ll address the Rabbi directly.
First of all, the Rabbi argued that the machine should not be called something that learns but something that is trained; as I understand it, he said this only because it does not understand.
But it sounds very strange that this fact is relevant to the issue here.
After all, the whole point of my argument is only to undermine the famous argument for the anti-Platonist approach, which says that one cannot understand the world without prior ideas—for example, because “things can be classified in infinitely many different ways,” in the style of arguments about essentialism in biology and the like. (Which the Rabbi also uses.)
So even though I agree that the machine does not understand, as in the analogy of Mary’s room, there is still no identity between “understanding” and “division” into different categories.
Factually, as far as I know, in unsupervised learning you need to provide the system with a large number of images (inputs), for example of dogs and cats, *without saying* which image is which. That is, without a dog/cat classification label—unlabeled. You tell it to classify the images into two groups, and it finds the algorithm for the most suitable classification. (For example, identifying patterns of shape, color, or some criterion we don’t even think of.) And factually, good systems succeed in classifying just like a human being—into dog and cat.
Another example is giving it a picture that contains lots of red and green apples; it will manage to “divide” between them and classify them into red apple and green apple. And then when it is given an apple next time, it will be able to predict whether it is green or red. (And again, here too the inputs do not come with any prior statement saying which apple is green and which is red.)
Also, the claim about input is even less clear to me, because how is that different from eyes? We see many items in the world and divide them into patterns. Does the fact that we have eyes refute Aristotle and strengthen Plato????
By the way, even if this algorithm operates under defined computation, why can’t one say that this is how our brain is programmed to operate? And in any case that would be without ideas.
I don’t understand what is unclear. One can cause a machine to behave in some way by some method. What does that have to do with Plato? He was speaking about the way we understand things. I can cause a soccer ball to fly northward—does that refute (the fictional) Plato, who said that one cannot walk northward without deciding to do so? Or did the soccer ball decide?
It’s not training; it’s learning. The machine learns.
Anything you say about the machine you can say about the human being.
But here it’s not exactly that we cause the machine to behave in some particular way; rather, it succeeds in carrying out classification “on its own” in a way that matches human classification.
If so, then it really is connected to anti-Platonism, which claimed that in order to perform classification one *must* possess prior ideas for that. Otherwise, things could be classified in any other arbitrary way. This is not a disproof, but it does cancel out the support for it from that proof. (Which, if I remember correctly, the Rabbi also uses.)
Maybe I’ll ask the other way around:
If we deny the ideal sense, so that in your view we enter a room that has lots of dogs and cats of many kinds and colors, with no prior acquaintance and no prior knowledge of animals in general and of dogs and cats in particular—
Would we be able to classify them into category “X” and category “Y,” or not?
If you think that this cannot be done, then the fact that the computer can do it this way shows that apparently no ideas are needed for this; rather it is connected to some kind of brain computation or something like that. (Aristotle, in your terms.)
As I said, the computer does not understand the difference; it only acts as if it understands. Plato, as I understand him, was arguing something about understanding. True, if one sees understanding as an epiphenomenon—that is, as a product that accompanies the brain’s computation rather than what generates it—then perhaps your argument does have something to it. One could say that the human being computes in his brain the way the software you described does, except that in the human being the result of the computation that occurs in the brain is accompanied by understanding in the intellect. Here, understanding is a byproduct of the computation done automatically.
But beyond that, one has to remember that a human being also programmed this computer, and the human being does have experience with such distinctions.
In any case, I have written several times that in my opinion such an argument is rather weak.
Thank you very much.
Regarding the first paragraph:
That is exactly the point I really meant; I just didn’t know how to phrase it in those words (understanding as an epiphenomenon). That is also why I didn’t understand the distinction in the first two lines and in the earlier replies, regarding the difference between understanding and a machine.
So, in short, I still don’t really understand what is meant by that view that distinguishes between understanding and mere classification (or “training”)? Also, what reason do we have to accept another approach rather than seeing understanding as a certain kind of “epiphenomenon” (on a dualistic medium, of course)? Isn’t that the reasonable and simple view? Even if we say that there is interaction in certain cases like deliberation, still with respect to the ability to classify at least, one can reject the ideal-based approach, and that significantly reduces the number of ideas in the heavens… Has the Rabbi written about this elsewhere? Because it really isn’t clear.
An analogy can be given for this: after all, the Rabbi would also agree that we have free choice (dualistic interaction), and still vision is an epiphenomenon of physical processes in the brain, which are only reflected in consciousness as colors.
So if that is true, there is no obstacle to assuming that classification can be carried out as a byproduct of computation done automatically. (And in any case, to give up the world of ideas.)
As for the claim that a human programmed the computer, I don’t know these algorithms well enough, but as far as I know the programmer specifically does not try to cause classification according to one criterion or another. Just as there is no label for those inputs.
And regarding the claim that this argument is weak—our Rabbi taught us that when there is no proof, people use slogans and ad hominem, at a time when it seems you understood it very well. “Then perhaps your argument does have something to it” 🙂
Vision is an epiphenomenon of physiological processes. But understanding is not, because otherwise our deliberation would have no meaning. The same is true of will. If it were an epiphenomenon of the brain, then we would have no free choice.
I wrote that this argument is weak because we have the ability to synthesize concepts. The line that runs between something from something and something from nothing is not sharp. This parallels the weakness of the anthropological proof (the proof for the existence of something from the very fact that we have its concept. Here too, a person can synthesize concepts).
I’m not sure that’s completely precise. Even if vision is an epiphenomenon, you still have free choice about where to go.
And likewise, even if classification between objects is an epiphenomenon of the brain, we could still have the ability to decide at a higher level which argument is correct, while using the current definitions or by creating a new definition.
B. Is understanding a concept essentially different from the kind of classification a computer does (assuming it becomes advanced enough)? After all, understanding too does not grasp the thing in itself but only its appearances. But if so, that is not very different from the computer succeeding in classifying things מתוך many inputs. And for us too, concepts are usually understood through repeated observation of the physical world, not through inward contemplation (see Aristotle’s mistakes).
As for the end, I understand that you think the argument supporting the Platonic approach is weak, but the fact that concepts can be synthesized still means, on your view, that there is a more basic concept.
Q
Everything there is mathematically well-defined, so I don’t see any contribution there to the philosophical discussion.
An unsupervised algorithm is not exactly training. You give the system all the inputs but without telling it the correct classification of each input. So what does the system do? There is, for example, an algorithm that does clustering, meaning division into groups. For instance, you feed in a thousand points on a plane and ask the system to find an “optimal” division into N groups of points such that the distances within each group are as small as possible and the distances between the groups are as large as possible. So all you have here are definitions of distance within a group and distances between groups, plus an iterative (deterministic) process that improves and converges.
If there are a thousand 10-by-10 images (that is, a thousand points in a 100-dimensional space) and you ask the system to find an optimal division into two groups, it is entirely possible that this division will overlap with the cat/dog division. Because the “distance” between two images of cats will be smaller than the “distance” between two groups of dogs. The distance function can be defined and adjusted in various ways. Even better results are to be expected if, instead of feeding in the images as they are, you input them after an encoding process (for example, the output of internal layers from another network that does other, broader things and was in fact trained in a supervised way).
By the way, even in supervised learning, since in the end this is just a strict mathematical process (even if calculated only numerically) of finding a minimum for a function with n variables, I find it hard to understand why people connect this to philosophical discussions.