Q&A: Robots on the Battlefield
Robots on the Battlefield
Question
Hello,
The religious leadership is the only one capable, it seems, of saving the State of Israel from religionization,
one whose gravest peak (or nadir) so far was expressed in the remarks to which I refer below.
The question is whether the religious leadership will understand, at least at the eighty-eighth minute, that reality demands a clear halakhic response.
In Tuesday’s issue, Brigadier General Ziv Avtalion wrote about the entry of robots into the battlefield, and argued that when the mission is dangerous, the use of robots is essential. Avtalion points, for example, to a scenario of infiltration into the country’s territory, after which “drones and vehicles are sent to encircle the terrorists… the entire decision-making process is carried out by a computer well acquainted with all the cases and responses, and all the forces that were deployed were robotic.” There is no doubt that such technological means are entering and will continue to enter the IDF, and no one disputes that saving soldiers’ lives is an important principle. However, it is not the only principle. If it were, it would be preferable to drop a hydrogen bomb and eliminate all the people found in any enemy territory. Is that the way of an army that sees itself as the most moral in the world?
Another problem raised by Brigadier General Avtalion’s remarks is the issue of computerizing moral judgment: like all descendants of Noah (and human beings in general), we are commanded in the Hebrew Bible (Tanakh) that “Whoever sheds the blood of man, by man shall his blood be shed; for in the image of God He made man.” This normative principle places a heavy responsibility on every person who bears arms, while at the same time expressing the principle of the sanctity of life. Shall we allow software to decide on taking human life? Moreover, do such systems even have the proper tools for that? It seems that the world is currently suffering from datification, a view that says information (data) is sufficient in itself for taking a moral stance. But leading researchers in robotics and morality such as Colin Allen think, for several reasons, that this is mistaken. Therefore, the value-based insistence of the IDF until now—that behind every weapon there should be human decision-makers (which led Air Force commander Amir Eshel to change the Hebrew acronym for an unmanned aircraft from one meaning “unmanned” to one meaning remotely manned)—is a basic human and Jewish principle that should be upheld.
Good news,
Answer
I’ll start from the end: I completely disagree with these claims. They strike me as downright absurd, from beginning to end.
First, similar problems arise with regard to autonomous vehicles, which are supposed to come into broad public use within a few years (they are already operating in various pilot programs around the world), and there the problems are far more severe, because roads are full of innocent people and the situations will occur thousands of times every minute (whereas on the battlefield these are isolated and relatively rare situations).
Second, the excessive caution here regarding the lives of innocent people on the enemy side stands against sacrificing the lives of our soldiers. Why do you prefer the former over the latter, without even giving a reason? This is an absurd thesis, yet you present it as though it were some self-evident principle. Especially since here we have a possible risk in exceptional situations to those innocents, versus a certain risk in every situation for our soldiers.
As an aside, I would note that a robot’s moral judgment may well be much better than a human being’s, since it is faster and less infected by emotions and fears. I do not understand how you trust a human being more than a robot (assuming it has been well programmed and tested in different situations). Its chance of error is far smaller than a human being’s.
Let me just note that this question does not have the slightest connection to the essence of morality or to the fact that the robot is a being lacking moral responsibility. The question here is consequentialist (preventing harm to innocents, not whom to put on trial for a mistake). The responsibility lies with the programmer and his commanders, who are human beings and have moral responsibility.
Therefore, I am entirely in favor of this direction on the level of principle, although of course I am not familiar with all the details.
Discussion on Answer
N., hello.
1. I didn’t deny the existence of military ethics; I argued that my military ethics does not prefer the lives of enemy innocents over the lives of my soldiers, and certainly does not prefer avoiding a possible risk to them at the price of a certain risk to our soldiers. In your view, should we have given up powerful weapons and focused on pistols because with them one can aim more precisely at combatants and avoid killing uninvolved people? What is the difference between this robot and automatic weapons and munitions that already make decisions in the field today? The difference is mainly quantitative. True, there are questions of proportionality, and I completely agree, but I really don’t see where proportionality breaks down here. At least not until you show how badly the robot system fails in dealing with such moral problems (see the next section).
2. As for your claims about the software, it is clear that you lack basic information about artificial intelligence. You think such a robot learns by means of classic software with IF commands, feeding it the correct answer for every possible situation. But that is not how one “teaches” a robot to act morally. The logic is completely different, and there certainly are ways to teach it moral conduct using what is called a “neural network.” The advantage of such learning is that one does not need to know all the situations and relevant answers in order to reach the ability to decide in all situations. The more situations with correct answers you feed it, the more it will know how to behave correctly in new situations—no less well than a person, and probably even better. This logic is not understood by those unfamiliar with it, but forming a position on this topic requires entering into it and understanding it. Otherwise you are simply not using the right tools to discuss the topic. If you assume that you need to feed the software answers for every situation, you are simply not up to date on this technology. Even in handwriting or face recognition, you won’t get very far with classical software, and that is why people use a neural network (a learning network), and the fact is that they achieve quite decent results (and it keeps improving all the time).
Therefore, contrary to your assumption, moral considerations definitely can be programmed. And one can probably do it quite well. In the next section I’ll add that I doubt a human being would act better or more correctly in these situations.
And no, I do not assume Jewish law is a closed body of knowledge, and no sources or thinkers are needed for that. It’s a simple fact. Even so, one can program the robot to act according to such an open body of knowledge, certainly in the specific circumstances of combat situations. It will probably do this no less well than a human being with his flexible and problematic judgment in such situations. Your hanging this topic on the question whether Jewish law is a closed body of knowledge also rests on your assumption that we are talking about classical programming and not a neural network. A neural network is built to deal precisely with such open domains.
You asked for a referral to a programmer. You should turn to computer scientists who are knowledgeable about artificial intelligence (any one of them can explain the basics reasonably well), and I faithfully promise you that you’ll discover wonders there. It’s a completely different logic from the one you assume (implicitly). As I wrote, autonomous cars are already here, and the moment their software demonstrates a good enough “moral” level, nobody will stop them. Believe me, companies do not invest billions in a machine that in the end will not be approved to operate, and without thinking that the “moral” barriers can be overcome. That wheel has already been invented (though I assume it is still being refined these days).
3. Beyond all this, as I wrote, a human being also has no way of knowing the answer in every situation, and I’m not at all sure his decisions will be better. So I do not see why he is preferable to the robot (with all his weaknesses—fears and powerful emotions in such situations). What is the correct answer in the trolley problem? Is it obvious to you that a human being will decide better than a robot? And I’ll ask further: what does Jewish law say about this? Exactly what ethics says about it. I assume most halakhic decisors would rule here according to their ethics (see next section).
4. The question whether its programmers should consult people of Jewish law or other ethicists is a different question, but it is just as true regarding the military ethics of human beings. There too they do not really consult people of Jewish law, so why complain about robots that aren’t halakhic? In general, in my opinion Jewish law does not have much unique to say on this matter. It is not essentially different from accepted ethics, and therefore I do not see the necessity of bringing people of Jewish law into these issues. I do not think they will necessarily do it better than others (I am among those who think morality is by definition universal. There is no such thing as “Jewish morality”).
All the best,
Michi, hello,
1.
Even a bow and arrow is an autonomous weapon, and in fact an electric fence is as well. But there are qualitative differences between them and a drone or humanoid robot or other system that acts completely on its own. The former do not decide for themselves which direction to shoot in, and are not “loitering” weapons, as they are called today in the IDF. That is a qualitative difference, like the difference between a stone and a rabbit. A qualitative difference requires a thorough accounting regarding the moral quality of the alternative now under discussion, not just its efficiency in the very narrow sense of the word.
2. Are you prepared to replace human judges with a camera that identifies
the defendant’s facial features, monitors his emotions during the hearing, and upon hearing various arguments decides on its own, on the basis of artificial intelligence, the legal fate of human beings? Is the judge’s humanity only a source of various errors and biases (as is popular to say today), or is it also a necessary condition for empathy and, in that sense, for moral judgment? Many today take the second point as obvious, and perhaps that is why when one asks about the ability to replace a judge with a machine, it gets ignored.
3. Autonomous cars are not a loitering killing device.
The fact that this is not their purpose is important. And although they are not moral agents, in this case there is a difference between them and devices and agents whose direct purpose is matters of life and death and the killing of human beings. The argument that “nobody will be able to stop them” was raised as an argument for compromising in the face of certain tyrannical regimes in the previous century, but when it comes to fundamental issues (and especially them) I do not see that as a moral argument. Many things “catch on” in the market, and if they are moral and useful—excellent. But many things enter today even though they are entirely harmful, and therefore I am not willing to accept in advance the claim that a new product should enter our lives without any criticism just because it is new or “innovative.” We have already seen the failed experiment of blindly introducing technology into the public sphere without proper regulation in the case of electric bicycles on Israel’s sidewalks; I hope autonomous weapons will arouse more prior discussion. But again, autonomous cars are not the issue here. Personally, I assume they will do a good job.
4. If you mean natural morality, etc.,
then of course I agree, but morality is not universal in at least two senses: a. algorithmic morality (such as Kant’s categorical imperative) is not necessarily valid morality—without the subjective component there is no human being and no morality, hence the critiques of Sartre, Gilligan, Dancy, and others of Kant’s pretension to universalize morality; b. relying on particular religious moral systems is, in my humble opinion, a condition for the ability of social ethics to function. That is why, for example, it took Hanan Porat in order to legislate the law “do not stand idly by your neighbor’s blood.” Apparently thin liberal ethics is not enough.
Also note
that you assume we can program ethics, while we live in a human society that for a hundred and fifty years has itself been at a dead end regarding ethics (see Anscombe, MacIntyre, etc.). So will the autonomous weapon operate according to the morality of Kant, Mill, Darwin, Nietzsche (I’ll refrain from mentioning Hitler by name…), or MacIntyre? And on the halakhic plane (where you are right regarding its integration into the general philosophical discourse), according to Rabbi Akiva or Ben Azzai? The inability to reach a satisfying answer to the first questions should at the very least raise question marks for us about leaping too hastily to the stage of programming them.
Have a good day,
Hello,
1. Not true. A bow and arrow are not an autonomous weapon. That is precisely the halakhic difference between “his fire is like his arrows” and “his fire is like his property.” The arrow carries within it the force of the person and causes damage through that force. Fire spreads with the aid of the wind, and it may indeed be a somewhat autonomous weapon (like damage caused by an animal). But neither an arrow nor an electric fence is such a thing. They do not make independent decisions.
2. If the camera proves reliable, then certainly yes. These software systems are thoroughly tested before use. These fears stem from unfamiliarity with artificial intelligence, as I wrote to you. Even empathy can be programmed (not the feeling of empathy, but its outputs, and that is what matters for judgment). There is no ignoring here (again, a misunderstanding about artificial intelligence). Of course this can be done only when it is clearly demonstrated that the software does in fact succeed at it. Have you ever seen the film HER? You should.
3. They are, yes. Why should I care what the machine’s intention is? Are you talking about the damages of an ox that intends to gore? Why should I care about the machine’s intention? The question is what it does, not what it intends, and not what it was designed for.
By the way, here you are mixing up technical problems of introducing technologies without sufficient testing with principled objections to replacing human beings by technology. I agree with the former and not with the latter. In your opinion electric bicycles were not properly tested (and in my opinion they were, but the law regarding them is not enforced. That is a completely different question), but what about cars? What is the difference between bicycles, which you oppose, and cars, which you don’t? Do bicycles make automatic decisions more than a car? This is simply a matter of insufficiently effective enforcement and nothing more. There is no need to turn this into an ideology. And even if you were right that the bicycles were not tested enough, then let them test them. I’m talking about the principled question, assuming they tested as much as needed.
4. Kant’s categorical imperative is about as far from being algorithmic as heaven is from earth. On the contrary, the standard criticism of it is that it contains no guidance at all about what to do in practice, because everything still depends on what I would want to become a general law. I won’t get into the silly law of “do not stand idly by your neighbor’s blood” here. That is evidence to the contrary. In any case, if you don’t like liberal ethics and want specifically religious ones (to me that is an oxymoron. Ethics by its nature is universal, and this is not the place to elaborate) — then by all means program your robot according to religious principles. There is no principled problem in doing that any more than liberal ethics.
With all due respect, your last argument is really absurd. If among human beings there are different ethical systems (I agree factually, though quite at the margins), then that shows the problem exists even if you leave the decision to human beings. How is that different from what a robot would do? This is an argument to the contrary. You are showing that even if we do not mechanize it, we will remain with different behaviors. So what did we gain by preventing mechanization? Basically you are saying there is no one correct ethics, and together with that warning against using a robot because maybe it will behave incorrectly.
Notice that you are implicitly assuming a very specific approach to moral decisions. In your view they are supposed to be judged not by their outcomes but only by the fact that human beings made them. That is, a decision is moral if a human being made it, regardless of its results. This is an absurd and very far-reaching version of deontological morality. Deontology can be a condition for the moral judgment of an act, but not an exclusive condition. For our purposes, of course, this is not important, because clearly what matters is the result. I am not interested in whether the robot is moral. What matters to me is that it not kill people unjustifiably and that it make correct decisions (in terms of outcomes).
Have an excellent day,
And there are (at least) two more very important considerations before we give up on the human agent in the army:
* The military sphere is the place where human societies experiment with technologies that later migrate into the civilian sphere (and today the separation between military and civilian is smaller than ever). This transition is usually done “with God’s help,” meaning they simply change the name—for example, from an unmanned aerial vehicle to a drone—but it is worth paying attention to this phenomenon of “conversion.” So in agreeing that the army should be non-human, you are hereby signing on to being willing that humanoid robots put you into a police car on the street (accompanied, of course, by an electronic arrest warrant issued by an algorithm, all perfectly legal), evacuate Amona / Umm al-Hiran without any human hand involved, that the bank send robots to evict you from your home if you fall into debt, and so on. Will this reduce bloodshed or increase it, and increase fairness? It certainly will not promote transparency. None of these things are science fiction. Their becoming accepted reality, if it happens, will be the outcome of social conventions, which people like Brigadier General Avtalion are trying to change—though to his credit I’ll say that I believe he is not aware of the gravity of what he is saying and its implications.
* The human agent has disadvantages but also advantages from a moral point of view. But as we know from the Nazis’ move from direct shooting to gas vans, the human difficulty in digesting killing is a disadvantage from the standpoint of military and industrial “efficiency,” but it is also evidence that the natural morality mentioned above causes human beings to be willing to kill only when they are convinced it is justified. Is that why all kinds of dubious technologies are being promoted in the IDF today for engineering soldiers’ consciousness? Giving up on the human agent will open the door to a very problematic reality, which does not seem to me likely to reduce injustices. The same is true on the macro level—today going to war involves convincing the public that the price in human lives justifies it. Remote-controlled war will eliminate that “barrier.”
The previous comment was deleted, and this one will be too, and I am now announcing that any further ones will also be deleted if they are not phrased respectfully and properly.
Our rabbi is humble like Hillel, and therefore answers and answers again with patience and at length (unnecessarily, in my opinion) to idiotic questions, but I was zealous for the honor of a Torah scholar. N.’s questions are fit for a science-fiction forum or Moshe Ratt’s website. The pseudo-intellectual presentation of a problem that does not exist points, in my eyes, to the decadence of the academy in some of the humanities fields from which the questioner comes.
Attached is an email exchange on the subject:
Q: Computer ethics: Who will educate the learning machines?
In recent years, the development of computing power, alongside various learning algorithms (machine learning, deep learning, etc.) and neural-network techniques, has created enormous growth in the field of artificial intelligence (AI). More and more “learning” machines are entering our lives and replacing tasks that until recently we thought only human beings could do.
But there’s a catch: we may lose control over the machines because we have given them the ability to change their own software.
Example: About a year ago Microsoft launched a learning program named Tay, meant to assist in conversation for those encountering problems with Microsoft software. Tay was taken offline after 24 hours, after learning from conversation partners to curse, use foul language, and the like.
We human beings are also a kind of “learning machine,” and we too (especially children) may learn improper forms of behavior. But the education system, parents, teachers, and society teach us not to use improper behaviors. For that we have “values.” But what are the values of computers with learning software? Who guarantees us that one day they will not destroy us, if they learn, for example, that we can disconnect them from the electricity?
Asimov tried to solve the problem by means of the three famous laws of robotics, but these are very difficult (and perhaps even impossible) to implement.
Attached is an article from this morning’s New York Times announcing the scientific effort that AI scientists are currently making to deal with this problem:
https://drive.google.com/open?id=0BwJAdMjYRm7IMkFtUHNWSnVIRUU
Michi: In my opinion these apocalyptic scenarios are exaggerated. How is this different from any technological tool that causes harm on behalf of human beings?
There is a similar exchange on my site about autonomous cars.
Q: The concern is loss of control.
Michi: What is the difference between your machine that you lost control over and an ordinary machine fully controlled by the enemy that is trying to eliminate you? Other than irrational hysteria, I see no difference whatsoever between the situations, except perhaps to the disadvantage of AI (because in the second case there is a deliberate effort to eliminate you and not merely an accidental loss of control).
Q: Against an effort to eliminate me I have control over means of defense.
Michi: And so too against an effort by your machine to eliminate you. Or against neither one nor the other. Exactly the same thing.
Q: The concern is loss of control, so that we won’t know where—
Michi: Exactly as you do not know where your enemy’s means of attack will go.
Q: Against an enemy I have good information about the means of attack (and also decent intelligence about his intentions), as well as control over means of defense (see Iron Dome, for example), whereas regarding AI the concern is loss of control over the information about where it is developing and what it is plotting, and therefore an inability to defend oneself.
Michi: Not at all.
The enemy too can use AI against you without losing control over it. And you also have intelligence about your own AI, since you built it.
Q: That is exactly the concern—that even though I built it, we may lose control of the intelligence regarding where it is developing.
Michi: As stated, I do not see the slightest difference.
Rabbi Michi dear,
I certainly think our soldiers’ lives take precedence, but that isn’t the issue. There is military ethics (its roots are already planted in the book of Deuteronomy), and that is what is supposed to distinguish us from all sorts of wicked and sinful people.
As for autonomous cars, there are enormous ethical difficulties, and you surely know Roly Belfer’s remarks on the subject. It’s very far from simple.
Mobileye, as I understand it, is simply waiting for the courts to tell whoever sues them what the answer is and who is right, but it would have been fitting for the world of Jewish law to state its view on this from the outset, and not by a “we’ll wing it” method.
Even if the answer is simply that on the large scale this will save many lives, and therefore it should be advanced.
Still, it is proper to discuss the programmer’s considerations: in the trolley problem, is it preferable to hit an elderly person? Two children?
If the car hits a cat, should it stop? And if it hits a child? No question is self-evident,
and what is troubling in Brigadier General Avtalion’s remarks is the initial assumption that this is simple—very simple.
Programmers will have to make many decisions about this, and the first people who ought to think about it, and lose sleep over it, are the people of Jewish law.
A robot’s judgment may indeed be better, no question. The question is whether moral considerations can be programmed.
A robot can certainly identify faces better, and from a greater distance. But what gives it the understanding that the person in front of it is someone it should kill, as opposed, say, to a little girl who just happened to pass by?
If you have an answer to that, or can point me to a programmer who knows how to answer it, I’d be glad to hear. The Holy One, blessed be He, is found, in this case too, in the small details.
The question of programming morality into robots is connected to a fundamental halakhic question (with God’s help I’ll get around to writing about this sometime):
Is Jewish law a closed body of knowledge? Can one, for example, derive from Maimonides’ Mishneh Torah in a clear-cut and simplistic way whether and how the State of Israel should be organized?
If you’re skeptical about that, and think—as Rabbi Daniel Sperber, may he live and be well, and Rabbi Eliezer Berkovits of blessed memory did, for example—that judgment and changing reality have an important place in halakhic decisions, then you’re in good company—
that of the amora Shmuel, of Rabbi Joseph Albo, the Maharshal, and others, who thought Torah and values are not an algorithm. The source of the word “religion,” by the way, is actually a Persian word whose meaning is data.
Not by chance do I insist that this idolatrous image of religion, as frozen and dried-out data, does not suit the Jewish religion in general, and Jewish law in particular.
Good evening and good news,