According to critics, mankind will be a huge step closer to self-extinction in the not-too-distant future. More specifically, this work could be carried out by lethal autonomous weapon systems (LAWS) equipped with artificial intelligence (AI). But how realistic such apocalyptic future scenarios really are? What is the current status of LAWS and how likely is an international ban on such weapon systems?

More or less unnoticed by the public, systems with weak AI are evolving: they improve the results of search queries on the Internet, are used in speech recognition or machine translation, and such systems will also be used to control drones and vehicles, to increase efficiency in logistics, in medicine and in many other areas in the near future.

Through the use of “deep learningneural networks“, the progress in the field of AI has been remarkable in recent years. Put simply, this approach first extracts abstract solution strategies from an extensive data collection on a specific problem. The solution strategies are then supplemented, expanded and improved using data known and unknown to the system – comparable to training. Finally, the perfected solution strategies are applied to specific problems. [1]

Theoretically, the Phalanx Close-In Weapon System CIWS could be operated autonomous whiteout a man in or on the loop.

Automatic weapons systems have been in use in the armed forces for decades, but with operators always involved in the detection, identification and selection of targets or in the final decision on the use of (lethal) force. In autonomous systems, on the other hand, these processes take place almost without human interference. Such systems do not have “free will”, but they are able to carry out certain tasks independently, without human interaction, under unforeseeable conditions, on the basis of their rules of engagement (Paul Scharre, “Army of None: Autonomous Weapons and the Future of War“, W. W. Norton & Company, 2018, 27ff). Theoretically, this is already the case with some defence systems, such as the Aegis Combat System, the Phalanx Close-In Weapon System CIWS, and modern air defence systems [2]. Currently, more than 30 states have such autonomous defence systems. However, this excludes drones that are still used operationally, because they are remote-controlled and not autonomous, at least in the decisive phase of the use of force (Scharre, “Army of None”, 4).

Loitering Munition also has a high degree of autonomy. It is a type of guided weapon that is initially launched without a specific target, can orbit over a target area for a long time, and then uses its sensors to attack the target. This includes, for example, the IAI Harop, which comes in the form of a stealth drone. It can autonomously eliminate radar sources from its waiting position above the target area, in which it can stay for 6 hours. Israel Aerospace Industries (IAI) had exported the Harop to Azerbaijan, where it was used in the Nagorno-Karabakh region for the first time – albeit not as originally planned. The device hit an Armenian bus with militia soldiers who were being transported to the contested region. Seven soldiers were killed (Thomas Gibbons-Neff, “Israeli-Made Kamikaze Drone Spotted in Nagorno-Karabakh Conflict“, Washington Post, 05.04.2016).

The military use of autonomous systems and the proliferation of the corresponding technologies are associated with some risks. At the strategic level, it should not be underestimated that the autonomous systems lack the ability to understand own actions in the overall context. LAWS will be used with a set of rules, which will define their actions. However, the context of a conflict can change rapidly. In such cases, the stubborn observance of preassigned rules, the lack of anticipation, empathy and gut feeling can lead to unforeseen or unwanted escalations. Every soldier knows that, in the course of a mission, the command received at the beginning may not coincide anymore with the original intention of the superior. A soldier must be able to adapt himself to the situation, so that the intention of the superior can be implemented at all times and all unforeseeable circumstances. The absence of this consideration of the overall context can lead to instability at the international level. If LAWS had already existed during the Cuban Missile Crisis, a U.S. policy might have ensured that Soviet ships (including submarines) would have been forcibly prevented from crossing the blockade line if necessary. A Soviet rule of engagement would probably have meant that the sinking of a strategic submarine would have to be answered with a regionally limited nuclear retaliation. If both systems would have begun to interact with each other uncontrollably with the respective guidelines, for example because the U.S. system detects a violation of the blockade line (whether rightfully or not), this would have lead to a catastrophe at a breakneck speed. [3]

Interestingly, such considerations do not play a decisive role in the discussion on an international ban on LAWS. The reason for this is that the efforts to ban LAWS are being driven almost exclusively by NGOs active in the field of human rights and international humanitarian law. This creates two different camps which demand a ban for different reasons:

Consequentialists fear negative effects on the use of LAWS because these systems will not be able to comply with the principles of international humanitarian law. These include, for example, the distinction between combatants and non-combatants, the principle of proportionality and the prevention of unnecessary suffering. The problem with this argumentation is that if LAWS were to violate the principles of international humanitarian law, their use would already be prohibited today – an additional ban is not necessary. On the contrary, such a ban would benefit those states which do not recognise international law and would therefore not comply with this ban either. Also unanswered is the question of how to proceed from the point of view of the consequentialists, if the progressive technology would enable a much more precise use of force through the use of LAWS, which could incapacitate the opponent in a much more targeted and selective way. [4] Wouldn’t LAWS have to be used as an outright priority in such a case?

Deontologists argue that the use of LAWS is unethical regardless of its effects, as is the case with torture, for example. Decisions on life and death would have to be made exclusively by people, for only they are in a position to morally weigh up such a decision in its full scope. Even if LAWS could reduce the number of deaths in wars, their use would violate human dignity (Scharre, “Army of None”, 285ff). The problem with this argument is that it is completely unrealistic – the days of a fight in which one human soldier faces another are long gone. Where is the dignity in being mowed down by a machine gun, shredded by a bomb, or killed by an infection? In wars, there is no right to a dignified death. In this way, deontologists do not criticize LAWS, but wars per se.

In early April, the Group of Governmental Experts (GGE) formed by the Convention on Certain Conventional Weapons (CCW) addressed the topic for the second time. However, it would be premature to conclude on the success of the campaign – the GGE is nothing more than a discussion forum without a negotiating mandate. A first decisive hurdle will be whether a generally accepted definition of LAWS can be formulated. Although 26 countries currently support a ban, with the exception of China, these countries are not technological pioneers in the field of LAWS. With few exceptions, these states seem to be more interested in restricting the capacity of more powerful states to act than in humanitarian or even ethical factors. This will make it difficult to get the USA, Russia, UK and France, all of which explicitly opposed such a ban, on board.

Even if, in the long term, the international community of states could impose such a ban, the genie can hardly be pushed back into the bottle. The developments that form the basis for LAWS are not military but civilian. The key to this is the software development, which is largely independent of future use and the eventually chosen hardware platform. The ability of an autonomous system to move around in an unknown building, to sketch and identify rooms, equipment and people inside, can be used positively in the hands of rescue forces, but negatively in the hands of terrorists (Scharre, “Army of None”, 121ff).

According to Frank Kendall, former U.S. Under Secretary of Defense for Acquisition, Technology and Logistics, the main threat posed by LAWS is not the application by armed forces, but the uncontrollable proliferation in which virtually anyone can access the corresponding technologies and use them for their ends (Scharre, “Army of None”, 120). [5] What a future with killer drones in the wrong hands could look like was impressively demonstrated last November in the video “Slaughterbots”, which caused quite a stir.

Conclusion
Automatic weapon systems have been a reality in the armed forces for decades, although always reserving the ultimate power of decision to humans. LAWS go one step further in this respect: they have the ability to carry out certain tasks independently and without human interaction under unforeseeable conditions. This is not a very distant vision of the future – simple autonomous systems in clearly defined fields of application, for example in air defence, theoretically already exist today. But progressive development and use of LAWS do not necessarily have to end in an apocalypse, although the challenges of such weapon systems in terms of ethics, international stability and proliferation are also considerable. A ban on LAWS not only tries to push the genie back into the bottle, which will hardly be possible, but also prevents the benefit of potential opportunities. It cannot be ruled out that LAWS could make a much more targeted and precise use of force possible. Taking into account the development efforts for the underlying technology, not driven primarily by the military field, and the interests of the states involved in the discussion on a LAWS ban, a comprehensive international ban on LAWS seems rather unlikely.