Thank you for being a Gates Notes Insider. I feel lucky that I get to connect with so many people like you. – Bill Gates

Not a Gates Notes Insider yet? Sign up

spacer

LOG IN

SIGN UP

EMAILPASSWORD

Forgot?

Log in

Or sign up with your social account:

Log in

Log in

Logout:

Become a Gates Notes Insider

Become a Gates Notes Insider

Join the Gates Notes community to access exclusive content, comment on stories, participate in giveaways, and more.

Already joined? Log in

spacer

LOG IN

SIGN UP

Sign up with your social account:

Sign up

Sign up

Or sign up with email:

TITLE

FIRST NAME

LAST NAME

EMAIL

This email is already registered. Enter a new email, try signing in or retrieve your password

PASSWORD

ADDRESS

Why are we collecting this information? Gates Notes may send a welcome note or other exclusive Insider mail from time to time. Additionally, some campaigns and content may only be available to users in certain areas. Gates Notes will never share and distribute your information with external parties.

ADDRESS LINE 1

Bill may send you a welcome note or other exclusive Insider mail from time to time. We will never share your information.

ADDRESS LINE 2

CITY

STATE / PROVINCE / REGION

ZIP / POSTAL CODE

COUNTRY

Sign up

Join the Gates Notes community to access exclusive content, comment on stories, subscribe to your favorite topics and more.We will never share or spam your email address. For more information see our Sign Up FAQ.By clicking "Sign Up" you agree to the Gates Notes Terms of Use / Privacy Policy.

Deactivating your account will unsubscribe you from Gates Notes emails, and will remove your profile and account information from public view on the Gates Notes. Please allow for 24 hours for the deactivation to fully process. You can sign back in at any time to reactivate your account and restore its content.

Deactivate My Acccount

Go Back

Your Gates Notes account has been deactivated.

Come back anytime.

Welcome back

In order to unsubscribe you will need to sign-in to your Gates Notes Insider account

Once signed in just go to your Account Settings page and set your subscription options as desired.

Sign In

Request account deletion

We’re sorry to see you go. Your request may take a few days to process; we want to double check things before hitting the big red button. Requesting an account deletion will permanently remove all of your profile content. If you’ve changed your mind about deleting your account, you can always hit cancel and deactivate instead.

Submit

Cancel

Thank You! Your request has been sent

Please complete your account verification. Resend verification email.

This verification token has expired.

Your email address has been verified. Update my profile.

Your account has been deactivated. Sign up to re-activate your account.

When ballistic missiles can see

When I was a kid, I read a lot of sci-fi books. One of the most common themes was “man vs. machine,” which often took the form of robots becoming self-aware and threatening humanity. This theme has also become a staple of Hollywood movies like The Terminator and The Matrix.

Despite the prevalence of this theme, I don’t lose any sleep worrying about this scenario. But I do think we should spend more time thinking about the implications—positive and negative—of recent progress in artificial intelligence, machine learning, and machine vision. For example, militaries have begun to develop drones, ships, subs, tanks, munitions, and robotic troops with increasing levels of intelligence and autonomy.

While this use of A.I. holds great promise for reducing civilian casualties and keeping more troops out of harm’s way, it also presents the possibility of unintended consequences if we’re not careful. Earlier this year, U.N. Secretary General António Guterres called global attention to these threats: “The weaponization of artificial intelligence is a growing concern. The prospect of weapons that can select and attack a target on their own raises multiple alarms…. The prospect of machines with the discretion and power to take human life is morally repugnant.”

Unfortunately, my first attempt to educate myself on autonomous weapons was a bust. I read a book that was dry and felt really outdated. Then a few months ago I picked up Army of None: Autonomous Weapons and the Future of War, by Paul Scharre. It’s the book I had been waiting for. I can’t recommend it highly enough.

Scharre is a great thinker who has both on-the-ground experience and a high-level view. He’s a former Army Ranger who served four tours of combat duty in Iraq and Afghanistan. He then went onto a policy role at the U.S. Department of Defense and led the working group that drafted the government’s policy on autonomous weapons. He’s currently a policy expert at the Center for a New American Security, a center-left think tank in DC.

He is also a good writer. Scharre writes clearly about a huge range of topics: computer science, military strategy, history, philosophy, psychology, and ethics. He gives you the right grounding to start participating in the debate over where our country should draw the line on these powerful technologies.

Scharre makes clear from the beginning that he has no problem with some well-bounded military uses of autonomy. For example, he brings you along for a tour of the U.S. Navy’s Aegis Combat System, an advanced system for tracking and guiding missiles at sea. Aegis has a mode of operation in which human operators delegate all firing decisions to an advanced computer (but can override them if necessary). Why would you want to put a computer in charge? If you’re out at sea and an enemy fires 50 missiles at you all at once, you’d be very happy to have a system that can react much faster than a human could.

“Autonomy has great benefits in environments where humans can’t survive.”

Army of None also shows that autonomy has great benefits in environments where humans can’t survive (such as flight situations with high G forces) or in which communications have broken down. It can be enormously helpful to have an unmanned drone, tank, or sub that carries out a clear, limited mission with little communication back and forth with human controllers.

In addition, autonomous weapons could potentially help save civilian lives. Scharre cites robotics experts who argue that “autonomous weapons … could be programmed to never break the laws of war…. They wouldn’t seek revenge. They wouldn’t get angry or scared. They would take emotion out of the equation. They could kill when necessary and then turn killing off in an instant.”

Despite these and other advantages, Scharre does not want the military ever to turn over judgment to computers. To make his case, he offers compelling real-life cases in which human judgment was essential for preventing needless killing, such as his own experiences in Afghanistan. “A young girl of maybe five or six headed out of the village and up our way, two goats in trail. Ostensibly she was just herding goats, but she [was actually] spotting for Taliban fighters.” Scharre’s unit did not shoot. Yes, it would have been legal, but he argues that it would not have been morally right. A robotic sniper following strict algorithms might well have opened fire the second it detected a radio in her hand.

Scharre ends the book by exploring the possibility of an international ban on fully autonomous weapons. He concludes that this kind of absolute ban is not likely to succeed. However, he holds out hope that enlightened self-interest could bring countries together to ban specific uses of autonomous weapons, such as those that target individual people. He also believes it’s feasible to establish non-binding rules of the road that could reduce the potential for autonomous systems to set each other off accidentally. He also believes we could update the international laws of war to embed a common principle for human involvement in lethal force.

There are no easy answers here. But I agree with Scharre that we have to guard against becoming “seduced by the allure of machines—their speed, their seeming perfection, their cold precision.” And we should not leave it up to military planners or the people writing software to determine where to draw the proper lines. We need many experts and citizens across the globe to get involved in this important debate.