Book Review

Follow me on Twitter

Superintelligence by Nick Bostrom

May 13, 2019

One day, machines will surpass humans in intelligence. The
question is not if but when. Already we have machines which are more
intelligent than humans on narrowly defined tasks such as chess, scrabble, checkers,
etc. This is known as Artificial Narrow Intelligence (ANI) which has already
been achieved. There will come a time when machines surpass humans on a broad variety
of general intelligence tasks. This is also known as Artificial General
Intelligence (AGI). What will happen after that? Post that the machines will
keep on getting better until they become superintelligent, known at Artificial
Superintelligence (ASI). At this point they will be way way more intelligent
than the most intelligent human being. Imagine the intelligence of human beings
when compared to the intelligence of an ant and you may begin to grasp the
gravity of the situation. This period of getting from AGI to ASI is known as
takeoff period.

Superintelligence is a book about the journey from ANI to
AGI to ASI. We will learn how can we get from ANI to AGI to ASI. We can reach
there via a variety of routes – it can be through Artificial Intelligence /
Machine Learning, or whole brain emulation, or biological cognitive
enhancements, or brain computer interface, etc. We will learn how long will it
take for us to get to ASI. The really interesting part is how long will it take
for us to get to ASI once we have reached AGI. When the takeoff happens, it can
be fast (hours or days), moderate (months or years) or slow (decades). And we
will learn what are the implications for human beings in each of these
scenarios.

From the perspective of human beings will this be a good
thing? When we are confronted with superintelligent machines, what will be our
relationship with them. What will be the values and motivations of superintelligence
machines and will they be on the same side as the humans. Will there be one
superintelligence or will there be multiple. Who will control them? Will it be
possible for humans to control them at all?

A superintelligent AI will have it within its power to takeover
humanity and subjugate or destroy us. So how can we prevent this from happening.
We can do that by being prepared and taking steps now to avoid such a
situation. This is known as the control problem. There are two broad ways in
which this can be accomplished – capability control methods or motivation
selection methods. As you may have guessed, capability control methods try to control
the capability of the AI through various methods. And motivation selection
methods aim to incorporate motivation or goals for the AI which are compatible
with what we want it to do.

After centuries of anaemic growth rates, the global GDP has
been on a steady upward march after the industrial revolution. We have
experienced unprecedented growth and prosperity during this period. However,
this will pale in comparison to what is coming once we achieve AGI. There will
be a huge increase in wealth post the transition but the wealth will not be
distributed evenly. The unemployment will obviously go up with more automation
as we approach AGI. And this will lead to an increase in inequality with highly
uneven distribution of wealth.

Superintelligence covers all of the above and more. The book
deals with a lot of esoteric and complicated concepts and is not easy to read
by any means. Especially the last part of the book which deals with the values
of the AI. How will an AI acquire values and how can we incorporate values in
the AI which are consistent with our own values. The good thing is that it is a
thought provoking book and will make you stop and think as you read it. The bad
thing is that I found it difficult to understand many concepts. At several
places, you have to take a leap of faith and take the authors word for it. But
I guess that is to be expected from any book on AI, as it is a complicated and
evolving subject. Some concepts may be difficult to understand for a layman
with little prior understanding. The author has simplified it as much as
possible but not any more.