'Former US Secretary of State Henry Kissinger has issued a stark warning to humanity: advances in artificial intelligence could lead to a world which humans will no longer be able to understand — and we should start preparing now.

What if machines learn to communicate with each other? What if they begin to establish their own objectives? What if they become so intelligent that they are making decisions beyond the capacity of the human mind?

Those are some of the questions the 95-year-old Kissinger poses in a piece published by the Atlantic under the apocalyptic headline: ‘How The Enlightenment Ends.’

Kissinger’s interest in artificial intelligence began when he learned about a computer program that had become an expert at Go — a game more complicated than chess. The machine learned to master the game by training itself through practice; it learned from its mistakes, redefined its algorithms as it went along — and became the literal definition of ‘practice makes perfect.’

Into the unknown

We are, Kissinger warns, in the midst of a “sweeping technical revolution whose consequences we have failed to fully reckon with and whose culmination may be a world relying on machines powered by data and algorithms and ungoverned by ethical or philosophical norms.”

Kissinger uses the example of a self-driving car. Driving a car requires judgements in impossible-to-predict circumstances. What would happen, he asks, if the car found itself having to decide between killing a grandparent or killing a child? Who would it choose, and why?

Artificial intelligence goes “far beyond” the kind of automation we are used to, he says, because AI has the ability to“establish its own objectives,” which makes it “inherently unstable.” In other words, through its processes, AI “develops an ability previously thought to be reserved for human beings.”'