Humans can generally trust what they see and hear — but
that won't be the case for long.
Advances in AI and CGI will soon make it possible for
anyone to create photorealistic video and audio.
Experts say it will transform information warfare,
allowing the creation of sophisticated propaganda and
misinformation.
The tech's impact will be profound, turbocharging
everything from fake news and hoaxes to revenge porn and DIY
entertainment.

But humanity is now rapidly approaching the holy grail of hoaxes:
Tools that will allow anyone to easily create fraudulent,
photo-realistic video and audio.

Thanks to advances in artificial intelligence (AI) and
computer-generated imagery (CGI) technology, over the coming
decade it will become trivial to produce fake media of public
figures and ordinary people saying and doing whatever hoaxers can
dream of — something that will have immense and worrying
implications for society.

It will open up worrying new fronts in information warfare, as
hostile governments weaponise the technology to sow falsehoods,
propaganda, and mistrust in target populations. The tools
will be a boon to malicious pranksters, giving them powerful new
tools to bully and blackmail, and even produce synthetic "revenge
porn" featuring their unwilling targets. And fraud schemes will
become ever-more sophisticated and difficult to detect, creating
uncertainty as to who is on the other end of any phone call or
video-conference.

This may sound sensational, but it's not science fiction. This
world is right around the corner — and humanity desperately needs
to prepare itself.

The technology is basic — but not for long

Right now, the technology required to easily produce fake audio
and video is in its infancy. It exists mainly in the form of tech
demos, research projects, and apps that have yet to see a
commercial release — but it hints at the world to come.

A few examples: In July, researchers at the University of
Washington used AI to produce a fake video of President Barack
Obama speaking, built by analysing tens of hours of footage of
his past speeches. (The audio used also came from an old speech.)

The tech to do this live already exists. In 2016, "Face2face"
researchers were able to take existing video footage of
high-profile political figures including George W. Bush, Vladimir
Putin, and Donald Trump, and make their facial expressions mimic
those of a human actor, all in real time.

People are also working to spoof human speech. Voice-mimicking
software called Lyrebird can take audio of someone speaking and
use it to synthesise a digital version of that person's voice —
something it showed off to disconcerting effect with demos of
Hillary Clinton, Obama, and Trump promoting it. It's in
development, and Adobe, the company behind Photoshop, is also
developing similar tools under the name Project Voco.

The next generation of information warfare

In early August 2016, the US had an international crisis on its
hands, and Americans were beginning to panic. As many as 10,000
armed police had surrounded the US Incirlik airbase in Turkey,
and Twitter users were worrying that the situation could rapidly
escalate — perhaps even with the nuclear weapons on the base
falling into the hands of the demonstrators.

This is an example of Russia's longstanding policy of "active
measures" — spreading misinformation for propaganda purposes or
to help it achieve its strategic objectives. Gregory C. Allen, an
adjunct fellow at the Center for a New American Security, argues
that these efforts from Russia — and others like them — will
receive a powerful shot in the arm from developments in CGI and
AI.

"We have seen foreign governments be more than willing to rely on
... propaganda in the text real and in the fabricated imagery
realm," he told Business Insider. "They have demonstrated their
willingness to sprint as fast as they can in this exact
direction, and making use of every tool that is available to
them."

The future could see authoritarian states using forged media to
help generate dissent in the populations of rival countries, much
like what happened at Incirlik — and to discredit and damage
political opposition at home.

Allen also
discussed the national security implications of artificial
intelligence in a recent paper, warning: "We will struggle to
know what to trust. Using cryptography and secure communication
channels, it may still be possible to, in some circumstances,
prove the authenticity of evidence. But, the 'seeing is
believing' aspect of evidence that dominates today — one where
the human eye or ear is almost always good enough — will be
compromised."

The tech is a bonanza for fraudsters

A popular technique employed by modern scammers is "CEO fraud" —
an email sent to a company employee, masquerading as from the CEO
or another executive, asking them to make a payment to an account
or take another action.

These kinds of attacks will soon have a whole new line of attack:
Voice.

Imagine your boss calls you up, and asks you to make a
transaction, or send over a password or a confidential document.
It's clearly her voice, she knows who you are, and you might even
make some small talk. Today, no-one would think anything was
amiss.

This is because, Allen says, you "are currently using voice
as an authentication technology, but [you] don't think of it as
an authentication technology because it's just a background of
human life that you can trust."

But in five or so years, that trust may have evaporated —
replaced by a mistrust of what you hear on the phone and even see
with your own eyes in a video-conference: "In the very near
future it's not going to be something that you can rely on.
Likewise, a video forgery techniques get further along, the same
will be true if you were to have a video chat with someone."

Old people and those are less tech-literate will be particularly
vulnerable, Francis Tseng, a copublisher of The New Inquiry (who
curates a project
tracking how technology can distort reality) suggested:
"Many people deal with their parents or grandparents falling prey
to phone scams ... And an easy rule of thumb to tell them is
'don't give out private information to anyone you don't know!'.
With these voice synthesis technologies, someone could easily
forge a phone call from you or another relative."

And there's a flipside to this: It will also cast some doubts on
even legitimate footage. If a politician or celebrity is caught
saying or doing something untoward, there will be an increasing
chance that the person could dismiss the video as being
fabricated.

It will transform cyberbullying

This technology won't just be misused to pursue political and
strategic objectives, or to defraud businesses: It will be a
weapon for bullies, capable of inflicting arbitrary cruelty.

In the hands of children, it seems likely to be misused to hijack
the image of victims', and to animate it for malicious purposes.
A child's digital avatar might be made to confess their love for
another, embarrassing them — or their voice could confess to a
misdemeanour, landing them in trouble with school authorities.

Justin Thies, who helped develop Face2face, predicted it would
"lift cyberbullying to a whole new level."

A spokesperson for child protection charity NSPCC acknowledged
the danger: "Emerging technologies, such as AI and CGI, pose both
potential risks and opportunities to young people and we must
make sure they do not leave children and young people exposed to
danger and harassment online.

"We know that cyber-bullying can be particularly devastating to
young people as it doesn't stop in the playground and follows
them home so they feel they cannot escape."

The episode indicates there is likely to be significant
interest in on-demand pornography produced using these
technologies in the years ahead, regardless of whether the
subjects of these CGI films give permission.

"Revenge porn" websites already exist dedicated to cataloguing
and sharing the intimate photos and videos of non-celebrities,
and it seems likely that media-editing technology will be used to
produce material featuring "ordinary" people, as well as the rich
and famous — bringing with it the widespread risk of shame and
blackmail.

A whole new world of entertainment awaits

The next "Jurassic Park" might be synthesised in a
teenager's bedroom.Universal
Pictures

Not every use case of this tech will be negative, however. The
internet is already home to a vibrant remix culture — just look
at the
Reddit community "Photoshop Battles" — and
photorealistic video-editing tools may well spark a huge wave of
DIY creativity.

"There could be a lot of interesting IP cases if amateur
filmmakers start synthesizing films using the likenesses of
celebrities and start profiting off that. I can imagine a whole
culture of bootleg films produced in this way," Tseng said.

The tech that powers face-modifying filters in apps like Snapchat
is "primitive compared to the Hollywood CGI or today, but it's
actually significantly more advanced than the Hollywood CGI of
the Eighties," Allen said. "So what we're seeing is the
state-of-the-art capabilities slowly come down in price and
availability such that amateurs have access to ultimately what
are rather impressive capabilities."

The tech likely to be used by the established entertainment
industry as well as amateurs, Tseng suggested: "We've also seen
movies adapt their scripts for certain markets (e.g. the 'Red
Dawn' remake changing the villains from China to North Korea).
There is already a practice of filming scenes to be slightly
different for different markets but this technology could lead to
it on a much larger scale, where even individuals experience a
version of a film totally personalized for them."

Just look at "Star Wars: Rogue One" for an example of how this
tech will be employed by Hollywood studios in years to come.
Peter Cushing reprised his role as Grand Moff Tarkin — even
though he had been dead for 22 years. His image was reconstructed
using CGI overlaid on a real actor.

Left:
The real Peter Cushing. Right: A digital reconstruction, two
decades after his death.Lucasfilm/Disney

This is all right around the corner

This is all currently theoretical. But it won't be long until it
becomes a reality.

"I think we are one to two years away from these sorts of
forgeries, especially in audio where progress is a
little bit easier," Allen said. "One to two years away from
forgeries being able to fool the untrained ear and somewhere
between five to 10 years away from them being able to evade
certain types of forensic analysis."

So how do we prepare? Journalists and organisations will have to
rely increasingly on cryptography to "sign" media, so it can be
verified when required. Big platforms like Facebook will have a
roll to play in policing for fraudulent material, Face2face's
Justus Thies argues: "Social-media companies as well as the
classical media companies have the responsibility to develop and
setup fraud detection systems to prevent spreading / shearing of
misinformation." And it will force ordinary people to be far more
skeptical about the media they consume.

In some cases, "it may be possible to come up with a video format
that simply rejects editing," Allen suggested. "But this will
still be a suboptimal solution compared to what we have now ...
in the best case scenario, this results in there [being] trained
experts who can discern the most likely version of the truth, and
that is just so far away from where we are today which is
amateurs can rely upon their own eyes to discern the truth."

We don't realise just how lucky we've been

These advances mean that humanity is rapidly approaching the end
of a unique period in human history. We "live in an amazing time
where the tech for documenting the truth is significantly more
advanced than the tech for fabricating the truth. This was
not always the case. If you think back to the invention of the
printing press, and early newspapers, it was just as easy to lie
in a newspaper as it was to tell the truth," Allen said.

"And with the invention of the photograph and the phonograph, or
recorded audio, we now live in a new technological equilibrium
where — provided you have the right instruments there — you can
prove something occurred ... we thought that was a permanent
technological outcome, and it is now clear that is a temporary
technological outcome. And that we cannot rely on this
technological balance of truth favouring truth forever."