For a period of time in 2013, the Timesreported this year, a full half of YouTube traffic was “bots masquerading as people,” a portion so high that employees feared an inflection point after which YouTube’s systems for detecting fraudulent traffic would begin to regard bot traffic as real and human traffic as fake. They called this hypothetical event “the Inversion.”

In late November, the Justice Department unsealed indictments against
eight people accused of fleecing advertisers of $36 million in two of
the largest digital ad-fraud operations ever uncovered. Digital
advertisers tend to want two things: people to look at their ads and
“premium” websites — i.e., established and legitimate publications — on
which to host them.
The two schemes at issue in the case, dubbed Methbot and 3ve by the
security researchers who found them, faked both. Hucksters infected 1.7
million computers with malware that remotely directed traffic to
“spoofed” websites — “empty websites designed for bot traffic”
that served up a video ad purchased from one of the internet’s vast
programmatic ad-exchanges, but that were designed, according to the
indictments, “to fool advertisers into thinking that an impression of
their ad was served on a premium publisher site,” like that of Vogue or
The Economist. Views, meanwhile, were faked by malware-infected
computers with marvelously sophisticated techniques to imitate humans:
bots “faked clicks, mouse movements, and social network login information to masquerade as engaged human consumers.”
Some were sent to browse the internet to gather tracking cookies from
other websites, just as a human visitor would have done through regular
behavior. Fake people with fake cookies and fake social-media accounts,
fake-moving their fake cursors, fake-clicking on fake websites — the
fraudsters had essentially created a simulacrum of the internet, where
the only real things were the ads.

The first development is the ability to fabricate audio and video evidence. Software that can do this is becoming readily available and doesn't require extraordinary computational resources. If you want to produce a persuasive video of someone speaking any script you'd like and if that person has a reasonable amount of available recorded video, you can synthesize that video into the fabrication software. The obvious place for this is politics: pick your target politician, put words in his or her mouth, then package this into propaganda or attack ads as desired.

Fabrication is much more than talking heads, of course. In keeping with the long tradition of early technology exploitation in pornography markets, another popular application is "deepfakes," where someone (a public figure or otherwise) is substituted into a starring role in a porn video (the term "deepfakes" is used both for the overall substitution technology and for the specific porn application). This is already happening, though the technology is as yet far from perfect. Beyond the obvious uses (e.g., advertising and propaganda), there are plentiful disturbing applications that remain unexplored, particularly when these can be introduced into authoritative contexts. Imagine, for example, being able to source fabrications such as police body-camera footage, CATV surveillance, or drone/satellite reconnaissance feeds. The nature of evidence is changing quickly.
...
While there's a great deal to be learned from our experiences over the past century, what's different today is the scale, the ready availability of these tools to interested individuals (rather than nation-states), and the move into audio/video contexts.

Researchers have discovered a vulnerability in Nuuo surveillance cameras which can be exploited to hijack these devices and tamper with footage and live feeds.

On Thursday, cybersecurity firm Digital Defense said that its Vulnerability Research Team (VRT) had uncovered a zero-day vulnerability in Nuuo NVRmini 2 Network Video Recorder firmware, software used by hundreds of thousands of surveillance cameras worldwide.
...
Back in September, researchers from Tenable revealed a remote code execution flaw in Nuuo cameras. This vulnerability, nicknamed Peekaboo, also permitted attackers to tamper with camera footage.

The only site that gives me that dizzying sensation of unreality as
often as Amazon does is YouTube, which plays host to weeks’ worth of
inverted, inhuman content. TV episodes that have been mirror-flipped to
avoid copyright takedowns air next to huckster vloggers flogging merch
who air next to anonymously produced videos that are ostensibly for children.
An animated video of Spider-Man and Elsa from Frozen riding tractors is
not, you know, not real: Some poor soul animated it and gave voice to
its actors, and I have no doubt that some number (dozens? Hundreds?
Millions? Sure, why not?) of kids have sat and watched it and found some
mystifying, occult enjoyment in it. But it’s certainly not “official,”
and it’s hard, watching it onscreen as an adult, to understand where it
came from and what it means that the view count beneath it is
continually ticking up.

These, at least, are mostly
bootleg videos of popular fictional characters, i.e., counterfeit
unreality. Counterfeit reality is still more difficult to find—for now.
In January 2018, an anonymous Redditor created a relatively easy-to-use
desktop-app implementation of “deepfakes,” the now-infamous technology
that uses artificial-intelligence image processing to replace one face
in a video with another — putting, say, a politician’s over a porn
star’s. A recent academic paper from researchers at the graphics-card company Nvidia
demonstrates a similar technique used to create images of
computer-generated “human” faces that look shockingly like photographs
of real people. (Next time Russians want to puppeteer a group of
invented Americans on Facebook, they won’t even need to steal photos of
real people.) Contrary to what you might expect, a world suffused with
deepfakes and other artificially generated photographic images won’t be
one in which “fake” images are routinely believed to be real, but one in
which “real” images are routinely believed to be fake — simply because,
in the wake of the Inversion, who’ll be able to tell the difference?

CineMontage, a journal for the Motion Picture Editors Guild, talked to editors who worked on the NBC reality show, who say that the image of Donald Trump "was carefully crafted and manufactured in postproduction to feature a persona of success, leadership, and glamour, despite the raw footage of the reality star that was often 'a disaster.'"

"We were told to not show anything that was considered too much of a 'peek behind the curtain,'" one editor, Jonathon Braun, told CineMontage.

The editors say one of their biggest challenges was in the boardroom, making Trump's often whimsical decisions about who was fired instead look "legitimate."

"Trump would often make arbitrary decisions which had nothing to do with people's merit," an anonymous editor said. "He'd make decisions based on whom he liked or disliked personally, whether it be for looks or lifestyle, or he'd keep someone that 'would make good TV' [according to Trump]."

This required creative editing to set up the firings in a way that would make them seem logical, according to the sources, and while manipulative editing is standard in reality TV, this was apparently on another level.

Imagine you’re a small, fast-growing business. You attract £20m from a single, institutional investor. Suddenly, there’s a public relations crisis. You hire a specialist to deal with the fallout. The institutional investor is concerned, but they don’t really get involved. Their expertise lies in risk and return, not reputation.

Now, imagine that the same investor gets thousands of its employees to continuously defend your reputation on social media. Imagine they include it as a condition of the investment, for free.

You’ve just imagined crowdfunding.
...
It multiplies the size of the online community willing to participate (which in the case of Monzo is already bigger than those with investments). The £20m the company aims to raise this week could easily add well over 10,000 equity investors in its business (the investment limit is £2,000 per person). Instead of trying to control reputation through relationships with a small number of custodians, the sheer volume of support has the capacity to swamp critical voices.

Anyone who has followed security breaches and penetrations over the past few years knows that the track record of protecting data aggregations from exfiltration and subsequent disclosure or exploitation is very poor. And there are many examples of attackers that have maintained a presence in organizational networks and systems over long periods of time once they have succeeded in an initial penetration. While a tremendous amount of data has been stolen, we hear very little about data that has been compromised or altered, particularly in a low-key way. I believe that in the long term, compromise is going to be much more damaging and destabilizing than disclosure or exfiltration.

I want to explicitly note here the difference between the act of quietly rewriting the record and enjoying the results of the rewrites that are accepted as truth and that of deliberately destroying the confidence of the public (including the scholarly community) by creating compromise, confusion, and ambiguity to suggest that the record cannot be trusted.

Metrics should be the most real thing on the internet: They are countable, trackable, and verifiable, and their existence undergirds the advertising business that drives our biggest social and search platforms. Yet not even Facebook, the world’s greatest data–gathering organization, seems able to produce genuine figures. In October, small advertisers filed suit against the social-media giant, accusing it of covering up, for a year, its significant overstatements of the time users spent watching videos on the platform (by 60 to 80 percent, Facebook says; by 150 to 900 percent, the plaintiffs say). According to an exhaustive list at MarketingLand, over the past two years Facebook has admitted to misreporting the reach of posts on Facebook Pages (in two different ways), the rate at which viewers complete ad videos, the average time spent reading its “Instant Articles,” the amount of referral traffic from Facebook to external websites, the number of views that videos received via Facebook’s mobile site, and the number of video views in Instant Articles.

Safiya Noble, an associate professor at the University of California, Los Angeles and author of Algorithms of Oppression, told me in an email that “we are dependent upon commercial search engines to sort truth from fiction, yet these too, are unreliable fact-checkers on many social and political issues. In essence, we are witnessing a full-blown failure of trust in online platforms at a time when they are the most influential force in undermining or protecting democratic ideals around the world.”
...
Advertising’s shift to digital has cannibalized the news media’s revenue, thus weakening the entire public sphere. And linking advertising to pageviews incentivizes media organizations to produce articles that perform well, sometimes at the expense of material that educates, entertains, or holds power-holders accountable. Targeted advertising provides tools for political advertisers and propagandists to micro-segment audiences in ways that inhibit a common understanding of reality. This creates a perfect storm for authoritarian populists like Rodrigo Duterte, Donald Trump, and Jairo Bolsanaro to seize power, with dire consequences for human rights.

It is remarkable what moral compromises people will make provided the
right incentives. Intelligence and foreign policy analyst Malcolm Nance
recently mentioned the acronym used in the intelligence community for
motives behind people becoming spies. MICE: Money, Ideology, Compromise
or Coercion, and Ego. It was money that made “Sashko” lie for a living.

Filmmaker Kate Stonehill's "Fake News Fairytale" tells the story of a
Macedonian teenager who took up spreading fake news for a quick buck
until he saw what havoc his actions contributed to thousands of mile away:

“I think there’s a common misconception that people who
write fake news must have a nefarious desire to influence politics one
way or another,” Stonehill told The Atlantic. “I’m sure some of
them undoubtedly do, but I met many people in Macedonia, including
Sashko, who were writing fake news simply because they can make some
money.”

[...]

Stonehill believes the first step to combatting the seductive proliferation
of falsehoods is opening up an honest, critical discussion about
technology, speech, and politics in order to better understand the
fake-news phenomenon. “How, when, and why did the truth lose its
currency?” she asked. “Who is profiting when the truth doesn’t matter?
In my opinion, we’re only just beginning to unpack the answers to these
questions.”

"Who is profiting when the truth doesn’t matter?" is the important question. The answer is obvious - those with the power to shape the lie. Governments, not just authoritarian governments, and powerful corporations. Here are a few US examples:

We are at the end of a historic process of governments of all stripes failing to weigh the short-term benefits of lying to their citizens against the long-term costs of eroding belief in government pronouncements. The Cold War, the JFK, RFK and MLK assassinations, Vietnam, Iraq, the War on Terror, austerity, and the foreclosure crisis all featured blatant lying by the government. They have led to a situation in which no-one in the reality-based community believes anything they hear from the current administration. Greg Sargent's tweetstorm makes important points about Trump's lying:

Why does Trump lie *all the time* about *everything,* even the most trivial, easily disprovable matters?

The frequency and the audacity of Trump’s disinformation is the *whole point* of it -- to wear you down. More and more of the lies slip past, undetected and uncorrected.
...
Once Trump’s lying is understood as concerted and deliberate disinformation, it becomes clear that the frequency and audacity of it is *the whole point.*

Those are features of the lying. They are central to declaring the power to say what reality is

The other crucial half of this is to destroy the credibility of the institutional press.

Previous presidents have tangled with the media. But Trump’s ongoing casting of the press as the "enemy of the people" is in important respects something new:
...
Trump is *openly and unapologetically* declaring that norms of consistency and standards of interplay with the institutional press *do not* bind him.

Over the same period corporations have run massive long-term disinformation campaigns about pollution, smoking, and climate change, among others. And billionaire right-wing media owners have facilitated them, while using their reach to intimidate governments (see Hack Attack, Nick Davies' astonishing account of the Murdoch press' "information operations" in the UK).

A four-pronged approach to the new information warfare environment seems to be emerging. One prong is greatly improved forensics; this is a mostly technical challenge, and memory organizations will be mainly users, not developers, of these technologies. Documentation of provenance and chain of custody are already natural actions for memory organizations; the challenge here is to make this work more transparent and rigorous and to allow broad participation. Capture of materials, particularly in a world of highly targeted and not easily visible channels, will be a third challenge at both technical and intellectual levels (though we are seeing some help now from platform providers). Finally, contextualization of fakes or suspected fakes is perhaps the greatest challenge, and the one that is least amenable to technological solutions.

Even in the academic context, Lynch's prongs have issues:

Forensics will always remain one side of a co-evolutionary process. Machine learning merely accelerates both sides of it. And even during periods when forensics have the upper hand, they must be applied to have an effect. Unless they are built in to browsers and enabled by default, even most scholars will not bother to apply them.

Documentation of provenance and chain of custody maybe a natural action for memory organizations, but in an era of shrinking real budgets it isn't a priority for funding as against, for example, subscriptions.

Capture of open access academic content isn't a priority for funding either; capture of paywalled content is difficult and sure to become more so as Elsevier absorbs the whole of the academic workflow.

Contextualization is resource intensive and, as Lynch points out, hard to automate. So, again, it will be hard to justify adequate funding.

Poor research design and data analysis encourage false-positive
findings. Such poor methods persist despite perennial calls for
improvement, suggesting that they result from something more than just
misunderstanding. The persistence of poor methods results partly from
incentives that favour them, leading to the natural selection of bad
science. This dynamic requires no conscious strategizing—no deliberate
cheating nor loafing—by scientists, only that publication is a principal
factor for career advancement.

Outside the academic context, Lynch's prongs are wholly inadequate to the scale of the problem. It is one of those problems so common in this era of massive inequality whose costs are imposed on everyone while the benefits accrue to a small, affluent minority. Fixing them requires overwhelming coordinated political action, the very thing destroying trust in digital content is designed to prevent.

2 comments:

"Burnett has often boasted that, for each televised hour of “The Apprentice,” his crews shot as many as three hundred hours of footage. The real alchemy of reality television is the editing—sifting through a compost heap of clips and piecing together an absorbing story. Jonathon Braun, an editor who started working with Burnett on “Survivor” and then worked on the first six seasons of “The Apprentice,” told me, “You don’t make anything up. But you accentuate things that you see as themes.” He readily conceded how distorting this process can be."