Orwell Meets Frankenstein: The Internet as a Monster of Mass Surveillance and Social Control

The following paragraphs are from a talk delivered by Pinboard founder Maciej Cegłowski at the recent Emerging Technologies for the Enterprise conference in Philadelphia. Citing as Exhibit A the colossal train wreck that was the 2016 American presidential election, Cegłowski basically explains how, in the current version of the Internet that has emerged over the past decade-plus, we have collectively created a technology that is perfectly calibrated for undermining Western democratic societies and ideals.

But as incisive as his analysis is, I seriously doubt that his (equally incisive) proposed solutions, described later in the piece, will ever be implemented to any meaningful extent. I mean, if we’re going to employ the explicitly Frankensteinian metaphor of “building a monster,” then it’s important to bear in mind that Victor Frankenstein and his wretched creation did not find their way to anything resembling a happy ending. (And note that Cegłowski himself acknowledges as much at the end of his piece when he closes his discussion of proposed solutions by asserting that “even though we’re likely to fail, all we can do is try.”)

This year especially there’s an uncomfortable feeling in the tech industry that we did something wrong, that in following our credo of “move fast and break things,” some of what we knocked down were the load-bearing walls of our democracy. . . .

A question few are asking is whether the tools of mass surveillance and social control we spent the last decade building could have had anything to do with the debacle of the 2017 [sic] election, or whether destroying local journalism and making national journalism so dependent on our platforms was, in retrospect, a good idea. . . .

We built the commercial internet by mastering techniques of persuasion and surveillance that we’ve extended to billions of people, including essentially the entire population of the Western democracies. But admitting that this tool of social control might be conducive to authoritarianism is not something we’re ready to face. After all, we’re good people. We like freedom. How could we have built tools that subvert it? . . .

The economic basis of the Internet is surveillance. Every interaction with a computing device leaves a data trail, and whole industries exist to consume this data. Unlike dystopian visions from the past, this surveillance is not just being conducted by governments or faceless corporations. Instead, it’s the work of a small number of sympathetic tech companies with likable founders, whose real dream is to build robots and Mars rockets and do cool things that make the world better. Surveillance just pays the bills. . . .

Orwell imagined a world in which the state could shamelessly rewrite the past. The Internet has taught us that people are happy to do this work themselves, provided they have their peer group with them, and a common enemy to unite against. They will happily construct alternative realities for themselves, and adjust them as necessary to fit the changing facts . . . .

A lot of what we call “disruption” in the tech industry has just been killing flawed but established institutions, and mining them for parts. When we do this, we make a dangerous assumption about our ability to undo our own bad decisions, or the time span required to build institutions that match the needs of new realities.

Right now, a small caste of programmers is in charge of the surveillance economy, and has broad latitude to change it. But this situation will not last for long. The kinds of black-box machine learning that have been so successful in the age of mass surveillance are going to become commoditized and will no longer require skilled artisans to deploy. . . .

Unless something happens to mobilize the tech workforce, or unless the advertising bubble finally bursts, we can expect the weird, topsy-turvy status quo of 2017 to solidify into the new reality.

Related

About Matt Cardin

Teeming Brain founder and editor Matt Cardin is the author of DARK AWAKENINGS, DIVINATIONS OF THE DEEP, A COURSE IN DEMONIC CREATIVITY: A WRITER'S GUIDE TO THE INNER GENIUS, and the forthcoming TO ROUSE LEVIATHAN. He is also the editor of BORN TO FEAR: INTERVIEWS WITH THOMAS LIGOTTI and the academic encyclopedias MUMMIES AROUND THE WORLD; GHOSTS, SPIRITS, AND PSYCHICS: THE PARANORMAL FROM ALCHEMY TO ZOMBIES; and HORROR LITERATURE THROUGH HISTORY.

About

The Teeming Brain explores news, trends, and developments in religion, horror, science fiction, fantasy, the paranormal, creativity, consciousness, and culture. It also tracks apocalyptic and dystopian trends in science, technology, politics, ecology, economics, the media, the arts, education, and society at large. Its founder and primary author is Matt Cardin.

Subscribe

Enter your email address to subscribe to The Teeming Brain Weekly, a digest of each week's content delivered to your in-box every Sunday:

Email

Email Format

html

text

mobile

Or subscribe via RSS using your favorite reader:

Search this site

Search for:

Support Us

If you enjoy The Teeming Brain's ongoing exploration, please consider making a donation to help defray the time and costs associated with this project.

OPTION 1: You can make a recurring (and voluntary) monthly donation of $3:

OPTION 2: You can make a one-time donation in any amount you choose:

Whichever option you choose, be sure to subscribe (for free) to our blog updates as well. See above. And thank you sincerely for your support!

Praise for the Teem

FOR MATT CARDIN:

"[Dark Awakenings is] a thinking-man's book of the macabre...Cardin's tales are rich with references to Lovecraft, Nietzsche, and other writers whose work gives them unusual philosophic depth." – Publishers Weekly

“It’s a bold writer who, in this day and age, tries to make modern horror fiction out of theology, but Cardin pulls it off.” – Darrell Schweitzer

“In the tradition of Poe and Lovecraft, Cardin's accomplishments as a writer are paralleled by his expertise as a literary critic and theorist.” – Thomas Ligotti

“Matt Cardin is one of those rare horror authors who is also a true scholar and intellectual.” – Jack Haringa

FOR RICHARD GAVIN:

"Literate horror fans who have yet to encounter Canadian author Richard Gavin are in for a treat. The lyrical prose is often at a higher level than usual presentations of otherworldly demons and malevolent forces." – Publishers Weekly

"Richard Gavin is one of the bright new stars in contemporary weird fiction. His richly textured style, deft character portrayal, and powerful horrific conceptions make every one of his tales a pleasure to read." – S. T. Joshi

"Gavin's storytelling can be masterly. As with Machen and Blackwood at their best, an epiphany or illumination is achieved, though Gavin's mysticism is darker and distinctly his own." – Wormwood

FOR STUART YOUNG:

"No one can accuse Stuart Young of avoiding the big issues -- with insight and verve, he tackles head-on the existence of God, the mystery of human consciousness and the transformative effects of psychedelic drugs." – Mark Chadbourne