I am honorbound not to comment on this link, for obvious reasons, but boy howdy is there a lot to say that didn't make it into the piece.posted by adamgreenfield at 3:37 PM on February 18 [61 favorites]

adamgreenfield'sMeFi's own adamgreenfield's Meffy just went stratospheric. I hope it doesn't destabilize the market.posted by mwhybark at 3:54 PM on February 18 [6 favorites]

If you have Netflix, watch "Nosedive". If you don't have Netflix, this scene gives you a pretty good peek at the horrors within.posted by JoeZydeco at 4:07 PM on February 18 [3 favorites]

What China does today, Russia and Turkey do tomorrow, and a UK cabinet minister suggests the following day.

Though, to be fair, the Western equivalent will be run by either Facebook (which is already becoming a system for self-reporting on one’s life to those with power over one) or Experian, or some combination of both.posted by acb at 4:19 PM on February 18 [7 favorites]

Actually, I’m sorry, I will say something:

It makes me really sad that so much of the response to this piece has been, “Gee, it’s just like an episode of Black Mirror.” (The same thing happened with the new Boston Dynamics video the other day.) It’s really made me rethink the role of that show in the culture: how it works, what effects it has, what it does. It seems to me to be dulling our capacity for preventative disgust, in that we see something like social credit, or a military robot capable of operating in domestic environments, and nod and say, “Yeah, that’s some real Black Mirror shit right there, huh?” and go back to the thing we were doing before.

And that doesn’t, you’ll forgive me, seem like a superhealthy response. I’m not accusing anyone here if doing that, necessarily, just noticing how often the show is invoked as a kind of palliative or preemptive gesture of learned helplessness.posted by adamgreenfield at 4:22 PM on February 18 [154 favorites]

Known by the anodyne name “social credit,” this system is designed to reach into every corner of existence both online and off.

Wasn't that also the name of the crankbait economic theory that Ezra Pound became a follower of? It was; and it was Stage IV Engineer's Disease.posted by thelonius at 4:24 PM on February 18 [4 favorites]

This peril is a necessary consequence of our capitalism-driven pretense that the word "private" is some kind of magic pixie dust that makes large-scale organization and activity under that flag fundamentally different from a government doing the same thing.

If it's the "private" East India Company taking over a subcontinent, that's a totally different thing from a nation conquering territory, until it isn't different. Political parties are "private" organizations, so their internal operations aren't really integral parts of government and so don't need to be accountable in that way. "Private" "credit reporting agencies" can trawl through the data of consumer-facing companies and centrally compile it exactly the same way the Nazis and IBM did, but it's a totally different thing from a government doing that. "Private" tech companies can use their technology to conduct ubiquitous, pervasive surveillance on everyone, and put the information produced in a database and sell it, but no siree there's no unconstitutional search and seizure happening if the U.S. government or other society-wide interests end up using the information from that database.

Obviously a nation without even nominal scruples about authoritarianism may come up with better, centralized, directly-government-controlled solutions in some cases for doing these sorts of things than will our fragmented "private" approaches, and when it's only euphemisms and fig leaves keeping those activities separate from the government, there won't be much resistance to doing away with the facade and adopting the superior solution from the authoritarian innovator.posted by XMLicious at 4:26 PM on February 18 [41 favorites]

Regarding other fictive explorations of social credit systems, may I heartily reccommend Gary Shteyngart's Super Sad True Love Story which crosses Black Mirror and Mr. Robot (avant le, eurh, lettre) with Tolstoy and Nabokov? I read it in one sitting on the beach within sight of the San Onofre nuclear plant and was rewarded with a full-body sunburn bad enough to both scar and require medical attention.posted by mwhybark at 4:28 PM on February 18 [13 favorites]

Private" "credit reporting agencies" can trawl through the data of consumer-facing companies and centrally compile it exactly the same way the Nazis and IBM did, but it's a totally different thing from a government doing that

I mean, yeah. In my opinion, how we allow credit agencies to operate in ways that - you can’t discrim based on status, except a number a private agency comes up with that is the history of your life? I’ve seen so many people essentially homeless on this basis, so while I have huge concerns about this, I don’t think we have a lot of room to think we’re so very different. The Chinese system is worse, but our overall system is still pretty bad.posted by corb at 4:40 PM on February 18 [12 favorites]

This peril is a necessary consequence of our capitalism-driven pretense that the word "private" is some kind of magic pixie dust that makes large-scale organization and activity under that flag fundamentally different from a government doing the same thing.

It depends. The difference is that a private version in the guise of credit rating agencies has effects on you in the public sphere but people aren't forced to observe their guidance like a theoretical government controlling society using this data. For instance, the Chinese government says I can't rent an apartment in area X because I'm an undesirable because I have too many vices. But in a Western country if I have a friend or relative that has an apartment, even if the credit rating agencies say "don't trust this guy as far as you can throw him", I can still live in an area.

I haven't seen Black Mirror, but this seems like some 1984 shit, which is absolutely terrifying.posted by Weeping_angel at 4:47 PM on February 18 [1 favorite]

I’m not accusing anyone here if doing that, necessarily, just noticing how often the show is invoked as a kind of palliative or preemptive gesture of learned helplessness.

I think the problem is the learned helplessness, more than anything. There's a commonality between the deterioration of the internet and, say, the regular mass shootings in the US, where no-one in a position to fix it seems capable or inclined to try.posted by Merus at 5:02 PM on February 18 [9 favorites]

I was thinking down the same Schteyngartian path as mwhybark, and ended up reading this review of SSTLS, the ending of which seems particularly dated even though it was written just five years ago:

The satire...seems hyperbolic, as if every feature that could be exaggerated is magnified by a hundred. For example, Americans love shopping so in the novel there are specialized poles which check the credit rating of passersby while billboards associate patriotism with spending. While satire is inherently born out of exaggeration the premise and setting of this novel seem particularly shrill.

I read the book when it first came out, and really enjoyed it, but at the time would've understood where this reviewer was coming from. Five years later in 2018, by contrast, this reviewer's take is no longer reasonable (esp. if you've read adamgreenfield's article!): the satire was clearly spot on.

To me this brings home how quickly sea changes in how we live are taking place. Because the technology exists and they can profit from it financially or in terms of control, businesses and governments are conducting massive social experiments that would have seemed crazy-impossible to most just five years ago. Tech operates with an "it's easier to ask forgiveness than permission" attitude, but they're playing at such a grand scale these days that when they get something wrong they can really Fuck. Things. Up. It's unsettling.

Learned helplessness makes sense as a response. "It's all so big, what can I possibly do to stop it? Where are the insititutional responses this stuff deserves?" It's like a fast-motion version of what we've done to the environment and the (long) time it took for the mainstream to accept that it's real and that we're at fault and that we could have done/can still do (?) something to ameliorate the damage we've done.

We haven't responded to climate change very effectively. It's difficult to be confident that we'll do any better with this. I'll fight the good fight until the day I die, but honestly learned helplessness is entirely too understandable these days.posted by Lyme Drop at 5:50 PM on February 18 [16 favorites]

Oh, sure, when China does it, suddenly it's scary and dystopian. When it's just a bunch of unaccountable corporations, no big deal.posted by tobascodagama at 5:51 PM on February 18 [43 favorites]

Oh, sure, when China does it, suddenly it's scary and dystopian. When it's just a bunch of unaccountable corporations, no big deal.

To me this brings home how quickly sea changes in how we live are taking place. Because the technology exists and they can profit from it financially or in terms of control, businesses and governments are conducting massive social experiments that would have seemed crazy-impossible to most just five years ago.

We need Sen. Warren & Co. to come in and shut those fuckers down. Legislative solutions are the only ones that work. In a sane world Mark Zuckerberg should have been dragged through endless inquiries into his company's intervention into our free and fair elections, and slapped with massive fines and brutal injunctions.posted by leotrotsky at 6:34 PM on February 18 [13 favorites]

Yeah, my first response was: we already do this in North America, it's just skin colour + credit score.

And it works exactly as well here as it will in China, which is to say, great for the people who already have it great, shitty for the people who already have it shitty.posted by klanawa at 6:42 PM on February 18 [19 favorites]

The difference is that a private version in the guise of credit rating agencies has effects on you in the public sphere but people aren't forced to observe their guidance like a theoretical government controlling society using this data. For instance, the Chinese government says I can't rent an apartment in area X because I'm an undesirable because I have too many vices. But in a Western country if I have a friend or relative that has an apartment, even if the credit rating agencies say "don't trust this guy as far as you can throw him", I can still live in an area.

Which in this case is kind of a world of difference.

I wasn't trying to say that Chinese practices and those of other countries are indistinguishable, just that the accumulated moral hazards from allowing large national and international organizations which are "private" to do things which would be shunned under the orchestration of a government—shunned for the mere potential for abuse—has weighted the scales so that nominally non-authoritarian governments may very quickly adopt practices from authoritarian governments with minimal resistance. Especially as convergence between in-no-remaining-way-that-can-be-taken-seriously-Communist China and the rest of the world continues.

But I have to point out that the specific example you provide, a friend or relative helping someone to evade the strictures placed on the populace at large, does not appear to be a phenomenon absent from China by any means. In fact as I understand it (from casual reading) the strength of such patronage relationships and networks and their historical ability to supersede laws, court decisions, and regulations was one of the major obstacles to foreign enterprises establishing themselves in China during the past few decades, even in some cases despite the blessing of the central government.posted by XMLicious at 6:58 PM on February 18 [10 favorites]

I was struck by the line about how people were refusing to use the reporting tool because the Cultural Revolution was still in their living memory. How long will that last? As we’re seeing in the U.S., some shared values begin to deteriorate when the generation that forged them disappears from public life.posted by chimpsonfilm at 6:58 PM on February 18 [24 favorites]

I’m not accusing anyone here if doing that, necessarily, just noticing how often the show is invoked as a kind of palliative or preemptive gesture of learned helplessness.

I think the problem is the learned helplessness, more than anything.

Yeah, I don't think Black Mirror and similar have any real responsibility for producing or even reinforcing that attitude - the tendency to casually dismiss stuff as "real Black Mirror shit" is just a current(ish) manifestation of a long-evident, and probably inherent, human tendency to dismiss lots of Real Scary Shit on the basis that it doesn't immediately affect you and, fingers crossed & you do the Right Thing, won't ever.

Historically, you could just as easily put the blame on any Twitter feedFacebook newsfeedblogTV networknewspaperliteraturepolitical ideologysocial classreligion you subscribe to…posted by Pinback at 7:13 PM on February 18

Death by a thousand algorithms, and China's just doing what NK does, only in a more indirect passive aggressive manner, and enough with the Black Mirror comparisons. Dystopian writers have been flirting with these concepts for over a century. The classic TV series The Twilight Zone practically strip mined them.posted by Beholder at 7:45 PM on February 18

I am reminded that I am overdue to check in on that puck of Brighton Beach, the one, the only Gary Shteyngart.posted by mwhybark at 7:54 PM on February 18

It makes me really sad that so much of the response to this piece has been, “Gee, it’s just like an episode of Black Mirror.”

If it helps any, such notions of social engineering scare the fucking willies out of me.posted by Greg_Ace at 8:27 PM on February 18 [1 favorite]

Dealing with institutions that rely on credit scores was an interesting and often frustrating experience as an international student who couldn’t get a credit score by definition. None of the paperwork that I had to submit to US Immigration proving I could pay my own way so that they’ll give me a visa mattered. The letters from my school saying what my situation was didn’t make a difference. No credit score? Automatically untrustworthy. When I wanted to rent I had to do a sublet, get a local to be my co-signer, or take over a friend’s lease.

How do immigrants and anybody who’s not a citizen fare under systems like these?posted by divabat at 8:36 PM on February 18 [8 favorites]

we see something like social credit, or a military robot capable of operating in domestic environments, and nod and say, “Yeah, that’s some real Black Mirror shit right there, huh?” and go back to the thing we were doing before.

A decade ago, it would have been "Yeah, that's some real Matrix shit." a decade or two before that, it would have been "That's some real William Gibson shit". A couple decades before that, "That's some real fill in the name of some Next Wave writer shit." For a given decade, we pick up the SF metaphor that fits.
And that doesn’t, you’ll forgive me, seem like a superhealthy response.

I think this would be best described as a "crime against humanity". Why aren't we embargoeing the fuck out of these fascist motherfuckers?posted by sexyrobot at 10:38 PM on February 18 [1 favorite]

many reasons, i assume Americans are profiting from this somehow. plus they did use our credit score system as a model -- and there have been a ton of US tech companies in the past few years trying to sort of generalize your credit score based on social media posts, purchasing history, and so on, although fortunately none seems to have really taken off.

plus I bet it sounds like a good idea to a lot of people in power, at least when they're among themselves.

i nearly had an Irish (EU) passport, if my parents had filled out some forms a few months sooner. i desperately wish I had one now.posted by vogon_poet at 11:10 PM on February 18 [2 favorites]

I think this would be best described as a "crime against humanity". Why aren't we embargoeing the fuck out of these fascist motherfuckers?

Embargoing China? Without China, the US is toast. They own like 20% of our debt, to the tune of about 1.2 trillion, last I checked.posted by xyzzy at 12:18 AM on February 19 [5 favorites]

What China does today, Russia and Turkey do tomorrow, and a UK cabinet minister suggests the following day.

This is basically ASBOs systemised and digitalised, isn't it? Blair would've loved this had the technology been ready for it in 2002.posted by MartinWisse at 12:24 AM on February 19 [6 favorites]

It might not be. What are you doing about it?

You mean... apart from writing a long magazine article giving a bunch of details and background to hammer home that it’s not a deliciously arch satire but the real thing, already invading people’s lives?posted by No-sword at 12:32 AM on February 19 [26 favorites]

Interesting article would love to have heard some Chinese voices in it. I have found many Chinese are well aware, and very critical, about the petty surveillance they endure - while there are also a disappointing number of little Red Guards as well.

I do think while the technology is new, the behaviour isn't in that you could look to east Germany for the effects of pervasive surveillance on the populace, or soviet Russia for the effect of provisioning services based on "loyalty".

And indeed I wonder of this might not be the scheme's undoing: ultimately it will be as opaque and arbitrary as harassment the CCP already engages in. The party's own corruption and changeable nature may work against it, we see the corrosion of corruption on university entrances, resident permits, number of children, and more already. I see this being employed more as a(nother) blunt and unfair tool for control and punishment, too unreliable to become a "system" rather than a method of perpetuating "loyalty" etc. China is less homogenous than people think in some ways, this tool and its uses will be corrupted like everything else, I suspect.posted by smoke at 2:35 AM on February 19 [2 favorites]

I'm totally with tobascodagama and klanawa on this and would like to add a perspective of someone from a post-communist country.

I mean, the crude communist utopia of the 19th century needed a nation of people who weren't there and when it found out, it wanted to hammer people into form using the rough tools of persecution and vague promises of a perfect future.
China is just this utopian communism evolving and learning, going down the engineering route which, let's face it, works quite well for the capitalist utopia right now.

Let's take a capitalist utopia: free enterprise, universal income, separation of wealth from political power.

How are we in the West doing on this?
(i) There are legions of other people working hard to manipulate your every decision, dream and activity to suit their goals in the name of free enterprise; (ii) universal income is fairly certain to come once those with wealth and power realize that to keep their privilege they will have to throw the scum some crumbs, and the last one, well, as long as (ii) happens, it is postponed indefinitely.

And China? Designing a system which uses basically the same input data that our corporations use, adding softer and harder incentives and lacing the result with softer and harder penalties, all in the name of lubricating the gears of an overpopulated economy and bracing it against the hard crash once the environment finally builds a hard wall right in front us.

I don't think it is completely reprehensible as an attempt, as good as letting the googles, amazons and facebooks of this world decide where the society will go and the billionaires buy presidents and their agendas in the supposedly democratic societies.

As for pet cases like the lawyer who was denied a flight and had to traverse 1200 miles by, what horror, a train, or maybe a car? How inhumane. The U.S. No Fly List is basically the same thing and I'm sure there are people even here on MeFi who would gladly see it extended to corrupt politicians and spineless lawyers.

(All this said, I'm absolutely no fan of utopias because they just don't work, and I'm no fan of globalized capitalism or the Chinese way either.)posted by Laotic at 2:55 AM on February 19 [1 favorite]

Missing from this conversation is Cathy O'Neil, whose splendidly titled _Weapons of Math Destruction_ is essential reading.
There she draws our attention to many problems, including the generation of e-scores to map people when credit scores are blocked.posted by doctornemo at 5:35 AM on February 19 [2 favorites]

A direct citation of Cathy's work was cut from the article. I agree that her book is a cornerstone of any conversation regarding the algorithmic regulation of everyday life, and discuss it extensively in my own Radical Technologies.posted by adamgreenfield at 5:38 AM on February 19 [13 favorites]

Which I'm looking forward to reading. It's on The Stack (TM).posted by doctornemo at 6:28 AM on February 19

For the record, especially a record being reviewed by a powerful AI from the future to determine my social worth via examination of my digital past, I would just like to say that my very sincere love of my country and its laws only grows as time goes by.

I love the smell of normality in the morning, and my heart is weighed down by the errancy of those who would refuse to perform their proper duty to society financially, emotionally, or physically. I only wish there were more and easier ways for reporting bad citizens to the great database, but I know that technological innovation proceeds at a pace judged best by the basilisk, and for that I am grateful.

Another American corporation that does this is Uber. They keep scores for customers just like they do for drivers, and if your score is low enough you can't get a ride. And of course we also had the odious company Klout, which happily failed in their effort to create this kind of score. I can't decide if I think it's better or worse that governments do reputation systems rather than companies. I do think one centralized reputation score is terrible.

Embargoing China? Without China, the US is toast. They own like 20% of our debt, to the tune of about 1.2 trillion, last I checked.xyzzy

This is not how national debt works, despite how often people like to bring it up in discussions of China, or Japan before them, "owning" the US.posted by Sangermaine at 10:35 AM on February 19 [7 favorites]

As for pet cases like the lawyer who was denied a flight and had to traverse 1200 miles by, what horror, a train, or maybe a car? How inhumane.

I would be more reticent in minimising the strictures the CCP places on activitists, were I you. Its no joke. The article mentions people could not catch a trains, either, but even that is trivialising the pressure the CCP can exert even without this algorithm.posted by smoke at 12:09 PM on February 19 [9 favorites]

It makes me really sad that so much of the response to this piece has been, “Gee, it’s just like an episode of Black Mirror.” (The same thing happened with the new Boston Dynamics video the other day.) It’s really made me rethink the role of that show in the culture: how it works, what effects it has, what it does. It seems to me to be dulling our capacity for preventative disgust

I'm really struggling to understand why you think this, and why so many people agree with you. Wouldn't exactly the same apply to 1984, A Clockwork Orange, Brave New World, A Modest Proposal, etc etc? People say "it's just like 1984" and have done for decades. People compare reality to satire all the time, and I just don't understand what that has to do with making them less inclined to take action. Can you explain? Because I'm not getting it but a lot of people evidently do.posted by howfar at 12:32 PM on February 19 [4 favorites]

This is not how national debt works

I did not say that China owned the US. I said it owned US debt. Ask any economist what would happen if China quickly dumped all its debt in retaliation for an embargo.posted by xyzzy at 2:24 PM on February 19

I'm really struggling to understand why you think this, and why so many people agree with you. Wouldn't exactly the same apply to 1984...People say "it's just like 1984" and have done for decades.

Well, I obviously can't speak for anyone else, but for me it has something to do with what people were comparing to Nineteen Eighty-Four vs what is getting compared to Black Mirror.

I mean, in all honesty, I have always disliked that sort of comparison, because then as now it's a thought-ender — a module of predigested language that serves as a terminal point, and tends strongly to prevent further consideration and inquiry. But let's assume for the sake of argument that there was something different about the comparison to Nineteen Eighty-Four that made it somewhat less abrasive.

Here's what I think it is: generally, when someone invoked Nineteen Eighty-Four, they were doing so to call out a symmetry or resonance between something in the real world, and the pervasive regime of observation, reality control and destruction of the individual that Orwell had crafted for his Oceania. This is quite a specific set of circumstances, really, and the Ingsoc regime manifested itself in ways that, if they worked wonderfully as metaphor, were also rather definite and concrete: memory holes, the Two Minutes' Hate, Room 101 and so on. Whether or not the comparison was apposite — whether or not those making the comparison had even read Nineteen Eighty-Four — it seemed to me that there was a certain amount of understanding in common that these were Bad Things and that any party on Earth dedicated to implementing them as policy was by definition Bad People.

Whether you happened to think that party was the Republican Party of Dick Cheney, the Socialist Unity Party of Erich Honecker or what have you, the comparison to Nineteen Eighty-Four was itself a rhetorical weapon, and you deployed it toward the defeat (however eventual or unlikely) of the enemy in question.

But when people make the comparison to Black Mirror, there isn't exactly a concrete set of circumstances they're responding to so much as a gestalt, and not so much an identifiable set of bad actors as a general, inchoate sense that something has gone wrong in the space between culture and its sociotechnical underpinnings ("What if smartphones...but too much?"). The comparison is rarely if ever used to indict a specific institution or mode of thought, only to indicate that one is au courant with respect to the bad juju in the air. It's amazingly nonspecific, nondirected and nondirective.

So when people drop the shibboleth into a discussion these days, it just doesn't feel like they're saying, "I recognize that China has embarked upon an extraordinarily dangerous path here, in a way that helps me dissect out the almost-equally-as-dangerous dynamics in my own culture, and I will do everything in my power to prevent that sort of thing from developing any further here." Or "Boston Dynamics has unaccountably dedicated themselves to a headlong and irresponsible rush toward developing a set of capabilities that has roundly been understood to be inimical to human life for at least the past two hundred years of fantastic literature, and we should take such measures as are necessary to prevent their success in this endeavor." It feels like they're saying "Yeah, that's straight-up some Black Mirror shit, huh?", and not a whole hell of a lot more.

That, anyway, is what the current situation feels like to me. I obviously haven't put any particularly deep thought into this account of it, but there it is. Like I say, it makes me sad.posted by adamgreenfield at 3:34 PM on February 19 [14 favorites]

Whether or not the comparison was apposite — whether or not those making the comparison had even read Nineteen Eighty-Four — it seemed to me that there was a certain amount of understanding in common that these were Bad Things and that any party on Earth dedicated to implementing them as policy was by definition Bad People.

This sounds like the core of it, I think. Were 1984 published now, or Black Mirror then; I'm pretty sure the dynamic wouldn't be any different. What's different is a sense of unification, where you could go "This is a universally bad thing", and a sufficiently significant amount of people would agree with it. Now not even Nazis are in that category, if contemporary politics is any indication. If you can't get everyone to go "literal white supremacy is bad", how can you get people to agree on "creating systems of interlocking soft power between corporations & governments and powering it with unverifiable broad data surveillance inputs will lead to bad things happening"?

Or, to reframe it, you're thinking about historical comparisons to 1984 from the perspective of people being in the decision loop, where it was something which people could feel connected to or some sense of being able to influence it.

But comparisons to Black Mirror are by-and-large from the outside, where these things are happening and there's such a feeling of disconnection from being able to have an influence, even when millions of people turn out to protests, that it's bad but what are you going to do about it? And even if you do gain ground, how long before it becomes co-opted to serve the system? (Which, for a more concrete comparison, look to Fifteen Million Merits, S01E02)

Ultimately, this sounds to me like a combination of nostalgia for the past & systemic learned hopelessness in the present. Or, to borrow an adage I've seen in many different forms but can't find the origin of: We (as youth/anyone under 30-ish?) weren't promised flying cars, we were promised corporate dystopia. Why be surprised that people's expectations are matching what they've seen?posted by CrystalDave at 3:51 PM on February 19 [2 favorites]

My objection to referencing Black Mirror is that it's a niche show a lot of people have not seen, but fans drop references to it like it's Star Trek. It always sounds like you're talking about to an idiom I've never heard. "What the hell does that mean, look in a black mirror. What's a black mirror?"

Nineteen Eighty-Four has been part of the culture for half a century, but Black Mirror hasn't earn that space. Besides, for every episode of Black Mirror, there are probably a dozen episodes of The Twilight Zone that made the same point.posted by riruro at 4:01 PM on February 19

Fifteen Million Merits among other things is Charlie Brooker leveling the criticism here of Black Mirror at himself as early as the second episode of the first season of the show. The co-opting of the protagonist at the end is pretty much a parody of Brooker's role as a media critic who's using the same media to make his point.posted by servoret at 4:18 PM on February 19 [1 favorite]

It's certainly wise to not get complacent about large corporations, particularly as they have achieved a staggering degree of regulatory capture particularly in the US (seemingly less so in Europe, but maybe I'm just further from it and can't see it as well). But there are still big differences between corporations—or for that matter, other people—doing something, and the state doing it.

As long as the state maintains an exclusive monopoly on violence, it has the final veto power: Ultima ratio regum is still alive and well. Sometimes they delegate that power down to private, nongovernmental entities—e.g. the East India Company, or arguably Blackwater—and that should be worrying, but it tends to be the exception rather than the rule.

When a corporation does something wrong, you can—in theory at least—always appeal to the government; when the government does something, you're shit out of luck, unless the government is interested in hearing your complaints about itself, which is typically an uphill battle. (Realistically, it becomes a question of whether you can find one part of government interested in, and hopefully formally empowered to, push around another part of government; but there's no natural hierarchy there to fall back on.)

So while I'm not saying we shouldn't be worried, I'd much rather be in a position where I'm lobbying the government to smack around a private credit ratings agency using its extant enforcement mechanisms and (if necessary) police powers (which is a polite term for "violence"), than trying to lobby the government to change how it itself does something, with the vast institutional inertia working against any change.posted by Kadin2048 at 7:09 PM on February 19 [2 favorites]

It always sounds like you're talking about to an idiom I've never heard. "What the hell does that mean, look in a black mirror. What's a black mirror?"

A black mirror is a scrying stone, a reflective surface, preferably made of obsidian, volcanic glass, on which a practitioner of the magic arts can see the future if properly, what, incanted and ritually prepared. They enter Western magical tradition after the Spanish conquest of the Americas, where polished obsidian surfaces were used in religious ritual.

Perhaps my response here is orthagonal to your point, but the reference would appear to be well established.

adamgreenfield, this article is very interesting indeed, and thanks both for your work on it and your interaction here. I'm among those who doesn't fully grasp your perception of the use of Black Mirror as a dismissive reference, although I do think this thread is contributing usefully to unpacking the concern you express. I'm also a bit skeptical about lionizing 1984 in contrast to the series; certainly 1984 and Animal Farm were cited with smug certainty by the teachers and politicians of my youth as proof positive that western capitalism was superior to Chinese and Soviet socialism at the same time as the books were deployed within socialist countries as teaching tools regarding shortcomings in Western political and economic systems (or so I recall, I suppose I ought to source that).

riruro's point, despite my response, is non-trivial. only digitally-privileged persons even have the opportunity to view the series, and therefore its cultural footprint is minimized.posted by mwhybark at 7:46 PM on February 19 [1 favorite]

...there are still big differences between corporations—or for that matter, other people—doing something, and the state doing it... As long as the state maintains an exclusive monopoly on violence, it has the final veto power: Ultima ratio regum is still alive and well. Sometimes they delegate that power down to private, nongovernmental entities—e.g. the East India Company, or arguably Blackwater—and that should be worrying, but it tends to be the exception rather than the rule.

I think you're making distinctions that don't really exist, but even beyond that the nominally-private exercise of force and violence outside of the supposed constraints of law are nowhere near as rare as you seem to be conveying. Certainly not globally, but for example the 1896 Plessy v. Ferguson court decision which entrenched racial segregation in the U.S. for the majority of the twentieth century was technically the result of a railroad company detaining and handing over to the "justice" system Homer Plessy, who was one of their passengers. (Though since it was a test case advanced by a civil rights organization, the circumstances in that specific incident were somewhat arranged.)

And although it feels like it also happened more than a hundred years ago, it's only 18 months since a "private security firm" (there's that word again!) used literal attack dogs against Dakota Access Pipeline protesters and most of the people in the country appeared to not find it terribly remarkable. Unsurprisingly it was revealed that TigerSwan internally regarded the protesters to be terrorists and insurgents, conducted extensive surveillance against them, closely cooperated with law enforcement at several levels, furnished those surveillance records to the government in support of further use of force in the form of legal action, and conducted a propaganda campaign against the protesters via social media.

In the case of segregation, I don't think it makes sense to regard the businesses which excluded non-white people from commerce and public facilities and the displays of violence that served to keep those people in line as separate phenomena or delineate them in a government versus non-government way.

By the same token, if tech companies conduct intrusive surveillance on all aspects of our lives, and the companies that run highway toll booths are the ones with the array of license plate readers that can track you as you move around in your car but the government is the one separately compelling every car to have a unique license plate, and the government spies much more concertedly on those companies to tap into their data collection or the companies technically have to voluntarily cooperate... but the surveillance apparatus still exists and is at the disposal of the government and other ruling forces, what does the public/private distinction matter? What do the words "Our government doesn't really spy on us, not directly" even mean, in this context?

It's like the conversion of "employees" to "contractors" and the accompanying loss of leverage and protections and compensation: the details and terminology of how we describe what's happening don't matter, not really, and a person from a thousand years in the past or a thousand years in the future would recognize it for what it is—wealth and power doing what wealth and power does.

China's social credit system is certainly more versatile, and can be more precisely calibrated from the apex of society, than a hydraulic empire demanding tributes from lords and their vassals and their vassals' vassals all the way down in ancient Mesopotamia, or a priest holding forth from the pulpit about rendering to Caesar that which is Caesar's, or a caste system; but it's fundamentally the same thing.

I somewhat prefer the pretexts and fictions with which we dress up the flows of power and privilege in our nominally-less-authoritarian-than-China modern societies, but if there's anything we've learned in the last few years it's that we shouldn't regard those fictions and pretexts as more substantial or less ephemeral than they really are. Vast institutional inertia has been notably steamrolled or simply ignored on at least a few occasions which come to mind. Or, we've discovered that the inertia is actually going in a decidedly more authoritarian and dystopian direction than we'd previously thought, as in the case of the Snowden revelations.posted by XMLicious at 2:41 AM on February 20 [10 favorites]

China's social credit system is certainly more versatile, and can be more precisely calibrated from the apex of society, than a hydraulic empire demanding tributes from lords and their vassals and their vassals' vassals all the way down in ancient Mesopotamia, or a priest holding forth from the pulpit about rendering to Caesar that which is Caesar's, or a caste system

...or the oldest social credit system in the world, which is generally referred to as "wealth".

The amount of money a person can wield is no more and no less than a measure of society's collective willingness to allow that person to do whatever the fuck they want whenever the fuck they want. People with high money scores don't need discounts on apartments because they can just own the apartments.

I somewhat prefer the pretexts and fictions with which we dress up the flows of power and privilege in our nominally-less-authoritarian-than-China modern societies

The data centre is looking for another dozen or so hires for its launch, underlining the ravenous appetite among asset managers to snap up more quantitative analysts adept at trawling through data sets like credit card purchases, satellite imagery and social media for investment signals.

This is an extremely misleading headline from someone who should know better. All he's talking about is the fact that later this year, the Chrome browser will begin to mark all web sites which don't accept an encrypted https: connection as "insecure" and the user will have to consciously override that to connect unencrypted. But you won't have any problem with actual archival sites like the Internet Archive because they already serve everything through https:.posted by XMLicious at 12:41 PM on February 21 [1 favorite]

The problem with Google’s actions should be obvious: the company is leveraging its monopoly in search to push the AMP format, and the company is leveraging its dominant position in browsers to punish sites with bad ads. That seems bad!

And yet, from a user perspective, the options I presented at the beginning — fast loading web pages with responsive designs that look great on mobile and the elimination of pop-up ads, ad overlays, and autoplaying videos with sounds — sounds pretty appealing!

This is the fundamental paradox presented by aggregation-based monopolies: by virtue of gaining users through the provision of a superior user experience, aggregators gain power over suppliers, which come onto the aggregator’s platforms on the aggregator’s terms, resulting in an even better experience for users, resulting in virtuous cycle. There is no better example than Google’s actions with AMP and Chrome ad-blocking: Google is quite explicitly dictating exactly how it is its suppliers will access its customers, and it is hard to argue that the experience is not significantly better because of it.

At the same time, what Google is doing seems nakedly uncompetitive — thus the paradox. The point of antitrust law — both the consumer-centric U.S. interpretation and the European competitor-centric one — is ultimately to protect consumer welfare. What happens when protecting consumer welfare requires acting uncompetitively?

In any properly run world, virtuous monopolies would be recognised as natural monopolies, and get nationalized.

In our world, the private corporations that run these monopolies will incrementally gain powers to compel and coerce via the threat of convenience removal that were formerly available only to governments via the threat of physical violence, and we will slide slowly but conveniently into unrepresentative feudalism.posted by flabdablet at 4:00 AM on February 23

There is an alternative approach: direct public provision. Many online services, most obviously matching platforms such as Uber, or universal social networks (those used by everyone rather than special communities), have clear public utility characteristics. So why not provide such services publicly? In other words, where governments see that private actors have identified a need for a digital marketplace (or even invented it), they should consider taking responsibility for providing the marketplace themselves. That’s because it is precisely when economic activity centres itself around a single platform that the market failures which we discussed on Monday — forces encouraging excessive market power — appear...

Furthermore, authorities can require online service providers to plug into related public services such as tax collection. The Estonian government, an exciting pioneer in using the internet for public services, has created an environment for taxi and ride-sharing apps that is not only friendly to competitors of Uber (such as homegrown Taxify) but digitally integrates automatic tax collection from the industry.

Tags

Share

About MetaFilter

MetaFilter is a weblog that anyone can contribute a link or a comment to. A typical weblog is one person posting their thoughts on the unique things they find on the web. This website exists to break down the barriers between people, to extend a weblog beyond just one person, and to foster discussion among its members.