Addressing the Harm of Total Surveillance: A Reply to Professor Neil Richards

The ethos of our age is “the more data, the better.”1 In nearly every sector of our society, information technologies identify, track, analyze, and classify individuals by collecting and aggregating data. Law enforcement, agencies, industry, employers, hospitals, transportation providers, Silicon Valley, and individuals are all engaged in the pervasive collection and analysis of data that ranges from the mundane to the deeply personal.2 Rather than being silos, these data gathering and surveillance systems are linked, shared, and integrated. Whether referred to as coveillance,3 sousveillance,4 bureaucratic surveillance,5 “surveillance-industrial complex,”6 “panvasive searches,”7 or business intelligence, total-information awareness is the objective.8

Response To

Consider Virtual Alabama.9 Google has built a customized database for Alabama’s Department of Homeland Security that combines three-dimensional satellite/aerial imagery of the state with geospatial analytics that reveal relationships, trends, and patterns in incoming data.10 Virtual Alabama can “track moving objects, monitor sensors, and overlay near-real time data sets.”11 Alabama will continue to add inputs,12 but the system already aggregates data from traffic cameras, real-time private and public video streams, GPS location data for police cruisers, building schematics, sex offenders’ addresses, and land-ownership records.13 The state’s 1500 public schools plan to link their video cameras into the system, providing live streaming 24 hours a day, 7 days a week.14 Virtual Alabama is also encouraging contributions from government agencies in exchange for access to the system.15 The stated goal of the program is to map all available data in the state.16

Virtual Alabama is part of a broader surveillance system sponsored by federal, state, and local governments and their private partners. In the wake of the 9/11 attacks, Congress adopted a number of innovations to break down ossified bureaucratic structures that previously impeded intelligence efforts to identify future threats. Among these innovations was the creation of the new Department of Homeland Security.17 Amidst these efforts, the United States rejected proposals to establish an intelligence agency akin to Britain’s MI5, which is devoted to domestic intelligence and surveillance, due to bureaucratic infighting and fear of a civil liberties firestorm.18 But what it eschewed formally, it pursued in fact.

Since 9/11, a surveillance state has been in development,19 accomplished in part by a network of fusion centers through which government agents and private-sector representatives “collect and share” information and intelligence.20 State- and locality-run fusion centers get most of their funding from federal grants.21 Their stated goal is to detect and prevent “all hazards, all crimes, all threats.”22 At the Washington Joint Analytical Center, for instance, analysts from the Department of Homeland Security, the FBI, state police, and Boeing generate and analyze “criminal and anti-terrorism intelligence.” 23

Congressional panels, journalists, and citizens have been told that fusion centers raise few privacy concerns and that their information gathering is focused and valuable.24 Contrary to these assurances, critics have argued that fusion centers erode civil liberties without concomitant gains for security.25 A recent Congressional report backs these concerns, demonstrating that fusion centers have amounted to a waste of resources.26

Fusion centers cast a wide and indiscriminate net. Data-mining tools analyze a broad array of personal data culled from public- and private-sector databases, the Internet, and public and private video cameras. Fusion centers access specially designed data-broker databases containing dossiers on hundreds of millions of individuals, including their Social Security numbers, property records, car rentals, credit reports, postal and shipping records, utility bills, gaming, insurance claims, social network activity, and drug- and food-store records.26 Some gather biometric data and utilize facial-recognition software.27 On-the-ground surveillance is collected, analyzed, and shared as well. For example, the San Diego fusion center purchased tiny cameras for law enforcement to attach to their shirt buttons, hats, and water bottles. Through the federal government’s “Information Sharing Environment,”30 information and intelligence is distributed to public entities, including state, local, and federal agencies, and private owners of “critical infrastructure,” such as transportation, medical, and telecommunications infrastructure.31

The scope of surveillance capacities continues to grow. Fusion centers and projects like Virtual Alabama may already have access to broadband providers’ deep packet inspection (DPI) technologies, which store and examine consumers’ online activities and communications.32 This would provide government and private collaborators with a window into online activities,33 which could then be exploited using data-mining and statistical-analysis tools capable of revealing more about us and our lives than we are willing to share with even intimate family members.34 More unsettling still is the potential combination of surveillance technologies with neuroanalytics to reveal, predict, and manipulate instinctual behavioral patterns of which we are not even aware.35

There can be no doubt that advanced surveillance technologies such as these raise serious privacy concerns. In his article, Professor Neil Richards offers a framework to “explain why and when surveillance is particularly dangerous and when it is not.”36 Richards contends that surveillance of intellectual activities is particularly harmful because it can undermine intellectual experimentation, which the First Amendment places at the heart of political freedom. Richards also raises concerns about governmental surveillance of benign activities because it gives undue power to governmental actors to unfairly classify, abuse, and manipulate those who are being watched; but it is clear that his driving concern is with intellectual privacy. We think that this focus is too narrow.

According to Richards, due to intellectual records’ relationship to First Amendment values, “surveillance of intellectual records — Internet search histories, email, web traffic, or telephone communications — is particularly harmful.”37 Richards argues that governmental surveillance seeking access to intellectual records should therefore be subjected to a high threshold of demonstrated need and suspicion before it is allowed by law.38 He argues also that individuals ought to be able to challenge in court “surveillance of intellectual activities.”39 Richards further proposes that “a reasonable fear of government surveillance that affects the subject’s intellectual activities (reading, thinking, and communicating) should be recognized as a harm sufficient to prove an injury in fact under standing doctrine.”40

Richards is right to call for the protection of “intellectual privacy.”41 Reflecting his concerns, the U.S. Senate’s Permanent Subcommittee on Investigations recently reported internal Department of Homeland Security warnings about agents routinely using fusion centers to collect intelligence on “First Amendment-protected activities lacking a nexus to violence or criminality,” including those of religious and political groups.42 One fusion center instructed law enforcement to collect information on supporters of third-party candidates, including the public movements of cars with bumper stickers supporting Ron Paul and Bob Barr.43 Expressing the impact of this sort of surveillance on intellectual privacy, one political activist explained that he feared being pulled over by a police officer because of political views expressed by his bumper sticker.44 Although much fusion center surveillance remains hidden, Richards’s concerns are valid and pressing; in the present, as in the past, there can be no doubt that surveillance systems interfere with expressive activities.

Although Richards aptly captures the dangers to intellectual freedom posed by technologically enhanced surveillance, we fear his policy prescriptions are both too narrow and too broad because they focus on “intellectual activities” as a necessary trigger and metric for judicial scrutiny of surveillance technologies.45 Our concerns run parallel to arguments we have made elsewhere against the so-called “mosaic theory” of quantitative privacy46 advanced by the D.C. Circuit47 and four Justices of the Supreme Court in United States v. Jones.48 Our argument there supports our objection here: by focusing too much on what information is gathered rather than how it is gathered, efforts to protect reasonable expectations of privacy threatened by new and developing surveillance technologies will disserve the legitimate interests of both information aggregators and their subjects.

One reason we are troubled by Richards’s focus on “intellectual activities” as the primary trigger for regulating surveillance technology is that it dooms us to contests over which kinds of conduct, experiences, and spaces implicate intellectual engagement and which do not.49 Is someone’s participation in a message board devoted to video games sufficiently intellectual to warrant protection? What about a telephone company’s records showing that someone made twenty phone calls in ten minutes’ time to a particular number without anyone picking up? Would we consider the route someone took going to the library an intellectual activity? Is it the form of the activity or what is being accomplished that matters most?

Setting aside obvious practical concerns, the process of determining which things are intellectual necessarily raises the specter of oppression. Courts and legislators would be required to select among competing conceptions of the good life, marking some “intellectual” activities as worthy of protection, while denying that protection to other “non-intellectual” activities. Inevitable contests over the content and scope of “intellectual privacy” will be, by their nature, subject to the whims and emergencies of the hour.50 In the face of terrorist threats, decisionmakers will surely promote a narrow definition of “intellectual privacy,” one that is capable of licensing programs like Virtual Alabama and fusion centers. Historically, decisionmakers have limited civil liberties in times of crisis and reversed course in times of peace,51 but the post-9/11 period shows no sign of the pendulum’s swinging back. Given the nature of political and judicial decisionmaking in our state of perpetually heightened security, protection, even of “intellectual privacy,” is most likely to be denied to the very outsiders, fringe thinkers, and social experimenters whom Richards is most concerned with protecting.52

Richards might argue that his account of “intellectual privacy” and his definition of “intellectual activities” are sufficiently capacious to obviate these concerns. Yet this very capaciousness proves our point. Whether “intellectual privacy” and “intellectual activities” will be read narrowly or broadly, and for that matter, what might constitute a narrow or broad reading, inevitably will be contested just as hotly as the borders of inclusion and exclusion. To draw a loose parallel, the debates among legal positivists and natural law theorists did not abate when Hart expanded the descriptive scope of positivism53 or when Dworkin did the same for naturalism.54 To the contrary, they simply expanded the number of battlefronts so that we now see bloody contests within both camps as well as between them.

The Supreme Court has acknowledged the weight of these sorts of concerns in the context of Fourth Amendment debates. For example, in Kyllo v. United States, the Court was invited to limit Fourth Amendment protection to activities in the home that can be regarded as “intimate.”55 Writing for the Court, Justice Scalia demurred precisely because he thought the Court had neither the qualifications nor the authority to determine what is and is not “intimate.”56 He therefore focused on the invasiveness of the technology itself — a heat detection device — and its potential to render a wide range of activities in the home, whether “intimate” or not, subject to government surveillance.57 By our lights, this is a wise path to follow. Although we find persuasive Richards’s description of the harms inflicted by totalizing surveillance on intellectual privacy, we are not persuaded that the law should use “intellectual activities” as a trigger for judicial scrutiny or as a special category for judicial treatment any more than the Court should use “intimacy” as a signal for Fourth Amendment regulation.

Rather than assigning primary importance to “intellectual activities” and presumably providing less protection against the acknowledged perils of broader types of surveillance, the law’s focus should be on the dangers of totalizing surveillance. Information privacy scholars58 and surveillance studies theorists59 alike have long adhered to this approach, and for good reason. Technologies like Virtual Alabama and the fusion-center network amass, link, analyze, and share mass quantities of information about individuals, much of which is quotidian. What is troubling about these technologies is not what information they gather, but rather the broad, indiscriminate, and continuous nature of the surveillance they facilitate.60 Video cameras may be trained on street corners, drugstore aisles, or a school’s bathroom entrances. The information they gather likely does not implicate intellectual activities. They nonetheless create and sustain the kind of surveillance state that is anathema to liberty and democratic culture.61 Fusion centers rely upon data-broker dossiers, much of which has nothing to do with intellectual endeavors. There is no doubt, however, that continuously streaming all of this information into the information-sharing environment facilitates the sort of broad and indiscriminate surveillance that is characteristic of a surveillance state.

In assessing the privacy interests threatened by such totalizing surveillance, we have in mind some of the lessons taught by Samuel Warren and Louis Brandeis in their foundational article The Right to Privacy.62 Of course, the surveillance technologies of their era could only record discrete slices of life. Nonetheless, Warren and Brandeis recognized that emerging surveillance capacities threatened individuals’ interests in being “let alone” in their “private life, habits, acts, and relations.”63 In Warren and Brandeis’s view, the watchful eye of “any other modern device for recording or reproducing scenes or sounds” interfered with the development of a person’s “inviolate personality.”64 In discussing a husband’s note to his son that he did not dine with his wife — a pedestrian communication by any measure — Warren and Brandeis explained that the privacy interest protected was “not the intellectual act of recording the fact that the husband did not dine with his wife,” but the unwanted observance of the “domestic occurrence” itself.65 Of course, these are precisely the concerns echoed by Justice Scalia on behalf of the Court in Kyllo.66

The threat posed by contemporary surveillance technologies lies in how much and how often people are watched. Modern technologies allow observers to detect, gather, and aggregate mass quantities of data about mundane daily acts and habits as well as “intellectual” ones.66 The continuous and indiscriminate surveillance they accomplish is damaging because it violates reasonable expectations of quantitative privacy, by which we mean privacy interests in large aggregations of information that are independent from particular interests in constituent parts of that whole.68 To be sure, the harms that Richards links to intellectual privacy are very much at stake in recognizing a right to quantitative privacy. But rather than being a function of the kind of information gathered, we think that the true threats to projects of self-development and democratic culture lie in the capacity of new and developing technologies to facilitate a surveillance state.

In adopting this view, we ally ourselves in part with commitments to a quantitative account of Fourth Amendment privacy promoted by at least five Justices of the Supreme Court last Term in United States v. Jones.69 In Jones, police officers investigating drug trafficking in and around the District of Columbia attached a GPS-enabled tracking device on defendant Jones’s car. By monitoring his movements over the course of a month, investigators were able to document both the patterns and the particulars of his travel, which played a critical role in his ultimate conviction. Although the Court resolved Jones on the narrow grounds of physical trespass, five justices wrote or joined concurring opinions showing sympathy for the proposition that citizens hold reasonable expectations of privacy in large quantities of data, even if they lack reasonable expectations of privacy in the constitutive parts of that whole.70 Thus, they would have held that Jones had a reasonable expectation in the aggregate of data documenting his public movements over the course of four weeks, even though he did not have any expectation of privacy in his public movements on any particular afternoon.71

The account of quantitative privacy advanced by the Jones concurrences has much in common with the views promoted by Warren and Brandeis. Specifically, the concurring Justices in Jones expressed worry that by “making available at a relatively low cost such a substantial quantum of intimate information about any person whom the Government, in its unfettered discretion, chooses to track,” programs of broad and indiscriminate surveillance will “chill[] associational and expressive freedoms,” and “alter the relationship between citizen and government in a way that is inimical to a democratic society.”72 Their concerns are well-grounded in original understandings of the Fourth Amendment.73 As Professor William Stuntz has shown, the Fourth Amendment was drafted partly in reaction to eighteenth-century cases involving the British government’s use of general warrants to seize personal diaries and letters in support of seditious-libel prosecutions that were designed to suppress political thought.74 Despite these roots, quantitative privacy is just beginning to receive recognition because it is only now under threat of extinction by technologies like Virtual Alabama and fusion centers.

There are two ways we might seek to protect quantitative privacy in an age of expanding surveillance technology. One strategy would focus on the aggregations of information assembled with respect to a particular person. This “mosaic” approach presents serious practical concerns along the lines we described with regard to intellectual privacy.75 As Professor Orin Kerr asks, where would we draw the line between aggregations that are and are not too invasive?76 How would we treat discrete aggregations assembled by different actors if the sum of those wholes would cross the invasiveness threshold, wherever it is drawn?77 More importantly, we do not see how this approach could actually preserve reasonable expectations of quantitative privacy. The harm is done, after all, by being watched in a totalizing way — or by the awareness that one might be so watched.78 Limiting the scope of information dossiers does little to address those concerns. In light of these challenges, we have argued elsewhere for regulating the technologies themselves.79 Our arguments there strongly suggest that Richards’s goal of protecting intellectual privacy would also be better served by adopting a technology-centered approach.

Of course, none of this argument is intended to discount the benefits of surveillance to national security, criminal justice, emergency response, public administration, or medical care.80 As Richards observes, any account of surveillance’s privacy harms is often resisted on the grounds that some surveillance is essential for the public good. But there is a line between surveillance that is essential for the public good and invasive total-information awareness technologies, and that line is easy to cross if unattended. This leaves us with the question of how to protect society from the gradual acceptance and institutionalization of total-information awareness technologies. Richards supports allowing individuals to challenge surveillance of intellectual activities in court as a cognizable harm. Here again, we worry that his proposal is unlikely to preserve the fundamental interests at stake.

Richards proposes to grant individuals standing to challenge governmental surveillance.81 Putting concerns about the constitutionality of such a challenge aside, his proposal may raise practical problems. Granting individuals standing to challenge governmental surveillance of them would overwhelm the courts. There are not enough judicial resources to adjudicate three hundred million such suits, each of which could be renewed — almost as soon as it is resolved — on nothing more than suspicion of continued surveillance because the focus, under Richards’s approach, is on what information is being gathered. The possibility of a class action would not help matters because individual issues of harm attached to what particular information is gathered would predominate.82 Suits are also bound to be met with claims of national security interest, to which courts routinely show considerable deference.83 For example, in litigation involving police surveillance of protestors at the 2004 Republican National Convention, the Second Circuit refused to allow discovery of officers’ field reports, even in redacted form, because they would reveal information about undercover operations and thus potentially hinder future ones.84

What is more, lawsuits designed to uncover surveillance of intellectual activities may be unable to identify the “intellectual records” gathered by government due to the way certain surveillance systems operate. Fusion centers, for instance, may access and analyze private and public databases and real-time video feeds without ever creating and storing records. Although fusion center surveillance of all individuals’ on and offline activities is continuous and totalizing, it does not necessarily produce records that could be packaged and produced as part of a discovery process. Ultimately, the vastness of contemporary governmental total-information awareness renders the judiciary incapable of reviewing the majority of situations on an individual basis. Furthermore, any individual cases that made it to judgment could no more chip away at discrete instances of governmental surveillance. Because they would focus on the intellectual privacy interests of specific litigants, these cases would not and could not challenge the system of totalizing surveillance as a whole.

Here again, we think that a technology-centered approach that seeks to protect quantitative privacy is far more promising. Not only would it avoid the constitutional and practical challenges of individual litigation based on the trigger and metric of intellectual privacy, a focus on the technology would also open the door to a wide range of alternative regulatory frameworks that could more efficiently and reliably strike a reasonable compromise between the legitimate interests of government and the privacy interests of citizens.85 For example, an independent board of experts, such as the Privacy and Civil Liberties Oversight Board (PCLOB), could perform an analysis of the privacy and civil liberties risks posed by surveillance technologies.86 PCLOB, now fully staffed,87 could mandate safeguards for the use of surveillance technologies that raise the specter of a surveillance state and make recommendations based on their privileged access to security analyses, piercing the veil secrecy that Richards laments.88 Board members, vetted for top-secret national security clearances, could attain a comprehensive view of domestic surveillance technologies that would enable them to recommend procedural protections for quantitative privacy to prevent governmental abuse.89 Such procedural protections would by nature protect the intellectual privacy interests at the heart of Richards’s proposal without the drawbacks of using intellectual privacy as a trigger and metric of action.

Although we live in a world of total surveillance, we need not accept its dangers — at least not without a fight. As Richards rightly warns, unconstrained surveillance can be profoundly harmful to intellectual privacy. It would be wrong, however, to conflate symptom and cure. What is most concerning, for us is the rapid adoption of technologies that increasingly facilitate persistent, continuous, and indiscriminate monitoring of our daily lives. Although harms to intellectual privacy are certainly central to our understanding of the interests at stake, it is this specter of a surveillance state that we think ought to be the center of judicial, legislative, and administrative solutions, not the particular intellectual privacy interests of individuals.

5. John Gilliom, Overseers of the Poor: Surveillance, Resistance, and the Limits of Privacy 18, 119 (2001) (exploring surveillance of the poor to administer public benefits).

6. Jay Stanley, Am. Civil Liberties Union, The Surveillance-Industrial Complex: How the American Government is Conscripting Business and Individuals in the Construction of a Surveillance Society (2004), available athttp://www.aclu.org/FilesPDFs/surveillance_report.pdf (documenting industry’s partnership with government to engage in monitoring of citizens).

26. SeeU.S. S. Permanent Subcomm. on Investigations, Majority and Minority Staff Report, Federal Support for and Involvement in State and Local Fusion centers (2012), available at Majority and Minority Staff Report].

43. See Citron & Pasquale, supra note 17, at 1458; see also id. at 1458-63 (discussing the chilling of expressive activities and risk of erroneous classification of individuals raised by fusion centers’ surveillance of religious, political, and racial groups).

52. These concerns — political grudges as well as crisis overreach — animated the Church Commission’s support of FISA after the intelligence surveillance abuses of the COINTELPRO era. Brief of Former Church Committee Members and Staff as Amici Curiae Supporting Respondents and Affirmance at 13, 18, Clapper v. Amnesty Int’l USA, No. 11-1025, 2012 WL 4480741, at *13, *18 (U.S. Feb. 26, 2013).

89. Id. at 1473. For instance, they could require immutable audit logs that promote governmental interest in national security with a commitment to “watch the watchers” by recording all of the uses of that technology.