SEATTLE — As the controversy surrounding the revelation that the National Security Agency (NSA) has been collecting metadata regarding the telephonic and Internet activities of average Americans continued to swirl out of control — including the name and video confession of the admitted leaker — the following question was posed to me: “How do I communicate with folks without having it casually Hoovered up by the NSA?”

Ignoring for a moment the slander involved in invoking both the names of the former FBI director and the renowned international vacuum company in connection with the most recent intelligence security debacle, the quick answer to the question is: “You can’t.” Absent the use of one-time encryption on closed communication systems, both the Internet and virtually all telephonic conversations are capable of being intercepted.

In fact, all types of electronic data are collected, collated, stored, and analyzed for retrieval by almost every modern industry in existence. It is not just the NSA that is doing the mass collection of data. Everyone is tapping into metadata, toll records, Internet data logs, credit reports, and pubic databases that collect millions of mundane bits of information. Doctors, lawyers, credit agencies, banks and real estate brokers, and every city, state, and federal government agency that regulates our lives collect and store digital information about all of us. The information they collect is there for the asking and can be purchased by anyone.

If this sounds onerous and Orwellian to you, it is simply because of the erroneous belief that collecting metadata is intrusive and a violation of privacy rights. After all, the United States has a long-standing tradition of resisting government intrusion into private lives. But the collection of metadata has been conducted as long as there have been companies willing to do the collecting.

Before there was Google, there were reverse telephone directories, reverse address books, business locator encyclopedias, tax records, and voter registration rolls — along with myriad private services that would research a name, address, or telephone number for you. Reporters of a certain vintage will remember all of these data-collection tools. The key to understanding modern data collection is that (1) everyone is doing it and everyone is in the data, and (2) publicly available information about known or potential bad guys collected by both government and non-governmental companies is how modern-day investigators, including the FBI, solve crime. That is a fact. And if it makes you uncomfortable, perhaps you should become a neo-luddite, and live the rest of your life in a remote Montana cabin writing manifestos in long-hand or on a typewriter. Living as a hermit is about the only way that you will avoid the digital collection onslaught that has become part of modern daily life.

In the current dust-up involving the NSA, people seem to be offended that it is the government using the data. They seem to miss the point that government isn’t actually doing the collecting; the metadata is merely being provided pursuant to legal court order after first being collected by both the telephone companies and Internet service providers. Both industries have way more information about you than the government has ever sought to collect.

There have been many good articles describing the legal rationale for the NSA’s collection of the data, including one by Stewart Baker, former general counsel for the agency. I will not attempt to repeat the legal or civil libertarian counterarguments here that have been made by others far more informed than me. Rather, since Americans can’t avoid the collection of our digital signatures by either the government or private corporations that knowingly sell it to anyone willing to pay a fee, perhaps it’s time to take a closer look at exactly how the government uses the information that it has “Hoovered,” in order to help lower our collective anxiety about Big Brother.

By mentioning Hoover, my questioner reminded me of a little-remembered fact about the late director that may provide some background and insight into the current controversy and how American investigators, primarily the FBI, use information collected by the NSA. Prior to obtaining his law degree and becoming director of the FBI, J. Edgar Hoover was a clerk at the Library of Congress.

There he learned that the collating and cross-referencing of information for easy recovery was important, and easy access by FBI investigators to law enforcement intelligence was what was needed to solve crime on a national scale. He later adapted what he learned at the Library of Congress for use in the FBI’s manual records management systems. As a young FBI agent — before computerized databases or even cell phones — I recall being issued red and blue pencils to mark documents for indexing in the FBI’s non-digital records system. Director Hoover recognized the investigative value of information contained in FBI files and designed a system wherein both field office and bureau file numbers were recorded on index cards along with whether the name being indexed was a “main subject” or merely a reference within the file. This system allowed clerks in any field office to connect bits of information back to the subject’s main file, which might be in a different office. It was a slow and laborious system, but very effective for tracking information across multiple field offices, states, and FBI headquarters.

Library science is also the perfect metaphor for making sense of the way the NSA uses metadata to protect the United States. The explanation for why — if not technically how — the NSA was collecting and using this data is important to understanding the potential for abuse that seems to be at the center of the controversy. In a way, the NSA functions like a giant reference library for the entire intelligence community, collecting books of information and electronic data on foreign individuals and corporations around the world. The books of information on foreigners are available to be checked out by any member of the intelligence community that has the proper clearance and a need to know the information. Members of the intelligence community request specific books on foreigners, and if the book doesn’t exist in the current catalog, the NSA library will attempt to obtain it.

Initially, the NSA was restricted to only collecting on foreign individuals and scrupulously avoided any collection that included Americans or domestic surveillance. All of that changed with the passage of the Patriot Act in 2001. We now know that the NSA library has been obtaining and cataloging encyclopedias of information about U.S. persons — loosely defined as U.S. citizens, permanent resident aliens, and U.S.-based corporations — pursuant to a Foreign Intelligence Surveillance Act (FISA) court order since at least 2008. Like any good reference library, however, the collection of books on American citizens has been segregated into a special restricted access section that could only be accessed with a library card, consisting of a FISA court order. They could not be checked out and no one at the library was allowed to read the books or even acknowledge the existence of this special collection. The names on the books in the collection were kept secret.

Initially, the FBI was only allowed to place requests with the reference librarian to check the catalog for foreign names. To gain access to a book about Americans in the special access collection, the FBI had to rely on the reference librarian at the NSA to determine if a foreign target was providing content to one of the books in the American section and relay that information to them. Then — and only then — could the FBI use that information to obtain a FISA court order and read the book on the American citizen.

Further information from the American books — content or interception of real-time conversations — would require another probable cause statement and affidavit to the Foreign Intelligence Surveillance Court essentially proving that the U.S. individual was being directed by a foreign power. The person could not just be a member, associate, or affiliate, but had to be an actual agent of a foreign power, before any actual interception of content or conversations could take place.

That very restrictive process to access the NSA library on U.S. metadata by the FBI continues today.

How do FBI agents and other investigators use metadata in their investigations? For the investigator, information is fundamental. The ability to query a known or suspected terrorist telephone number and get back all of the numbers he or she has called in the past three to six months or longer is a tremendous investigative advantage. The data assists the investigator in focusing his efforts on the most likely suspects and associates of the actual target for asset or informant development, to locate other possible suspects, or as a cooperating witness against the actual target of the investigation.

Metadata may narrow the suspect pool from over a thousand suspects to maybe a half dozen or more, certainly a more manageable number. It is important, however, to remember that data absent content — that is the actual conversation between two individuals — is of limited probative value. It only provides items of lead value and correlation, not evidence of a crime, although analytical products can sometimes be used as circumstantial evidence at trial. Metadata is primarily used to establish reasonable suspicion for opening a case and occasionally probable cause to request legal process to obtain further information from the NSA.

In my career, I have had several opportunities to work with the NSA and metadata, but pre-9/11, FBI investigations utilizing NSA data only targeted known or suspected foreign terrorist groups or members, never U.S. persons. The standard for investigating a U.S. person was significantly higher, requiring probable cause to obtain a FISA wiretap warrant.

In the 1980s, I investigated the American end of a known foreign terrorist group. The problem then — as is frequently the case now — was that all of the American suspects were either naturalized U.S. citizens or U.S. persons. It made the investigation significantly more difficult because the use of any invasive extraordinary techniques like wiretaps or surreptitious searches required the FBI to meet the highest probable cause standards. And when one of the subjects of the investigation had what a national security lawyer at the Department of Justice referred to as a “Damascus conversion” and claimed to no longer be a member of the group being investigated, that ended the wiretap.

Today, the NSA neither provides information on U.S. persons, nor targets them for data interception absent a court order. If the FBI requests information on a U.S. person, what is often returned by the NSA consists of an affirmative or negative response that the signals intelligence being requested — usually a telephone or credit card number — exists in their database. It is then up to the FBI to develop the probable cause necessary to obtain a court order or warrant to obtain the actual information from the NSA. While creative writing can be used to develop probable cause — and was sometimes suggested by more senior agents — my boss at the time used to say, “You can’t make chicken salad out of chicken shit.” The evidence either existed or it didn’t, and no agent worth his salt would really bet his badge and credentials by stretching the truth to make up probable cause when discovery of the fabrication would surely result dismissal and prosecution.

Now that the leaker has been revealed to be Edward Snowden, a 29-year-old contractor for Booz Allen, the true problem of a digital library of information on Americans comes to light. That is, anyone with access to the NSA library can steal a book about Americans and publish the information if they choose to do so. The irony of the situation involving Snowden is that, if he is convicted, it will be based, at least in part, on his own telephonic communications and Internet data logs compiled by the NSA and provided to the FBI in response to a FISA court order — exactly the way the domestic digital collection system involving the investigation of Americans is supposed to work. Snowden’s revelations have also proven that we have crossed the digital Rubicon; there is no going back to a time when FBI and NSA files were manual and reasonable internal security measures were sufficient to safeguard our individual privacy.

It’s time to learn to either live with it, or legislate the collection of metadata by the government into oblivion and risk the inherent consequences of that decision. The choice is yours.

David Gomez is a former assistant special agent-in-charge and counterterrorism program manager with the FBI. He now runs HLS Global Consultants, a risk-mitigation consulting firm. He wrote this column for Foreign Policy. Follow him on Twitter: @AllThingsHLS.

As you comment, please be respectful of other commenters and other viewpoints. Our goal with article comments is to provide a space for civil, informative and constructive conversations. We reserve the right to remove any comment we deem to be defamatory, rude, insulting to others, hateful, off-topic or reckless to the community. See our full terms of use here.

More in Opinion

THE SHOWBOAT’S FUTURE Wanted: a management partner who will “creatively explore new programming and service opportunities” at a unique St. Paul riverfront facility. It’s the former Minnesota Centennial Showboat at Harriet Island Regional Park, and the city’s Parks and Recreation Department says its call for ideas is the “opportunity to re-imagine what this riverboat could be.” The department is seeking...

TEACHING USEFUL SKILLS Schools, are you listening? Joe Soucheray’s Dec. 1 column was brilliant. (“What’s next? Fancy signs in lunchrooms telling kids to chew?) Taking students with disciplinary problems and giving them a useful way to be productive, even teaching them a skill they could possibly use for future employment, is exactly what our schools should be doing. Snow and...

TRUMP’S NEXT CHAPTER Atticus Finch, the beloved father and lawyer in Harper Lee’s “To Kill a Mockingbird,” is one of literature’s most admirable characters, defending a black man accused of raping a white woman in a bigoted Southern town. But what has come to be considered Lee’s original manuscript, “Go Set a Watchman,” which was published in 2015 after lying...

Though the presidential campaign did not focus much attention on it, the federal debt confronting President-elect Donald Trump is greater, as a percentage of the economy, than at any time since Harry Truman’s World War II term of office. Trump more than Democrat Hillary Clinton did give lip service but few specifics about the size of the growing national debt...

The best way to solve the conundrum of 11 million undocumented immigrants who are living, working and contributing to America right now has a name: “245(i).” Section 245(i) of the Immigration and Nationality Act is a law that enjoyed bipartisan support in Congress in its 13 years of existence. It is still on the books, though it expired in 2001...

St. Paul’s Right of Way (ROW) assessment program has certainly had a rough few months. First, nonprofits succeeded in getting the Minnesota Supreme Court to declare the assessment a tax subject to constitutional restrictions on taxing nonprofits. In a rare moment of ideological Kumbaya, conservative and progressive policy organizations joined hands to argue against it at the Supreme Court, as...