Transparency Tip: Scouring Public Records with Scout

If you do any kind of shopping online, then you’re probably already familiar with the concept of a metasearch engine—a search engine that searches other search engines. When you’re looking for airline tickets or a used copy of The King in Yellow, you might use a metasearch engine to find out which online retailer is offering the best price.

There is also a metasearch for public records: Scout. Created and lovingly maintained by the Sunlight Foundation, Scout will take a search query and run it through six different continually updated databases of documents. The site also allows you to create email and RSS alerts and track search terms and legislation through a user account.

EFF currently maintains a public page that tracks NSA reform, including congressional bills and speeches, official notices, and court opinions. You can visit it to see current reform efforts and subscribe to our alerts.

The short version: Just type in a word and hit search, or check “Advanced” and enter a phrase with quotes. If you want regular updates, click “Create Alert” on any given page. Go to “Manage Alerts” to set Scout to email you immediately or daily, or to send you a text message. That’s also where you can grab the RSS feed.

Simple as that.

Scout’s strength is its speed, convenience, and the way it helps researchers discover relevant documents that they may not have known existed.

For EFF, Scout is handy for discovering and tracking bills that would impact our rights. “FISA” (Foreign Intelligence Surveillance Act) is a particularly fruitful search term for following NSA reform, while “DMCA” (Digital Millennium Copyright Act) will return results relevant to changes to copyright law. It’s helpful to compare these results to what legislators are saying about the bills, which you can find if you also look at the results for “Speeches in Congress.” Scout allows you to sift through those speeches by state, party, chamber, and individual lawmaker. (Here’s a solid anti-NSA screed from Sen. Patrick Leahy.)

Scout will also return results from state legislation, which would be difficult to track if you had to go to each state’s individual websites. Since state legislators have the authority to regulate commerce and set law enforcement policies, they will often take up electronic privacy issues, such as drones, automatic license plate readers, and consumer data mining. You can also get a good sense of how the states feel about federal issues—for example, Alaska, Virginia, California, Pennsylvania, Michigan, and Ohio all have legislation or resolutions opposing mass surveillance by the NSA.

However, because data standards vary across the country, state legislation isn’t quite as easy for Scout to process as federal legislation. If you identify a bill worth deeper inspection, you should take the bill number from Scout and plug it directly into the individual state legislature’s web site.

A substantial amount of US policy is determined through the federal regulatory process, which is why you’ll also want to review the results Scout finds in the federal register. These can range from notices from bodies such as the Privacy and Civil Liberties Oversight Board to a listing of all the gifts (e.g., luxury wrist watches, rugs, vases, ceremonial blades, firearms) the CIA chief accepted from anonymous foreign governments because “non-acceptance would cause embarrassment to donor and U.S. Government.”

And, of course, the oversight reports—obtained from the Government Accountability Office and various inspectors general—are often filled with important nuggets about questionable government programs. These reports are also good for grabbing statistics and background details on particular issues. Scout only displays a summary of the reports, so you’ll need to follow the PDF link to read each in full.

One last tip: As all-powerful as Scout seems, it’s not infallible and that’s one reason why it always links back to the original material. If in doubt, or if you’re conducting a fact check, always refer back to the source.

Other resources:

USA.gov – A not-particularly-reliable search engine that is suppose to return results from across government networks. If there is a specific document you’re after, you’re not likely to find it here. It’s like dropping a net into an unorganized ocean of information; you could very well find something meaty you didn’t know you were looking for or you could find nothing but old boots and plankton. On the other hand, if you’re looking for public domain images of anything related to government, USA.gov’s image search is a great resource.

Google Advanced Search – Here’s a little hack for Google that will sometimes return different results than USA.gov. Put in your search term as you normally would, but add in “site:.gov” right after it (also try site:.mil for military sites).

With both USA.gov and Google, you can add in “filetype:pdf”, “filetype:ppt” or “filetype:xls” to your search to narrow results to only include PDF, PowerPoint or Excel files.

Related Updates

For the fourth year, EFF is naming and shaming government officials and agencies around the country who stand in the way of transparency. We honor these information gatekeepers with The Foilies, our tongue-in-cheek “awards” during Sunshine Week, which runs from March 11-17, 2018. Think of it like “The Golden Raspberries,”...

In a promising step toward transparency, the Eastern District of Texas (the court that sees many of the nation’s patent cases) recently announced an amendment to its Local Rules that would require parties to file redacted versions of documents that contain confidential information. Previously, parties would file whole...

EFF Senior Staff Attorney Jennifer Lynch and Investigative Researcher Dave Maass last night received the First Amendment Coalition’s 2017 Free Speech & Open Government Award in recognition for their work bringing transparency and accountability to law enforcement’s collection and use of automated license plate...

Consider this: Deputy Attorney General Rod Rosenstein has been going around talking about “responsible encryption” for some time now—proselytizing for encryption that’s somehow only accessible by the government—something we all know to be unworkable. If the Department of Justice (DOJ) is taking this aggressive public position about what...

EFF is representing The Stranger, a Pulitzer Prize-winning newspaper, in a petition to unseal secret government electronic surveillance dockets and requests in the Western District of Washington federal court. The petition was filed in November 2017. The government routinely asks courts around the country for electronic surveillance warrants and...

As we detailed yesterday, a bill introduced this week by Sens. Ron Wyden and Rand Paul would represent the most comprehensive reform so far of Section 702, the law that authorizes the government to engage in mass warrantless surveillance of the Internet. EFF supports the bill, known as the...

Since first appearing on the streets of New York City in 2016, LinkNYC’s free public Wi-Fi kiosks have prompted controversy. The initial version of the kiosks’ privacy policy was particularly invasive: it allowed for LinkNYC to store personal browser history, time spent on a particular website, and lacked clarity...

EFF has petitioned a federal appeals court to reconsider its flawed ruling in our national security letter case on behalf of CREDO Mobile and Cloudflare. In July, the Ninth Circuit in San Francisco upheld the NSL statute against our First Amendment challenge. The court ruled that the...

The government has released eleven secret court orders and opinions as a result of an EFF lawsuit seeking to make significant decisions by the Foreign Intelligence Surveillance Court (FISC) public. The documents, which you can read here, primarily deal with requests by U.S. intelligence agencies to access business...

As the days wind down for the California legislature to pass bills, transparency advocates have seen landmark measures fall by the wayside. Without explanation, an Assembly committee shelved legislation that would have shined light on police use of surveillance technologies, including a requirement that police departments seek approval from...