White House Privacy Survey Falls Short of Its Purpose

Research experts say questions are overly broad

A White House survey that asks consumers for their opinion about big data and privacy may yield a few political talking points but not much meaningful or useful data, experts say.

The survey popped up Friday on the White House's Web page complete with a video message from John Podesta, counselor to the president. In mid-January, when President Obama spoke about the changes that needed to be made for national security, he also gave Podesta the daunting task of conducting a 90-day review of big data and privacy beyond national security to include commercial uses as well.

"We know it's a complicated issue," Podesta says in the video. "If you're watching this video, I know you're interacting with technology all the time … We'd like to hear from you. Which technologies or use of data is most transforming your day-to-day life? Which technologies or use of data give you pause?"

The survey asks three broad questions: "How much do you trust [a list of] institutions with your data? How much do these types of data [i.e., audio, location] collection concern you? How much do the following data practices [i.e., collection, storage] concern you?"

Experts say the questions, focused on feelings of "concern" and "trust," lack context to elicit any real understanding.

"The questions are extremely broad," said Professor Joseph Turow, the associate dean for graduate studies of the Annenberg School for Communication at University of Pennsylvania. "I don't know what I would do with the answers. It's hard for me to believe this is a serious survey."

No effort is made to talk about context or how the data is used, which could make a big difference in how consumers might answer. "Purpose matters," said Howard Fienberg, director of government affairs for the Marketing Research Association. "If someone is collecting your data for research, which aggregates the information to learn about groups, that is very different from collecting your data for purposes of spying on you, the individual, for national security or law enforcement."

Fienberg and Turow, both experts in survey research, argue that the results would be unrepresentative. There is no screening process to help identify respondents. "The White House appears not to care about whether the input they are gathering comes from Americans or American residents versus anyone in the rest of the world," Fienberg said.

"We don't know who [the respondents] are," Turow said. "It would be incorrect to use these as any serious indicators of the population."