quinta-feira, 12 de março de 2015

Science factory

What used to be the job of university students, being surveyed for
scientific and academic research, is now often being done by people
online, which could arguably lead to flawed and unreliable data.

Mechanical Turk is an online job forum – an informal work force of
people who participate in studies – basically a pool of professional
survey-takers. Researchers are now taking data from such sources because
it’s inexpensive and quick. But there could be discrepancies on the
reliability of answers given. What kinds of surveys are we talking
about?

“You have a lot of psychology, social science research, but also
political scientists are using it. You even see it in medical research,”
said Jenny Marder, who reported the story for PBS Newshour. “And we
looked at a lot of the studies that are using Mechanical Turk, and a lot
of them are asking really big questions. There’s a lot of research on
human behavior, but also on teen alcohol abuse, a lot of research on
decision-making, how people perceive scientists and climate scientists.
So these aren’t obscure studies. And they’re asking some pretty big
questions.”

This issue with surveys like those from Mechanical Turk is that many
people do them from home, even thousands of them, and there’s no way to
know if the person was distracted, actually paying attention, or if they
are on automatic because they have seen some of the same questions
repeated over and over. The people who take these surveys get paid
(albeit, very little) even if they answer questions in an automatic, not
entirely thoughtful way. So what is the incentive to really sit down,
focus, and answer truthfully?

Data isn’t valuable if it isn’t an accurate reflection of the subject’s
perspective and the topic, but it continues to be used.