Monday, 21 June 2010

What happens when you recruit dozens of students to perform a maze-based computer task and then you ratchet up the competitive pressure? Does their performance improve or do they just cheat more?

Christiane Schwieren and Doris Weichselbaumer found out by having 33 men and 32 women at the Universitat Pompeu Fabra in Barcelona spend 30 minutes completing on-screen mazes. Crucially, half the students were paid according to how many mazes they completed whereas the half in the 'highly competitive' condition were only paid per maze if they were the top performer in their group of six students.

The students in the highly competitive condition narrowed their eyes, rolled up their sleeves, focused their minds and cheated. That's right, the students playing under the more competitive prize rules didn't complete any more mazes than students in the control group, they just cheated more.

To be more specific, the female students in the highly competitive condition cheated more. That is, although across both conditions there was no overall difference between men and women in the amount they cheated, only women responded to the competition intensity by cheating more. Schwieren and Weichselbaumer dug deeper into their results and actually this wasn't a gender issue. Competition increased cheating specifically among poorer performers and it just happened that the poorer performers tended to be female.

How did the researchers measure cheating? After a brief practice, the students were told to continue completing mazes on level 2 difficulty, but they could choose to break the rules by switching to an easier level. The game also gave the option of clicking a button to be guided through the maze solutions. Finally, the students could lie at the end on a score sheet about how many mazes they'd completed. Earlier the researchers had loaded a spy programme on the computers. This took a screen shot on each mouse click, thus revealing the students' true actions.

'It turns out that individuals who are less able to fulfill the assigned task do not only have a higher probability to cheat, they also cheat in more different ways,' the researchers said. 'It appears that poor performers either feel entitled to cheat in a system that does not give them any legitimate opportunities to succeed, or they engage in "face saving" activity to avoid embarrassment for their poor performance."
_________________________________

17 comments:

Bob and Laura - I'm a little confused. The fact that it was only female participants who responded to greater competition by cheating more, I framed as 'not a gender issue' - isn't that the opposite of sexist?

There was no sexism in the above post, I suggest you leave your hangups at the door when you choose to read a psychological study because it won't always conform to your world view and if you're of an easily offended persuasion you'll find this a field fraught with perceived insults. The author even clarified the result in detail so that if, in future, anyone should wish to claim women are cheats and cite this as proof then one can merely direct the arguer to this page.

I suggest the both of you grow up and realise that sexism isn't just anything potentially negative said about men or women.

I have seen a similar behavior in myself. I regularly play football as a defender. The days I am playing against stronger teams, I just smack the ball out from near the post. When I am playing against regular teams, I tend to clear more "sanely". :-)

I think this little spat highlights the need for precision in our reading, interpretation and reporting of data, and common sense when responding to blogs.It seems to me that both the writer of this blog and the researchers were trying to present a balanced and unsexist interpretation of the results of this experiment. The data apparently showed that more women cheated, but they didn't cheat because they were women but because they were worse at performing the task. There is no suggestion that they are worse at performing the task because they are women, just that in this sample this was the case.However, because there are pretty equal numbers of women and men in the sample would it not be legitimate to ask questions relating to ability at this task and gender?If a larger sample were taken and it was found that there was no difference overall in the performance of women and men, but there was still a tendency for men or women to cheat more, would it not be legitimate to ask why? Therefor the fact that more women cheated than men in this group was of interest but rightly discounted on the basis of further interrogation of the data. This is the way science works.I am vehemently opposed to any form of prejudice but in my view this does not preclude the exploration of differences.

The sample sizes are on the small side for the generalizations, especially if you desegregate by gender and ability level. We have seen a lot of cheating reports regarding high stakes tests as a result of NCLB. I think this study longs for replication with larger numbers. Go for it. DrDougGreen.Com.

"To be more specific, the female students in the highly competitive condition cheated more. That is, although across both conditions there was no overall difference between men and women in the amount they cheated, only women responded to the competition intensity by cheating more. Schwieren and Weichselbaumer dug deeper into their results and actually this wasn't a gender issue. Competition increased cheating specifically among poorer performers and it just happened that the poorer performers tended to be female."

This is a strange way to present the evidence. For comparison, consider these statements: "To be more specific, the short students jumped higher. . . . Digging deeper into results showed that this was not actually a height issue. Lighter weight students jumped higher, and it just happened that lighter weight students tended to be shorter."

Why not just say, "To be more specific, lighter weight students jumped higher"? That is the finding, after all. Why start with something that is not the finding, and then offer a correction later?

Truth to say, I wondered why women might cheat more -- and why not? That's what was said. Why mention gender differences at all? Just report the findings.

Let me expand on my comment about the headline. If you are faced with a skilled task and a time limit of 30 minutes, how do you improve your performance? The headline suggests that that was a realistic option. It wasn't, was it? {sigh}

Also, consider the first paragraph:

"What happens when you recruit dozens of students to perform a maze-based computer task and then you ratchet up the competitive pressure? Does their performance improve or do they just cheat more?"

Catchy, but that was not what was studied, was it? There were two experimental conditions, but no "ratcheting up" of competitive pressure. {sigh, again}

Billyum, your little analogy re height/weight and jumping doesn't work for several reasons. For starters gender is a categorical variable whilst height is continuous. Also height and weight would be expected to correlate whereas it wasn't at all obvious that gender and performance would correlate in this task. Apart from the fact that your analogy doesn't work, you're wrong for another reason. The researchers specifically looked for a gender effect and they found one, so my report reflects their analysis. They found that only the women responded to increased competition by cheating more - FACT. It would have been remiss of me not to have reflected that in my brief report. However, the researchers explored their data and there was an interesting reason why, at a group level, only the women responded to pressure by cheating. It was because they were poorer performers at this task. This process of discovery through examination of data is part of the joy of science and hopefully in a small way I managed to capture a sense of that process in my precis. You're also wrong about the first paragraph - the whole point of the two conditions was that one instilled an atmosphere of more intense competition. Both conditions were intended to be competitive - after all, the control condition required that the students compete against the clock to earn money, but the competitive atmosphere was ratcheted up in the other condition, by making changes to the reward system, to see what effect this would have.

Digest: "The researchers specifically looked for a gender effect and they found one, so my report reflects their analysis. They found that only the women responded to increased competition by cheating more - FACT."

I was not talking about fact, but about language. Your presentation of their results was on its face misleading. True, you corrected the impression later, but a certain amount of damage had already been done.

Digest: "It would have been remiss of me not to have reflected that in my brief report."

That's your judgement call. :) But you should have said what their ultimate findings were first and talked about the gender question later.

Digest: "However, the researchers explored their data and there was an interesting reason why, at a group level, only the women responded to pressure by cheating. It was because they were poorer performers at this task. This process of discovery through examination of data is part of the joy of science and hopefully in a small way I managed to capture a sense of that process in my precis."

My Fisherian stat professor would probably have objected. As a Bayesian, I am not particularly bothered. :)

Digest: "You're also wrong about the first paragraph - the whole point of the two conditions was that one instilled an atmosphere of more intense competition. Both conditions were intended to be competitive - after all, the control condition required that the students compete against the clock to earn money, but the competitive atmosphere was ratcheted up in the other condition, by making changes to the reward system, to see what effect this would have."

The first paragraph indicates a temporal change in conditions, not two differing conditions, which the experimental design seemed to be. Here is the first paragraph, emphasis mine:

"What happens when you recruit dozens of students to perform a maze-based computer task ***and then*** you ratchet up the competitive pressure? "

It is plausible that a change that "ratchets up the competitive pressure" would induce cheating, but that's not exactly what was studied, was it?

Billyum: you said 'Your presentation of their results was on its face misleading'. I'm sorry, I can't let you get away with such false assertions. There isn't the remotest possibility that a reader could come away misled. 'this wasn't a gender issue' I wrote. You said your concern was with language, not fact. The statement 'this wasn't a gender issue' could not be in plainer language. I went further. I also made it clear that overall there was no gender difference in cheating. Let me reiterate, the fact that only female participants responded to greater competition by cheating more wasn't some obscure quirk of the data that I chose to highlight - it was an issue explicitly discussed and analysed by the researchers. What's more they dealt with the gender finding FIRST. It was only through deeper probing that they discovered the performance-related confound. Billyum, I sense that you desperately want this to be a case of bad science reporting, when it's not. The piece isn't titled 'women cheat under pressure'. The introduction doesn't claim boldly 'women but not men cheat when the going gets tough'. The detail about the gender effect comes later and is fairly and accurately discussed.

While I agree this wasn't a sexist post and I expect certainly from your comments that you did not intend any sexism. However, your posts also indicate a not complete understanding of how the typical person reads. People tend to skim. The simple fact that you started the paragraph stating that women cheated more than men will lead many readers to take that characterization away. Yes, you did state clearly two-thirds of the way down the paragraph that it was not a gender issue. But this is what's called burying the lead. Your typical reader will not take that message away. They will take the message you began the paragraph with. In that sense, the post can be construed as misleading. You did report it fairly and accurately, but without a clear understanding of how most people read.

jdmimic: thanks for your speculation about how people read. You also said: 'this is what's called burying the lead'. Here you are mistaken. The gender finding and its explanation are not the lead for this story. The lead for this story is that people respond to greater competition by cheating. That's flagged up in the post title and in the third paragraph - i.e. prior to discussion of the gender finding and its explanation! You claim that the message that readers will take away from this post is that women respond to competition by cheating whilst men do not. For that to be true, far from skimming, you'd have to zoom straight for the seventh sentence and quite frankly not read any of the rest of the post.

Wow Christian hold on a minute,what do you have to do with women and their inclination with cheating due to according to their low capabilities to succeed? And that is according to you,because male competitors have the same possibilities of being cheatears,some even more than women.And I don't think you should consider cheating if you don't even have a solid base to express your point of view.Not only the poor performers are the cheaters,maybe some just needed a time to fresh their minds.Social interactions are the food of the geniouses.Well,i wonder,which is your case.