About Me

My interests include veganism and vegetarianism, health, ethics, politics and culture, media, and the environment. I have three kids; I teach college part-time, study piano and attempt to garden. I knit. I blog on just about anything, but many posts are related to my somewhat pathetic quest to eat better, be more mindful of the environment, and be a more responsible news consumer. Sometimes I write about parenting, but, like so many Mommy bloggers, my kids have recently told me not to. :) Thanks for reading.

Friday, May 13, 2011

NOTE: The title of this post was updated May 28, 2011 to reflect the content more accurately. I criticize the media and poor scholarship.

I had not intended to post anything else here about Lara Logan or her case.I have been writing an article (not yet published) on mob violence, in which Logan's case will be mentioned and, in doing research for that, I decided I needed to see the well-publicized 2008 study on sexual harassment in Egypt, done by the Egyptian Center for Women's Rights (ECWR).The media (CNN, CBS, Salon.com, Washington Post, LA Times, and others) all had articles or news segments where they highlighted the punchline of that study: 98% of foreign women visitors and 83% of Egyptian women had experienced harassment in Egypt; 62% of Egyptian men admitted to harassing women and that 53% of Egyptian men blamed women for “bringing it on”.

The media got that much right. That was, indeed, the major finding worth highlighting.

The media’s rather direct connection to the Logan case was that Logan was assaulted, in part, because Egypt clearly has such a problem with sexual harassment.

Uh. That is kinda right.

Maybe it’s because I teach methods. I needed to see the study itself, because something about those astounding statistics didn’t sit right with me; you see, we hardly ever see 98% in any study of human behavior. Even 83% is high. Those are the kind of stats that make one want to sit up and make sure the study was done correctly or, at the very least, to figure out how those statistics could be arrived at even in a well-designed study.

It took me a couple weeks to get my hands on the study (ironically, once I found it, I wondered why it had taken me so long…). The title, Clouds in Egypt’s Sky: Sexual Harassment, from Verbal Harassment to Rape, suggests that indeed, the authors are going to document rape in their study.

The media didn’t take much time to see whether the study was well done. They saw what most people would have seen: big sample size (1010 Egyptian women, 1010 Egyptian men, 109 foreign women), multiple cites (three), and big impressive, shocking finding (see my first paragraph).

It turns out, even with these seeming strengths of the study, the study itself is seriously flawed. And that knowledge should give us all a big dose of caution in how to relate it to something as heinous as Logan’s experience.

THE PROBLEMS WITH THE STUDY

1) The definition of sexual harassment is very broad.

The authors defined it as “unwanted sexual conduct deliberately perpetrated by the harasser, resulting in sexual, physical, or psychological abuse of the victim regardless of location, whether in the workplace, the street, public transportation, educational institution, or even in private places such as home…it may include behavior such as ogling, gestures, offers to perform sexual acts, questions of a private and sexual nature, displaying sexual photos or pictures, unwanted touching, etc.” (pp. 2-3).

Whenever a concept is defined broadly you inevitably increase the number of people who will fit the criteria for that concept.

While I do agree that there are many forms of sexual harassment – and thus am not offended by a broad definition per se – it is very problematic to add together so many disparate experiences without also reporting their incidence individually.

To lump together “explicit comments” with “assault” – which is exactly what these authors do – is akin to counting candy bar theft alongside tax evasion.

2)The authors do not report the different types of sexual harassment individually.

There is no way, from reading the report, to know what percentage of Egyptian women or foreign women in their sample experienced each type of harassment. Or what percentage experienced multiple types. Or which type was the most common. These are rather important details that most researchers would have known to include.

3) Though the subtitle of the report mentions rape, the authors do not report actually documenting a case of rape.

They write that they uncovered seven forms of harassment: touching, noises (whistling, hissing noises, kissing sounds), ogling of women’s bodies, verbal harassment of a sexually explicit nature, stalking or following, phone harassment, and indecent exposure. Why did the authors indicate that their study included rape if, in their report, it appears that it does not?

Note that in both the definition of sexual harassment AND in the description of the seven kinds of harassment they claim to have uncovered, a case like Logan’s does not “fit “.

4) The authors rely on a convenience sample, not a random one.

I am sympathetic to the reality that random sampling is expensive and time-consuming. But any methodology student knows (or should know) – and certainly anybody with a Ph.D. should know – that if your sample was not randomly chosen, you cannot assume your data are anything more than suggestive. Though many people misunderstand this, “random” is not equivalent to “talking to random people on the street”. There is actually nothing random about random sampling – the most simple explanation of one type of random sampling (there is more than one) is that a random sample is selected in such a way that all samples of the same size have an equal chance of being chosen from the population. Everyone in the population theoretically has an equal chance of being in the sample.

When you have a random sample THEN you can say (within a calculated margin of error) that you have uncovered a pattern in the larger society.

The authors have a nice sample size (2129), but “big” does not mean “representative”.

These authors did not have a random sample so they cannot say (and the media analyzing their study cannot say) that “98% of foreign women in Egypt experience sexual harassment”. The authors did not randomly sample from all of Egypt. In fact, they did not randomly sample from the three sites they did use.

ALL they can say is that in their sample, they found that 98% of foreign women had experienced at least one type of harassment. Instead, they write up that finding without the absolutely necessary caveat of “in our sample”. And then it got reported that way…I’m assuming some journalists need a crash course in how to read academic studies.

I’m volunteering.

5)The three cites the authors sampled from are all in the same part of Egypt.

This should be fairly obvious: you cannot claim to have uncovered patterns of sexual harassment for a country as huge as Egypt with only three sites, particularly when the sites are all basically within miles of Cairo.

Their three sites were Cairo, Giza, and Qalyubeya. I googled several maps of Egypt; Cairo and Giza are right next to each other and Qalyubeya is 28 kilometers North of Cairo. Egypt is a big country.

I’d rename the study “Incidence of Various Types of Sexual Harassment Reported In a Convenience Sample of Egyptian and Foreign Women Living In and Near Cairo.”But that’s not nearly as appealing as the title they used.I know. We academics are such kill joys.

THE MEDIA’S RESPONSIBILITY

I get that the media doesn’t have time for lengthy analyses of studies (although, honestly, isn’t that *part* of their job?). Even if the media didn’t want to unpack all the problems in this study (there are more than the ones I’ve detailed here), they should at least have unpacked two:

1)The Definitional Problem: you define something very broadly, you predictably find many who fit the definition. When this happens, you have to question whether the definition yields meaningful data.

2)The Reporting Problem: the authors presumably collected data on individual types of sexual harassment. But they did not report it. Without that data (it is nowhere in the report), we do not know which types of harassment are most common or whether the most severe forms are even often seen.

If the media had examined just these two flaws in the study, I think somebody would have suggested the obvious: Would not a study done in the US with such a broad definition of sexual harassment also result in similar statistics of between 80 and 98% of American women reporting to have experienced at least one kind?

To just use my life as an example, I've experienced (all in the US):

men exposing themselves to me

unwanted sexually explicit comments

sexually explicit photos shown to me

gestures

sexual propositions

unwanted touching

While all of these instances were scary and threatening, they pale in comparison to what Logan experienced. Yet they appear to be exactly the kinds of harassment uncovered in the ECWR study, which was then used by so many news outlets as a way to partially explain why Logan's assault happened.

The analogy needs a very strong emphasis on the word "partial," for Logan's experience bears, frankly, only faint resemblance to the "everyday" kinds of street harassment uncovered in this study.

I’m not questioning that Egypt has a serious, endemic problem with sexual harassment. Other data easily convince me of this. Read Leiby’s Washington Post article on sexual harassment in Egypt, or Amnesty International's article on the “virginity tests” forced on female Tahrir protesters or look up harassmap.org to see how women in Egypt have documented where and at what times of the day they have been assaulted. Finally, read up on the UC Davis professor who was beaten up by a mob in Tahrir Square the week before Logan. Of course, if you haven't heard of Logan's case, I'm going to assume you've spent the last three months under a rock somewhere.
Egypt has a big problem with sexual harassment and sexual violence, and I've only given you a smattering of the data sources out there that confirm this.

But the ECWR study was overused by many media outlets because, frankly, its main finding was easily digestible for both media consumers and media creators. Who could stay away from a statistic showing that between 80-100% of women in Egypt, domestic and foreign, experience sexual harassment?

But would the media have been so complacent about a study done in the US with the same finding? Probably not, or at least I hope not. I think media would have immediately suspected that something about how the study was done may have affected the finding. They should have seen the red flags with this study, too.

These statistics were used in a careless way, without attention to the fact that the study itself was methodologically weak, mostly due to a problem in operationalizing the term "sexual harassment" and in failing to report percentages of women experiencing each kind of harassment. To rely so heavily on one poorly designed study in order to try to understand Logan’s case is methodologically suspect, but was still convincing to a vast majority of media consumers.

I’m giving the media a “C” on this one. We consumers deserve a little better.

4 comments:

You tell 'em. More seriously, I appreciate your effort to critique a popular argument as well as the data story it rests upon. Well done. I'm going to tuck your post away for use as a real-world example next time I teach World Politics and/or Globalization.

Excellent post, Elaine. I had wondered about that study when I first heard it quoted, and not just because I traveled in Egypt and experienced no problems whatsoever. I always wonder about methodology in studies, and try to teach my students to question any methodolgy that doesn't look right. Unfortunately, I teach PR and advertising students, not news reporters. I believe a basic research course, or at least an in-depth unit on interpreting research should be required of all undergraduate news reporting students. It is required in graduate school, but few working journalists get advanced degrees.

In the study's defense (I've been noodling all night about this): ECWR is, likely, an *activist* organization and this was, as they said, an *exploratory* study. As such, the statistic is a) useful to them, and b) good fodder for a better study, now that they have somewhat documented the "lay of the land". My criticism is both of how the study itself is written (particularly of the authors' neglect of reporting the rates for each *type* of harassment), AND of how the media reported it. To be fair, blame lies in both places, but it wasn't hard, really, to see the study's weaknesses. Do you think (cough) that some journalists just heard the punchline, ran with it, and never read the actual primary source?

Yes, I do think some, if not all media reporting the story read the headline and ran with it without reading the whole thing or even the abstract. This is a problem with the media today--the "24 Hour News Cycle" is now more like 24 seconds. We used to break news on the radio, give details on the evening news (film at 11) and follow up with an in-depth piece in the next morning's paper. Not anymore. Rather than check facts, everybody wants to be the one breaking the news, and that's often done online. As a Public Relations practitioner, this is a problem, because bad information about your client can get out there and go viral before you even know it's been published. The last year alone is full of cases where stories went viral, were reported in mainstream media, and later were debunked when responsible reporters took the time to investigate.

In a case like this, a questionable study gets lots of play online before anybody has actually read it or questioned its methodology. And we all know that research performed by groups with an agenda or preconceived notion can be biased.