Tuesday, June 24, 2008

Australians are...leading the way as the heavyweight champions of the world - with arguably a greater proportion of obese citizens than even the notoriously supersized Americans

60% of the Australian population is overweight, apparently. That's nine million people, according to the study, 'Australia's Future Fat Bomb' (seriously).

The Age, at least, points out that the BMI index is a crude tool for measuring how overweight someone is (it doesn't allow for any differentiation between muscle and fat).Neither The Financial Times, The Telegraph, The Mirror, or The Times note this. None of the newspapers mention that the sample used to estimate the figure was highly unrepresentative, as it consisted of 14, 000 people who attended a National Blood Pressure Screening Day, that is, 14, 000 people who were worried enough about their blood pressure to have it checked; or 14, 000 of whom a higher than average proportion could probably be assumed to be overweight, as the two conditions are correlated (or, more importantly, commonly known to be correlated, such that if you are overweight you're more likely to be worried about your blood pressure). Also, the age distribution of the sample may have been skewed such that people of middle age and older were over-represented.The report itself is obfucatory on this issue - it states that 'close to 14, 000' Australian adults aged 18 to 95 participated - close to? The authors have no qualms in printing figures like 7, 352, 529 (number of Australian men aged 18 and over) and 2, 412, 192 (the number of Australian women 'likely overweight'), so why so coy when it comes to piddling tens of thousands? They don't give figures for each of the age ranges they examine, except for the 45-64 age group, of which there were 5, 873. It makes me wonder if perhaps the older age groups (45+) were heavily over represented in the sample, something the hard figures, had they been provided, would have made clear thus (further) undermining the claim that 60% of the Australian population aged 18 and more is overweight.Further, according the the Australian National Health Survey 2004-2005, the age group 45 - 64 has the highest proportion of people who are overweight (50% women, 72% men aged 45 to 54; 58% women, 72% men aged 55-64 compared with 28% women and 36% men aged 18-24). If this group, and those older (56% women and 58% men overweight aged 65-74 - though they're roughish figures as the graph isn't entirely clear) made up the main body of the sample of 14, 000, then it raises serious questions about the usefulness of this study in estimating the proportion of the Australian population that is actually overweight.Page 10 of the study notes that 'Although the full methods and results of this study are in preparation for publication they have been accepted for presentation by Dr Carrington at the prestigious European Society of Cardiology Scientific Meeting in Munich (September 2008)'. Which is fair enough, but doesn't seem like a good enough reason to hold back on either methods or results before releasing the report on the world with the schlocksome title 'Australia's Future Fat Bomb'. Like most things, if and when the methods used and results obtained are questioned by the authors' academic peers, it won't be picked up by the press, and the uncertain figure of 9 million fat Australians will have taken on the status of commonly accepted fact.

Monday, June 23, 2008

Numerous studies have shown that the media is the primary source of scientific information for the public after they leave school (Detjen, 1995; Nelkin, 1995; in an Irish context, Trench, 2007 - although he admits that this claim is difficult to test rigorously). Conrad (1999, p. 285) says 'Science journalists [are] gatekeepers for the infusion of scientific information into the public sphere'. Not alone does coverage of science issues have an effect on the lay public, it may also influence the process of science itself. Journal articles covered in The New York Times received 72.8% more citations in the science literature than control articles in the twelve months following that coverage (Phillips, Kanter, Bednarczyk & Tastad, 1991). The effect of the coverage of science issues on public attitudes and behaviour is difficult to assess, for many reasons (Gregory and Miller, 1998) but there is evidence that the media can influence people's behaviour in some instances (Miller, 1999; Grilli, Ramsay, Minozzi, 2002) - for example, coverage of the ozone controversy led to people buying fewer aerosol sprays (Nelkin, 1995) or, anecdotally, the decline in MMR vaccine take-up coincident with the most fevered reporting of its alleged link with autism (Health Protection Agency (Britain), 2005). Logan, Fears & Wilson (1997), in an examination of coverage in the late 1980s of risks from exposure to electromagnetic fields, argue this coverage influenced the implementation of regulatory statutes in several US states, despite a National Academy of Sciences review concluding that the available research was equivocal on the link between exposure and health risks. They suggest that 'journalists can inadvertently initiate a process in which legislation gets ahead of scientific evidence'.

Scientists vs journalists: an overview

The clash of cultures between science and journalism has been the subject of long debate in the science communication literature (Trench, 2007). Science is slow and precise, while journalism is fast, short, and often imprecise (Hartz and Chappell, 1997). The language used in each field is quite different. The language of science is guarded and qualified, while journalists make much use of metaphors, and are principally concerned with making their writing readable, simple, understandable and entertaining (McCall, 1988; Anton and McCourt, 1995; Nelkin, 1995).

Scientists have frequently pointed out that the media ignore both the process and the substance of science (Nelkin, 1995). The reliance of journalists on pre-packaged information such as press releases and staged events for science information has come in for a good deal of criticism (Shepherd, 1979; 1981; Nelkin, 1995; Agnell, 1996; Saari et al., 1998), as has the tendency to cover science in an episodic fashion, with an emphasis on 'breakthroughs' and 'magic bullets' (Wilkins and Patterson, 1987; Logan, 1998). The tendency of the media to cover emerging, or 'breakthrough' science as fact, despite its preliminary data being, perhaps, very tentative has also been frequently criticised (Nelkin, 1995; Logan et al., 1997; Logan et al., 2000b).

Accuracy

Many surveys have found that scientists' main complaint about press coverage is that it is inaccurate (Dunwoody, 1993) or incomplete (Tankard & Ryan, 1974; Pulford, 1976; Borman, 1978; Pellechia, 1997) especially when it comes to methodological details (Dunwoody, 1986; Goldstein, 1986; Pellechia, 1997). This view is not unanimous, however - Wilkes & Kravitz (1992), using interviews with first authors of science papers, found that 86% rated coverage of their studies as accurate. This echoes previous findings that accuracy ratings are higher when scientists are asked to evaluate news reports of their own work than when they are asked to evaluate science reporting in general (Dunwoody & Scott, 1982; Pulford, 1976; Tichenor, Olien, Harrison, & Donohue, 1970). However, Bubela & Caulfield (2004) in an analysis of gene discovery stories in the Canadian print media (one which was not based on interviews) found that 82% of the newspaper reports assessed contained 'no significant technical or scientific errors'.

While journalists may be largely successful in correctly reproducing the figures produced in journal articles, they are less successful in putting these figures in context. Methodological details are crucial if the results of a study are to be rendered meaningful (Tankard & Ryan, 1974). Despite this, Singer (1990), in a comparison of news reports of scientific studies in the American media with the original research articles found that 48% gave no mention of research methods at all; of those that did mention research methods, 35% gave inadequate information, and 7.1% presented methodolgical information that was simply wrong.

Another 1990 study found that it was not common practice in either to include methodological details in newspaper reports (Evans, Krippendorf, Yoon, Posluszny & Thomas, 1990); a finding replicated by Pellechia (1997) in an analysis of three prestige US newspapers over three decades. Also frequently found to be lacking in science stories are qualifiying statements or other information that would limit the findings or conclusions of the research (Dunwoody, 1986; Goldstein, 1986). Adequate information is crucial 'to meet the needs of an intelligent nonspecialist who wants to evaluate the situation being reported on' (Klaidman, 1990, p.120). In the absence of such information, it is hardly surprising that Hargreaves et al. (2003) found 79% of respondents reporting that they at least occasionally 'felt confused about scientific issues'.

Public Relations and Press Releases

A number of studies have examined the correlation between a research report being press released by a major science journal and its subsequent coverage in the press. De Semir, Ribas & Revuelta (1998), in a study of 142 newspaper articles referring to studies published in the British Medical Journal, Science, Nature and The Lancet found that 84% referred to studies that had featured in press releases. Entwistle (1995) found a similar congruence, with studies in the BMJ and The Lancet which had been press released accounting for 86% of subsequent newspaper stories. Bartlett, Sterne & Egger (2002) found an even more extreme correlation, with every article in The Times and The Sun which reported on studies from the BMJ and The Lancet having been press released.

Public relations officers and press releases are seldom quoted as sources in newspaper reports - in the Hargreaves et al. (2003) survey above they are quoted as a source in 1%, 6% and 5% of MMR, GMR and climate change stories respectively. Bubela & Caulfield (2004) found a similar reticence in citing press releases as source - only 2 articles out of 627 studied did so.

This is interesting in the context of a (possibly unrepresentative) quote from a science journalist in Hargreaves and Ferguson (2000):

“Scientists are useless, which is why there are armies of PR people in universities, research

councils and funding agencies. In fact when you are not dealing with a straight good news

science story, and instead with any kind of story with an implication for how science appears

or what it plans to do, it is almost impossible to get past the PR people and talk to a real

scientist.”

It is conceivable, therefore, that a story may 'quote' a scientist, but that this quote has been derived from PR personnel or a press release drawn up by the PR department of the institution the scientist works in. However, health correspondents interviewed by Entwistle (1995) said they would not rely on press releases alone as a source for their stories, and regarded access to the full text of the journal article as essential to provide them with adequate information for their story. As these claims were self-reported, they are, perhaps, questionable. Woloshin & Schwartz (2002) found that of 127 press releases issued by 7 high profile journals, only 23% noted study limitations, while industry funding was noted in only 22% of 23 studies receiving such funding. In light of this finding, it would seem essential that journalists do not rely on press releases alone.

Sources: Unpublished research, Opinion and Maverick Science

Not all newspaper coverage of medical issues is based directly on published research in academic journals. The credibility of reporting on health issues is further undermined by pre-emptive coverage of studies which have not yet been published, and of clinical trials that have not yet reached completion. Schwartz, Woolshin & Baczek (2002) studied news stories on research abstracts presented at scientific meetings, including the 12th World AIDS conference, the meeting of the American Heart Association, and that of the Radiological Society of North America. They found that the studies presented at the meetings, despite not having been published or validated, received substantial attention in the media. They also found that many of these studies had weak designs, were small, or were based on animal or laboratory studies; 25% of the abstracts which were covered in the media remained unpublished 3 years later. Pre-emptive coverage may also impact on the public's perception of the validity of the scientific method. Between 1997 and 2002 the drug Pleconaril was widely reported as being a miracle cure for the common cold, despite still being in clinical trials (Schwitzer, 2003). The drug never got past this stage and onto the market as it failed to win FDA approval. 'This kind of story...allows the public to distrust science...it becomes easy for people to feel that scientists don't know what they're doing' (Dr. Ronald B. Turner, in Schwitzer, 2003).

A 2003 British Economic and Social Research council report on media coverage of three major science news stories - the MMR-autism controversy, climate change and cloning/genetic medical research - found that journalists quoted a scientist as the source of information for their story in 36% of cases for MMR, while the public was cited as source in 20% of cases. This contrasts with scientist as source in 58% of cases for Cloning/GMR stories and the public in just 11% of cases (Hargreaves, Lewis and Speers, 2003) This is notable: the MMR story was one with which the public could engage very directly, while GMR stories are more abstract and distant from the every day lives of the public. As Hargreaves et al note: 'the use of the public as a source in science stories would seem to be particularly important in any attempt to engage people with science stories...It is a way of signifiying that the topic is of public interest'. In neither case were the public better placed to argue the scientific merits or demerits than scientists, but because of the public's perceived engagement in the MMR controversy, their opinions were given greater parity with scientific opinion than in the case of GMR.

The appeal to the public for information is indicative of the media's strategy of balancing views in controversial stories. While this is a central journalistic value (see e.g. Westersthahl's 1983 model of objectivity in journalism which has balance as a core element), its effect in this controversy 'tended merely to indicate that there were two competing bodies of evidence rather than offer[ing] more substantive evaluations of the case for or against a link [between MMR and autism]' (Hargreaves et al., p. 23). Going beyond the problems with using this balanced 'political model' (Check, 1987) of reporting, even defining the boundaries of legitimate science can be problematic, as in the case of 'maverick' science (Dearing, 1995) - wherein an individual scientist may propose an unorthodox theory which receives wide coverage despite having a brittle or non-existent evidence base. Reporters seldom ask how sources know what they know, or what evidence this knowledge is based on (Tavris, 1986).

Hargreaves, I. and G. Ferguson. (2000). Who's misunderstanding whom? An inquiry into the relationship between science and the media. A report produced for the Economic and Social Research Council, 12 September 2000. [online] http://www.esrc.ac.uk/esrccontent/PublicationsList/whom/whofirst.html. (Accessed on 24/01/08).

Hargreaves, I., Lewis, J. and Speers, T. (2003). Towards a better map: Science, the public and the media. Economic and Social Research Council. [online] http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Images/Mapdocfinal_tcm6-5505.pdf. Accessed 25/01/08.

On June 20th The Irish Independent lifted a piece from the London Times reporting on a new study investigating differential gene expression in male and female brains. The Times manages to include the caveat, from the study's authors, that 'their work needed to be followed up to examine whether any human behavioural or health differences were related to the sex-specific gene expression profiles'. This is lopped out of the Irish Independent article, leaving us only with the impression that ' Women and men may genuinely think in different ways, according to research that has found subtle genetic variations between their brains'.The press release for the study even contains the very specific warning that 'the study does not determine whether these differences in gene expression are in any way functionally significant. Such questions remain to be answered by future studies'.

where W equals the thickness of Cheddar in millimetres, b the thickness of bread, d the dough flavour modifier, s the thickness of margarine or butter, m the thickness of mayonnaise, c the creaminess modifier, v the thickness of tomato, p the depth of pickle and l the thickness of the lettuce layer.

The Star has it: 'Boffins' cheesy formula'.

The formula is the result of research conducted by senior research fellow Geoff Nute and colleagues at the university’s Sensory & Consumer Group in the Division of Farm Animal Science. Using human assessors and complex technological measuring devices, Geoff’s team has successfully ‘mapped’ the flavour profile of hundreds of samples of Cheddar.

Geoff explains: “We used specially trained human taste testers to sample a range of Cheddar cheeses in a carefully controlled environment and combined results from these tests with instrumental data obtained using colorometers and pressure sensors to obtain precise measurements of variants such as yellowness, crumbliness, creaminess and tanginess.

“The results of our research have been extrapolated to produce a formula which takes into account modifying characteristics of individual cheeses and the ratio of popular fillings and achieves a mathematical balance of flavours in order to gauge the correct thickness of the Cheddar.”

Philip Crawford, chairman of the West Country Farmhouse Cheesemakers group, adds: “We are very proud of our authentic farmhouse Cheddar which we make by hand on our farms using only milk from our own cows. This means each variety of West Country Farmhouse Cheddar has a unique character and we were fascinated to know which combinations of sandwich fillings work best with each cheese. Collaborating with Mr Nute and his team we have managed to create the Cheddarometer and reveal the blueprint to everyone’s perfect cheese sandwich.”

Geoff Nute's staff page on the University of Bristol website lists his most recent research as being an analysis of the 'Eating quality of pork loin steaks from light slaughter weight boars and boars vaccinated with IMPROVAC™' presented at a conference in Spain.

I don't doubt the research was done, I'd just like to see it if the West Country Farmhouse Cheddar are to trumpet the academic credentials of their cheddarometer.