Monday, November 19, 2007

New Data from Scottish National Health Service Show that Smoking Ban/Heart Attack Study Was Junk Science, Show Danger of Science by Press Release

New data released last week by the Scottish National Health Service reveal that the national smoking ban had no detectable effect on heart attack hospital admissions, despite an earlier claim made by University of Glasgow researchers and disseminated to the media via press release that the smoking ban resulted in a startling 17% decline in heart attacks.

The data, which are reported in an article by Michael Blastland published by the BBC News, were reported by the National Health Service's Informational Services Division. They include all hospital admissions for heart attack nationwide, reported as annual standardized rates. The data show a decline of about 8% in the year following the smoking ban, which is only slightly higher than the approximate 6% decline the year preceding the smoking ban.

If one looks at the trend in heart attack rates from 1998 through 2007, one finds that based on the pre-existing trends in heart attack rates, there has been no apparent change in the rate of heart attacks following the smoking ban, which was implemented in March 2006.

These data also confirm my earlier analysis, in which I pointed out that a huge decline in heart attacks in Scotland from 1999 to 2000 occurred prior to the smoking ban and reveal the large degree of baseline variability in the data, meaning that only a junk science study would conclude that the observed decline in heart attacks during the 10 months following the smoking ban was attributable to the smoking ban. In fact, the data released last week reveal that the largest drop in heart attack admission rates occurred not following the smoking ban, but from 1999 to 2000.

Blastland writes: "It was dramatic research that made headlines everywhere. A 17% fall in the number of heart attacks in the year since Scotland stubbed out smoking in public places. Startling - if true. Few questioned the research when it was revealed two months ago. ... Then a week ago, with rather less fanfare, routine statistics on hospital activity were published by the official source for health data in Scotland, as they are every year, this time including the time since the ban. These show a fall in heart attacks for the year from March 2006 - not of 17%, but less than half as much at about 8%. ... The percentage falls in the three years before the ban were 5.1%, 4.7% and 5.7%. ... The latest release also makes clear that even an 8% fall in heart attacks is not unprecedented. There was another, larger drop between 1999 and 2000 of about 11%. This seems to demonstrate significant variability around the trend, suggesting that last year's 8% drop might even be the result of chance. It is conceivable, although perhaps unlikely, that the smoking ban had no effect at all. The figures could be a result of no more than the ordinary ups and downs of statistical variation from one year to the next."

Blastland's observations are consistent with my commentary of September 13, in which I wrote:

"To see why a thorough review of the actual study is necessary, just look at the hospital discharge data for heart attacks in Scotland. We find evidence of a large single-year decline in heart attacks of about 10% which occurred in the absence of any smoking ban. This indicates that there can be a large year-to-year variation in heart attacks in Scotland and makes it especially difficult to determine whether a 17% annual decline in heart attacks is due in part to random variation, and if so, to what extent. From the information that was provided to us, it is impossible to conclude that the smoking ban was responsible for the observed decline in heart attack admissions."

"It is also important to consider the fact that the more rapid decline in heart attack admissions in Scotland appears to have begun prior to the smoking ban. A drop in heart attack admissions of 17% (equal to what was observed after the smoking ban) actually occurred during the three-month period immediately prior to the smoking ban. It would be very questionable to attribute the observed heart attack decline to the smoking ban in the face of this evidence that heart attacks were declining substantially prior to the ban. It is entirely possible that some other factor is responsible for the observed decline."

As it turns out, it does not even appear that there has been a change in the rate of decline in heart attack admissions in Scotland. Clearly, the conclusions drawn by the researchers and disseminated to the media and the public prior to peer review of their study were premature.

The researchers who conducted the study had issued a press release on September 10, in which they highlighted the conclusion that there had been a 17% decline in heart attacks which was largely attributable to the smoking ban. The media widely disseminated this conclusion, with headlines such as: "Scottish Smoking Ban Leads to Huge Drop in Heart Attacks."

Despite issuing their press release, the researchers did not make their study available for public review and it appears that they are still not releasing it. Blastland reports that "because the data on which the StopIt study was based has never been published, and nor has the study itself, it is impossible to say exactly how it was done. Attempts to obtain it or to talk to the lead researcher have gone unanswered."

The Rest of the Story

There are two important implications of this story.

First, it demonstrates that tobacco control science is quickly deteriorating into junk science. I believe that investigator bias is becoming so strong that it is leading to completely invalid study conclusions. The Helena study conclusions, and now the conclusions from Pueblo, Piedmont, Bowling Green, Scotland, and Ireland, have all been cast into doubt. An apparently strong bias is clouding the scientific rigor of the papers being produced in this area. It represents, in my view, a true and unfortunate deterioration of the scientific integrity within the tobacco control movement.

Second, this story demonstrates the dangers of the science by press release approach which has now become the standard for these smoking ban/heart attack studies. It shows precisely why it is not prudent to issue findings that have not been validated by a peer review process. Suppose that upon peer review (or in this case, simply upon release of more complete data), it turns out that the study findings are unwarranted and invalid. Are the researchers going to then issue a press release stating that they were wrong, that their conclusion was premature and invalid, and that all news articles that the public throughout the world read to the contrary should be disregarded?

I highly doubt it. I doubt that the researchers of the Scotland heart attack study are going to pull an Emily Litella and issue a press release tomorrow saying: "Never mind."

The point is that the word is already out there. The conclusions have already been disseminated. It is too late to retract them now that they have been shown to be wrong.

This is why I view it to be inappropriate and unethical to release the results of a scientific study prior to peer review and publication. There is one exception, and that is IF (and only if) one makes the study available for public scrutiny and review. If you are going to go to the press to disseminate study findings, then you have to be willing to allow the public to see the study upon which those findings are based. Otherwise, there is no opportunity for review and scrutiny of your work.

In this case, the researchers apparently did not make their study available for public review and it appears that they still refuse to release it for public review.

The scientific standards of the tobacco control movement are rapidly crumbling. And the ethical standards don't seem to be that far behind.

No comments:

About Me

Dr. Siegel is a Professor in the Department of Community Health Sciences, Boston University School of Public Health. He has 25 years of experience in the field of tobacco control. He previously spent two years working at the Office on Smoking and Health at CDC, where he conducted research on secondhand smoke and cigarette advertising. He has published nearly 70 papers related to tobacco. He testified in the landmark Engle lawsuit against the tobacco companies, which resulted in an unprecedented $145 billion verdict against the industry. He teaches social and behavioral sciences, mass communication and public health, and public health advocacy in the Masters of Public Health program.