Friday, April 05, 2013

Meta-analysis in the middle

My general view both in general and as a policymaker in my small local-office pond is that a widely held (few expert dissenters) and strongly held (high confidence) consensus position might as well be a fact as far as the policymaking is concerned. A contradictory study is just that, a study. If it turns out to be accurate, then the consensus will eventually fracture soon enough and create problems for policymakers, but there's no need or even a rational way for policymaking to jump the gun.

In the gray zone between just one study and a consensus is the meta-analysis, and I'm still trying to figure out what policymakers should do with them. If the meta-analysis reinforces the consensus position - say, Oreskes' review of scientific abstracts on climate change - then it's just a helpful tool in examining the consensus. OTOH, if the meta-analysis seems to point in a different direction - say, Choi et al. on adverse neurological effects of high-fluoride exposure in China and Iran - that presents a bit of a problem.

My assumption, generally, is that the consensus should be handling this. The main Canadian health research arm, Health Canada, looked at a lot of the same studies and reached a different conclusion:

These studies were included in the review conducted by the Expert Panel on fluoride convened by Health Canada in 2007. Despite the consistency in the results from these studies, the panel agreed that the weight of evidence does not support a link between fluoride and IQ deficit. There are significant concerns regarding the available studies, including quality, credibility, and methodological weaknesses, such as the lack of control for confounding factors, the small number of subjects, and the dose of exposure (Health Canada, 2008). Most of these studies performed in China were also included in the reviews conducted by other organizations and/or committees, which also mentioned that the significance of these studies is uncertain (IPCS, 2002; ATSDR, 2003; NRC, 2006).

I think Health Canada is probably closer to showing what the fluoride consensus is than Choi, although I think there's reason to believe the consensus, that therapeutic levels of fluoridated water are generally safe, isn't as solid as the climate consensus. Worth noting one difference between meta-analysis and consensus is the meta-analysis is only as good as the meta-analyst, while consensus hopefully has a broader base.