This week the US Food and Drug Administration voted not to ban GlaxoSmithKline's diabetes drug rosiglitazone (brand name Avandia). Their vote has been reported as a victory for the company. I don't think so: this saga tells an ugly story about our collective medical incompetence.

Rosiglitazone was first marketed in 1999. From the outset it was a magnet for disappointing behaviour. That first year Dr John Buse discussed an increased risk of heart problems at a pair of academic meetings. He was silenced. GSK made direct contact, then moved on to his head of department. Buse felt pressured to sign various legal documents and after wading through documents for several months, in 2007 the US Senate committee on finance released a report describing the treatment of Dr Buse as "intimidation".

In 2003, the Uppsala drug monitoring group of the World Health Organisation contacted GSK about an unusually large number of reports associating rosiglitazone with heart problems. GSK conducted two internal meta-analyses of their data in 2005 and 2006. These showed the risk was real, but although both GSK and the FDA had these results, neither made any public statement, and they were not published until 2008.

Why then? In 2004 GSK were caught ‑ famously - hiding data showing side effects of the antidepressant paroxetine in children: a court settlement required them to post all clinical trial results voluntarily on a public website.

Using this data source, cardiologist Prof Steve Nissen and colleagues published a landmark meta-analysis in 2007 showing a 43% increase in the risk of heart attack on rosiglitazone. People with diabetes are already at increased risk of heart problems.

The FDA found a similar risk in their own calculations, but voted in 2007 to keep the drug on the market. This is not insane: diabetes is tricky, 300 million people have it worldwide, a great many die from it, and rosiglitazone is unusually good at controlling blood sugar. Lots of dangerous drugs are kept on the market and then used less frequently, in extreme circumstances.

A consensus algorithm from the American Diabetes Association and the European Association for the Study of Diabetes, meanwhile, unanimously recommended against rosiglitazone. Although annual sales for rosiglitazone fell, they still remained over $1bn (£650m). Concerns continued to mount. So did the bad behaviour. In 2007, Nissen caught GSK out discussing a copy of his unpublished paper, which they had obtained improperly.

Then on 28 June this year Nissen published an updated meta-analysis of 56 trials in over 35,000 patients. Again it found an increased risk of heart problems. GSK's response to all this has been like the responses you get from homeopaths. There are seven trials since 2007, they said, showing no excess risk: fine, except there are 56 which collectively do show an excess risk. There is this other meta-analysis, they said, which looked at 164 trials: fine, except it's published in a fairly obscure journal, and it looked at trials lasting more than four weeks, when the others set the bar at trials over 24 weeks, because a heart risk takes time to develop. In any case, this other meta-analysis is not brilliant for GSK's case, since it points out that the company denied access to data from six trials which we know to have taken place. There is no excuse for companies withholding data from academics and doctors. But most revealingare the deep-rooted flaws this story exposes in our rather ad hoc systems for gathering, analysing, and disseminating evidence on risks and benefits of treatments.

This drug has been on the market since 1999, and it has seen billions of dollars of sales every year. There has been plenty of real patient experience of this treatment, but we have failed to capture it for analysis. Most of the trials included in these meta-analyses were not specifically designed to look at heart problems, and so the data on these is unpredictably inaccurate.

In an ideal world, for every patient, wherever possible, we could be gathering anonymised outcome data and comparing this against medication history. In an ideal world, wherever there is genuine uncertainty about which treatment is best, a patient would be randomised to one treatment, and their progress monitored. In an ideal world, these notions would be so embedded in our notion of what healthcare looks like that no patient would be bothered by it.

This isn't fanciful, or difficult, or disproportionately expensive. Instead we have a hotchpotch of incomplete monitoring systems and unforgivable secrecy.