Share

Derek Lowe's commentary on drug discovery and the pharma industry. An editorially independent blog from the publishers of Science Translational Medicine. All content is Derek’s own, and he does not in any way speak for his employer.

Alzheimer's Disease

Neurotrope: A Shameful PR Stunt

There was some Alzheimer’s news the other day, but it wasn’t actually about Alzheimer’s. Not very much. It was more about hype, press releases, and a grievous lack of understanding of statistics.

Via @AndyBiotech and @biotechtoreador on Twitter, I came across this release about an announcement from a small company called Neurotrope, who have been developing bryostatin as a possible Alzheimer’s therapeutic. (Update: a spokesperson for Neurotrope has contacting me, saying that the release I’m quoting from, which was put out by someone called the “Small Cap Investor Group”, has no connection with the company. I have to also note, though, that its author, Sharon di Stefano, specifically states that she spoke with the company’s CSO and CEO about the trial. Her record with small-cap stocks is, to put it mildly, not encouraging). (Update 2: see this article by Adam Feuerstein about di Stefano and how her article ended up being circulated at all. Something odd is going on). Natural products chemists and organic synthesis fans will perk up their ears at that name, because the bryostatins are well known as examples of complex marine natural products. Bryostatin-1 has been reported over the years to have CNS effects, including enhanced memory in animal models, which led to the clinical effort on Alzheimer’s.

But we all know what the clinical success rate is in that field – arguably zero, and inarguably very close to it. Here, though, is what the Small Cap Investor Group flacks had to say about the company’s Phase IIa results:

Its Phase IIb of 147 patients underwent different dosages of Bryostatin along different timelines to see which worked best. Although the study was geared mostly for safety and tolerability, a surprise ensued – efficacy. Even more impressive was the generation of statistically significant results that exceeded FDA standards, which hold that a ‘p-value’, defined as the probability that good results are not due to chance, should be less than 0.1 in Phase II trials. In other words, 90% of patients are responding to a treatment. Neurotrope, on two standardized tests blessed by leading AD organizations, delivered figures even lower, meaning greater probability of results not being accidental.

This is wrong on so many levels. For starters, the definition of a p-value is completely wrong, and so brainlessly wrong that no company should have allowed such rot to go out under their company masthead. Now, p-values get misinterpreted constantly, often by people who should know better, but this is above and beyond. No, make that “below and behind”; that’s more accurate. This number does not mean that 90% of the patients are responding. And to go on, the FDA does not have some standard that a p-value has to be less than 0.1, because most people would start with 0.05 and go down from there if they’re basing decisions on that number. The p-value among patients who completed the therapy was < 0.07. Most statisticians with any dignity refer to values above 0.05 as “not significant”. This is embarrassing stuff.

And the results that Neurotrope “delivered” are being overstated as well, crap about statistical significance aside. Their p-values are one-tailed, meaning that they’re testing for significance while (statistically) ignoring the possibility of failure. This is inappropriate for a investigational drug study, so much so that this tutorial site at UCLA uses the example of testing a drug’s efficacy as a clear example of when you don’t want to use a one-tailed p value. I would suggest that Neurotrope pay close attention to another section of that same FAQ page:

Choosing a one-tailed test for the sole purpose of attaining significance is not appropriate. Choosing a one-tailed test after running a two-tailed test that failed to reject the null hypothesis is not appropriate, no matter how “close” to significant the two-tailed test was. Using statistical tests inappropriately can lead to invalid results that are not replicable and highly questionable–a steep price to pay for a significance star in your results table!

I could not have put it better. Furthermore, the test that the company makes much of is the Severe Impairment Battery (SIB). It’s a 100-point scale, and the company says that the bryostatin-treated patients improved by 2.6 points on it at 13 weeks (difference between treatment group and placebo). Big deal. Aricept (donepezil), an approved Alzheimer’s drug which is already known not to slow the progression of the disease one bit, but merely improve symptoms for a while, has an improvement of 5 SIB points at that same time point (as AndyBiotech noted from the drug’s package insert). 2.6 points is not clinically meaningful – as the company itself shows, it’s only statistically meaningful by the most charitable interpretation possible.

That fluff piece about the company, then, is ridiculous from a scientific standpoint. What am I saying, though – it’s ridiculous from any standpoint. Here, have some more:

“Neurotrope, with scientific vigor, showed positive, significant results from its ‘Completer’ group of patients. . .(bryostatin) acts like a car battery jump-start to grey matter, restoring life and energy to brain components that have died or gone dormant from AD. Early proof from the Phase II study should be viewed with excitement from the AD community.“

How anyone is allowed to go on like this about a publicly traded company is beyond me. “Restoring life and energy to brain components that have died”? I used to hear more believable pitches from Mexican border-blaster AM stations on late-night car rides in the 1970s; this sounds like Reverend Ike selling Prayer Cloths. And at least he admitted up front that he was just in it for the money. <s>What’s Neurotrope’s excuse for this publicity stunt?</s> (Update, as mentioned above, Neurotrope is disavowing any connection to the quotes above – as well they should.)

hmm..it says the US Attorney for the NJ District + SEC are after him. Thus, I really wouldn’t be surprised at all if The Donald takes a personal interest in squelching the oppressors of this fine, upstanding fellow businessman

Dear Buyers,
True, I was not part of Goldman Sachs, and when you’re a sell-side analyst with a dearth of institutional relationships but want to practice your trade, you go where you can. I was not involved in any shady business in so-called bucket shops – I only produced work I loved, independent of any influence, with good performance.

I thought that Neurotrope’s “results” would be addressed in this blog and I am glad that it was. I have read misleading analysis touting other Alzheimer’s drugs, but never quite to the extent of this one. Here is what one critic–Professor George Perry–reportedly said about the shortcomings of the press release:

“Despite Neurotrope’s Ph 2b being only a 3 month trial, there was a 30% drop out rate. Did you catch how Neurotrope factored those drop out patients into their PR of the trial’s results? The lack of transparency from Neurotrope is disturbing and unusual. No Ph 2a scores to scrutinize, nor scores from the compassionate use patients. Only brief mention of side effects associated with the Ph 2b, no details. No scores of the 40 microgram group released. No adcs-adl scores released. Choosing to only release SIB aggregate score for the 20 microgram group. No combined overall aggregate score for all the patients enrolled in the trial.”

Neurotrope’s bryostatin-1 is a protein kinase C alpha and epsilon activator and protein kinase C alpha is a trigger for Alzheimer’s disease because it increases oxidative stress.

“Malinow’s team found that when mice are missing the PKC alpha gene, neurons functioned normally, even when amyloid beta was present. Then, when they restored PKC alpha, amyloid beta once again impaired neuronal function. In other words, amyloid beta doesn’t inhibit brain function unless PKC alpha is active.”

Both protein kinase C activation and the beneficial pathway by which it improves memory and learning (the phosphatidyinositol 3-kinase/Akt pathway) are damaged in Alzheimer’s disease so Neurotrope is relying upon bryostatin-1 to activate one enzyme that is damaged (protein kinase C) to activate another enzyme that is damaged in Alzheimer’s disease (phosphatidyinositol 3-kinase). The chances of success using this approach are virtually nil.

So all around–from the science, to the misuse of statistics, to the unsupported hype–this was to put it as kindly as possible–an unfortunate press release.

I am pleased to discover another fan of the great Reverend Ike. When I lived in the Bronx, I often found luridly colorful brochures advertising The Rev’s offerings under my door. My recollection is that for $20/month you could join the Prayer Plan and for $30/month you could join the Blessing Plan. His tabernacle and sanctuary were in a converted old movie theater in Washington Heights. He was fond of quoting the line, “The best thing you can do for the poor is not be one of them.”

I am sorry to report that The Rev departed this vale of tears in 2009. However, I’m pleased to learn that he has a worthy successor in Neurotrope.

While I agree with the bulk of your argument, I would dispute the premise that a p value of 0.05 is sacrosanct. In spite of the fact that Ronald Fisher proposed that this level was some kind of minimum that all statistical tests should exceed, it is still an arbitrary value. There is no de novo reason that p<0.07 is a less valid threshold than p<0.05.

One: Does anybody know what type of Bryostatin was used? Was it Bryostatin 1, or a different member of the group?

Two: Anybody know what dosages were used? The Bryostatins are, as was previously mentioned, a famously complex family of molecules — and preparing enough for almost 150 patients cannot have been cheap or easy. (For this reason alone, I wouldn’t call Neurotrope’s project just a “pump and dump” — for, were that the case, they would likely have picked a cheaper and more predictable chemical to bring to trial! This is assuming, of course, that the doses used were in the milligrams, not the micrograms or nanograms.)

From The article quoted by Mr. Lowe – Maria Carrillo, Chief Science Officer of the Alzheimer’s Association said even though this Phase II study was small and of short duration, “there was a trend towards improvement in memory, thinking and behavior”

Dr. Dean Hartley, another research heavyweight at the Association, admits there is nothing else approved that “slows down or prevents the disease.” A true medical need exists and if more detailed data unveiled later this summer is good, FDA may grant its coveted Breakthrough Designation.

The company lists all current work as on bryostatin 1, but dangles “bryologs” in one of their pipeline figures. The doses are quite low, apparently, which is not a surprise, because bryostatin 1 is VERY potent in doing it’s thing. So the cost of the dose is not necessarily that high.

Bryo 1 failed in the cancer trials it was entered into, and has not been a godsend for HIV latency either, and it’s kind of sad to see it repurposed in such a setting. It’s also sad to see the warped interpretation of clinical data to pump the stock.

Thank you, Mr. Romanovsky. What I wrote was certainly not a press release and I am NOT a stock promoter. I just like the company. Many replies here, including from the highly-educated Dr. Lowe, did not take the time to understand mITT analysis. I did, and I appreciate your post.

Suuure, you probably love mITT, as it serves promotional purposes. From paper evaluating use of mITT:

We found a strong association between trials classified as mITT and for-profit agency sponsorship (adjusted OR 7.41 [95%CI, 3.14 to 17.48]; P < .001) as well as the presence of authors' conflicts of interest (adjusted OR 5.14 [95%CI, 2.12 to 12.48]; P < .001).

I don’t see any difference in PR stunts between this company and any of the Big Pharmas selling their anti-ALZ compounds. They are all guilty of not actually doing anything while providing false hopes for a devastating disease.

We’re all going to die. The only questions are whether you will have your marbles when you do and what will be the cause. Personally I don’t think this disease is devastating because it kills you – it’s devastating because you slowly loose that which makes you you.

Derek Lowe(Not the Low ball pitcher) “There was some Alzheimer’s news the other day, but it wasn’t actually about Alzheimer’s. Not very much. It was more about hype, press releases, and a grievous lack of understanding of statistics.” Correction you SOB There was a Phase 2 Alzheimer trial result for safety and it was effective using (mITT) analysis statistics methods.

Thank you, Intent Investor. I’ve studied clinical trials for years, and mITT was new to me so I spent a few hours of research. FDA is increasingly accepting this methodology, which helps, through computer simulation, fill in blanks for drop-outs that have been tested for safety, tolerability and efficacy, but had not made it to the ‘finish line’. It’s becoming more common for AD research – Biogen uses it, as AD patients often forget appointments or are unable to function enough to complete clinical trials. I am surprised Dr. Lowe, with his fine credentials, does not recognize this. But you get it, and it’s much appreciated.