I believe this is biology's century. I've covered science and medicine for Forbes from the Human Genome Project through Vioxx to the blossoming DNA technology changing the world today.
Email me, follow me on Twitter, circle me onGoogle Plus, or subscribe to my Facebook page.

Should Roche's Failed Trial Give Hope To Alzheimer's Patients?

Roche’s Genentech unit just released a mid-stage study of a drug that seeks to slow the progression of Alzheimer’s by blocking the accumulation of a protein called amyloid beta. Like every other medicine that has attempted this approach – which remains the main one being pursued by drug companies to battle Alzheimer’s – the drug failed to yield a statistically significant benefit.

Unlike those previous failures, however, there is a glimmer of hope in the data on Roche’s drug, crenezumab, which was presented today in Copenhagen at the annual meeting of the Alzheimer’s Association. Why? The data seem to tell a consistent story that the drug may have an effect.

Crenezumab didn’t result in a statistically significant benefit compared to placebo on performance in tests designed to assess how well patients were doing, but the group that got the drug did do better. This difference was consistent across different subgroups.

The patients with the mildest disease seemed to get the biggest benefit from crenezumab. This is very much what Eli Lilly saw with its amyloid beta drug, solezumab. Again, the consistency of the data is a reason for hope.

There are reasons to believe that crenezumab has a better chance than previous beta amyloid blockers. Bapineuzumab, developed by Pfizer, Johnson & Johnson, and Elan, caused dangerous brain swelling. Crenezumab, invented by Swiss biotech AC Immune and licensed by Roche, doesn’t seem to cause this side effect, and that has allowed Roche to give it at doses 50 times higher than those used with bapineuzumab. The dose in this study was 50% higher than in previous ones, and Roche is still working to figure out if it can push it still higher.

That is not to say that the odds of success are high. Generally, in drug development, companies now follow an approach that they don’t want to develop drugs unless they show clear benefit in early studies. What is being done in Alzheimer’s is much riskier: companies are swinging wildly for the fences in the hope that they will hit an out-of-the-park home run. As Timothy Anderson at Bernstein Research wrote in a note to investors:

“As a general rule, drug companies normally won’t advance compounds into phase 3 development unless the probability of success is estimated to be in the range of 60-70%. With the Alzheimer’s disease drugs described in this report, the probabilities are likely much lower (even optimistic experts won’t claim a figure higher than ~25%). What drives this? The tremendous size of commercial opportunity. As we have shown previously (Exhibit 4), a drug that successfully modifies AD progression could make Lipitor (sales peaked at ~$12B) look like a mid-sized product. “

We’re in the land of long shots here. But Roche is taking an innovation it learned in cancer – to use the mid-stages of the clinical trial process to conduct rigorous tests of an experimental medicine’s potential – here. And if you squint the right way, the crenezumab data do indicate that it might be worth moving into a larger study aimed at getting the drug approved.

A low dose of the drug was studied in 184 patients, 62 of whom got placebo; this part of the study failed unequivocally. Another 247 patients (84 on placebo and 163 on crenezumab) got the higher dose, and here there’s some hope.

The goal was for patients to do better both on two sets of tests: the 12-item cognitive subscale of the Alzheimer’s Disease Assessment Scale (ADAS-cog12), and the Clinical Dementia Rating-Sum of Boxes (CDR-SOB). By strict statistical measures, which require a p-value of less than 0.05, meaning a less than 1 in 20 chance that a result happened randomly, the study failed on both measures.

However, Carole Ho, Roche’s Group Medical Director of Exploratory Clinical Development, says that Genentech powers its phase II trials to deliver a p value of less than 0.2; by this measure, the study succeeded on ADAS-Cog12, but not on the second measure, CDR-SOB.

Here is what that result looks like:

As you can see, this isn’t a big difference. The placebo patients saw their ADAS-Cog12 scores drop 10.56 points, compared to 8.79 points for those on crenezumab. That’s just a 1.78 point difference (and that p value is just 0.19, barely making Roche’s internal hurdle). The amount of overall decline in both groups is a lot bigger than the difference between them; would individual patients even be able to notice this difference?

But things look a lot better if you cut the data into the patients into different subsets based on the Mini-State Mental Examination (MMSE) at the beginning of the study. A higher score on this test means better cognitive function.

In the least sick patients, the difference between the crenezumab and placebo groups is nearly doubled, and the p value would be significant if just those patients had been tested. (Don’t get too excited; statisticians don’t count subgroups like this because the odds are if you look at enough subgroups, *something* will turn out to be statistically significant.)

And hey, using another standardized test, the Dantes Subject Standardized Tests, you see the same trend, with a much bigger benefit on the patients with higher MMSE scores.

And in BLAZE, another clinical trial, the drug again seems to result in a difference in ADAS-Cog12 in those with a high MMSE score, but not in those with a low one.

Post Your Comment

Post Your Reply

Forbes writers have the ability to call out member comments they find particularly interesting. Called-out comments are highlighted across the Forbes network. You'll be notified if your comment is called out.

Comments

The data are clearly not positive, but the consistency arguments you present could indeed suggest a modicum of activity. However, the similarity with “what Lilly saw” with solezumab needs to be tempered: LLY ran two large trials; one saw an effect in the subset with least severe dementia (consistent with what is seen here with crenezumab); but the other (where the whole population was the mild disease state) saw no such effect. Everyone wants to believe that anti-amyloids might work in very early disease, but the largest study to date in such a population (with solezumab) showed no such effect.

The other questions is this: having accepted all the caveats and therefore accept that this picture MIGHT be real is whether such a profile, were it validated in a sufficiently large enough trial, is actually clinically of any value at all. The concern is not just the small magnitude of the effect, but critically the time-course. If you look at all the charts in the article, in each case the degree of benefit at 72 weeks was the same as at 48wks. Look at the ADAS-cog12 graphs (whether in the whole population or in the least severe subset) and the picture is clear. Any effect there is occurs in the first half of the study period. The decline in the last 6 months is absolutely parallel between the groups.

A material benefit that delayed the disease for a while and then failed would still be useful. But a small benefit that isn’t sustained will scarcely translate into any worthwhile clinical effect in the real world (outside of a trial setting). Patients and carers will not even notice.

So there is a double-whammy concern here: (a) is the picture real, given the statistical issues; and (b) a bigger problem – even if this IS real, is that profile encouraging enough to approve a drug, or see it used?

I also agree that it would be difficult to stop the program (as LLY proved, its difficult to stop a program with pretty convincing negative data!). Roche/Genentech will continue, and may even be successful. But I don’t see much hope here that anti-amyloids offer more than a tiny sliver of efficacy against dementia.

Even if Roche continue; even if they eventually get approved, this is the moment we can finally see how little of dementia can be treated with amyloid-targeting approaches, and how badly we need other mechanisms to be targeted.

I had similar thoughts to David (also posted/discussed here: http://lnkd.in/dngZ-Qs):

Sadly, I think this is just another case of false signal and hope emerging from statistical noise by cherry picking sub-groups to find what you want to see. While the p-value is below 0.05 for the sub-group, that is no longer enough to be statistically significant once you start dissecting the data and exploring multiple, smaller sub-groups. As Matt Herper explained, if you look at enough sub-groups, you are bound to find an outlier that falls in the right spot. And yet so many companies fall into the same trap, where hope beats statistics, resulting in desperate gambling that usually ends in costly failure.

Furthermore, even if the result is statistically significant, is the small difference meaningful enough to patients and their families? That is of course a very subjective question, but still I would argue probably not.

Having said all this, I do believe the results show some glimmer hope for another drug in another Phase II trial, but not for this one in a huge Phase III trial. The drug has had its chance to be sufficiently de-risked prior to Phase III, and despite the seductive signs of hope, it has failed. I would want to set the bar as high as possible in Phase II, and make sure that the pre-set endpoints are properly met (rather than relying on post-hoc data analysis) before risking yet another failure in Phase III, which might discourage even more pharma companies from working in this critical but most challenging area.

Kelvin — I really want to see the test Roche is doing with this drug in Columbia in patients who have a gene that makes them almost certain to get Alzheimer’s. That’s the best test proposed so far of this hypothesis. I’m impressed with how deliberately Roche is moving compared to Elan and Lilly.

Unfortunately this type of data “dredging” rarely leads to success. Big pharma, including Roche, have spent billions of dollars over the years confirming this very fact. At best, post-hoc analyses of the type that are leading to the suggestion that hope remains for this program should be viewed as hypothesis-generating exercises, rather than data-generating. History proves that the likelihood of generating positive data following a failed definitive mid-stage trial is even lower than prior to the generation of the first negative study. Bapineusumab and rosiglitazone are just a few of the recent examples where big pharma companies ignored definitive, failed mid-stage Alzheimer’s study data and bet the farm on a major phase 3 program on the basis of post-hoc analyses of their phase 2 failed studies. And the post-hoc phase 2 data “looked” really impressive.

There are numerous statistical and study design considerations that explain why post-hoc data cannot be used reliably to predict success of future studies. You can understand one of the issues by looking at the numbers of subjects per group in some of the analyses displayed in your article. No knowledgeable drug developer would claim (with a straight face) that studies of this size would ever be sufficient to claim an effect, let alone use these analyses to justify investing another $500M. Post-hoc analyses are always underpowered, biased, non-randomized, non-prespecified, and usually random assessments that should be ignored. But post-hoc analyses do serve a useful purpose in that they allow internal decision makers to “kick the can down the road” and defer responsibility for failure until a later point in time, at the expense of shareholders, who will foot the bill, and at the expense of patients, whose hopes for a cure will assuredly be dashed.

I agree that attempted replications of post-hoc findings rarely work out well. However, one interesting thing I haven’t seen mentioned is that the people who responded well to the drug received it IV, while the majority of the patients received a subcutaneous injection. It’s quite possible that crenezumab has a brain penetration issue. Roche might consider a mild only study with IV dosing.

That’s moderately good news. It’s challenging and critical research. Many hope that funding for Alzheimer’s research continues to grow. And, for those companies that succeed to create profitable drugs that have, in part, been supported through public funding, I’d like to see some leveraging of federal and CDC funding back to support sliding fee scale patients in Community Health Centers for research that is successful.