A Secondary Examination of The Adventure of the Cardboard Box

In November of 1995 stroke care as we know it drastically and permanently changed. With the publication of NINDS-2 the NEJM ushered in the interventional era of acute ischemic stroke (1). No longer were we powerless in our management of these patients. Finally we could offer them more than an aspirin to chew on, a corner to sit in, and an appointment with a neurologist in the morning. And yet NINDS-2 was not the first trial examining thrombolytic therapy for acute ischemic stroke. In fact three trials were published prior to NINDS-2 all of which were negative (NINDS-1, MAST-I, ECASS-1) with two finding an increase in mortality in patients given thromblytics (1,2,3). With the publication of NINDS-2 all this was forgotten. NINDS-2 was impressively positive, demonstrating a 13% absolute increase in patients who were given tPA that were alive and independent (mRS of 0 or 1) at 90 days (1). Supporters justified the 6% absolute increase in symptomatic intracranial hemorrhage by arguing that it did not increase 90-day mortality (21% vs 17%). Despite these impressive results there were still three negative trials to account for. What made NINDS-2 different than all the trials that came before it? Was it the agent? Supporters claim that tPA was the superior thrombolytic and we should ignore all trials studying other agents. Was it time? NINDS examined patients who received tPA within 180 minutes of symptom onset (half in under 90 minutes); two of the earlier trials examined patients who received thrombolytc therapy over a much broader treatment window. Was it the patient population? The authors of NINDS used very strict selection criteria to determine which patients were acceptable candidates. There was of course a fourth reason proposed by a less enthusiastic contingent, that being random chance. This more skeptical party posited that an intervention that possesses little or no efficacy, if studied enough times would eventually demonstrate positive results simply by chance alone. They reminded the more eager supporters of tPA therapy that though the findings of NINDS-2 may be true, taking these results at face value without further validation was not only bad science, but even worse medicine. Despite these warnings the FDA fast tracked the approval of tPA for acute ischemic stroke in under 3-hours and all other trials attempting to validate this benefit were abandoned. As Elliot Grosbard, Genentech scientist, said in internal communications in regards to further trials comparing streptokinase to tPA for acute coronary syndrome;

We do not know how another trial would turn out, and if we don’t come out ahead we would have a terribly self inflicted wound… (another study) may be a good thing for America, but it wouldn’t be a good thing for us.

Four consecutive trials were published following NINDS (all examining time windows greater than 3 hours) all six were negative and four demonstrating harm (4,5,6,7,8.9). It wasn’t until the 2008 publication of ECASS-3 that another trial examining thrombolytics for ischemic stroke demonstrated benefit (10). These benefits though not as impressive as NINDS-2 were convincing enough to make us forget the seven other negative trials examining similar time windows. Unlike NINDS-2 we were unable to claim ECASS-3 was different than these other negative trials examining similar patients during similar time windows. So instead, we just ignored them. Like our nostalgia for our endless childhood summers we have chosen to selectively interpret the literature that confirms our biases. Remembering only the fireworks, campfire tales, and days spent in the crashing waves of the Atlantic Ocean, we conveniently forget the sun burnt shoulders, poison ivy scorched legs and the tattered knees so commonly acquired during childhood adventures. This tunnel-vision has (mis)guided stroke care for the last two decades. Investigators continue to role the dice, ignoring all numbers that do not suit their purposes.

19 years after the publication of NINDS changed stroke management, the Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands (given the unfortunate acronym MR CLEAN) once again threatens to overthrow the infrastructure of stroke care (11). The results of this multi-center trial comparing endovascular therapy to standard care were presented at the 9th annual World Stroke Conference in Istanbul held in October 2014. Though these results were not published, with the fanfare NINDS-2 experienced after its initial announcement, it difficult not to see the similarities between these two trials. Like NINDS-2, this is the first trial to show benefit of a novel therapy in acute ischemic stroke. Like NINDS, these results are in direct contrast to the 3 trials published to date examining endovascular interventions for acute ischemic stroke.

On February 7th 2013, NEJM published 3 articles all examining the efficacy of endovascular treatment for acute ischemic stroke. All 3 trials were universally negative each one failing to demonstrate benefit in their own unique manner (12,13,14). These trials were instantly discredited by endovascular apologists stating a number of reasons why they should be ignored. For one they enrolled the wrong patients. These trials primarily used only a non-contrast CT to select patients appropriate for endovascular therapies. Most experts argue this is not the optimal imaging technique for selecting patients for endovascular interventions and all patients should have undergone CT-angiography before enrollment. These trials examined endovascular therapy during the wrong time period. Proponents of these interventions argued that time-to-clot retrieval was far longer than their current standards and these delays erased any benefits endovascular therapy may have provided. Finally and most importantly these trials used the wrong equipment. The Merci retrieval device was prominently featured in all three of these initial trials. A device that most interventionists now consider antiquated and, in a number of small trials demonstrated suboptimal performance when compared to newer devices used currently by most interventionists.

In direct contrast to these results, MR CLEAN is a significantly positive trial. Although the official manuscript has yet to be published a great deal can be garnered from the poster presentation abstract, press release and previously published protocol and statistical analysis plan (11, 15, 16). MR CLEAN is a multicenter randomized clinical trial comparing endovascular treatments for acute ischemic stroke conducted in the Netherlands. Using the prospective open label, blinded endpoint (PROBE) design comparing patients randomized to receive either endovascular treatment (consisting of intra-arterial tPA or urokinase followed by clot retrieval) or standard therapy (91% of which received intravenous tPA alone). Over a 5-year period, authors enrolled 500 patients aged 18 years or older with acute ischemic stroke and a symptomatic anterior proximal artery occlusion, which could be treated within 6 hours after stroke onset.

The authors found patients randomized to endovascular treatment were more likely to have improved neurological outcomes at 90 days with an adjusted odds ratio of 1.67 (95% confidence interval [CI], 1.21 – 2.30). Though there was a significantly higher rate of adverse reactions in the endovascular group (47% vs 42%), these rates did not seem to affect either 90 day neurological outcomes or morality (21%vs 22%). If you were to use the NINDS-2 definition of good neurological outcomes (mRS of 0 or 1), endovascular treatment would have demonstrated a 14% absolute increase in patients alive and independent at 90 days, with a number needed-to-treat (NNT) of 7.

So why was MR CLEAN positive when the 3 trials which came before were negative? Was it the patients? It is true that the authors of MR CLEAN were far more selective in the patients they included. In fact authors required patients to have an occlusion of distal intracranial carotid artery or middle cerebral artery(M1, M2) or anterior cerebral artery (A1) demonstrated with CT angiography (CTA), magnetic resonance angiography (MRA) or digital subtraction angiography (DSA) before they were enrolled in the trial. Was it time? Patients received both IV tPA and endovascular treatments far faster than any of the patients in IMS-3, Synthesis or MR RESCUE. In the MR CLEAN trial patients received their IV tPA on average 85 to 87 minutes after symptom onset and underwent endovascular therapy 196 to 204 minutes after symptom onset. Was it the devices used? In contrast to the initial 3 trials, 97% of the endovascular interventions performed in the MR CLEAN cohort utilized a modern retrievable stent device. Or was it just the random fickle nature of fortune that provided us with these impressive results?

Currently we do not have the full publication of MR CLEAN, so a detailed analysis of the results proves difficult. That being said there are a number of interesting points we can take away from the published protocol and results presented during the 9th annual stroke conference. Firstly the authors claim success in their primary outcome, which they define as “the score on the mRS at 90 days” (11,15). They claim this benefit by citing an adjusted odds ratio of 1.67 (95% confidence interval [CI], 1.21 – 2.30). What are we to take from this odds ratio? What exactly were they measuring and what imbalances were they attempting to adjust that randomization would not account for? In the statistical analysis portion of their protocol the authors only slightly expand on the vague nature of this outcome. The authors used an ordinal analysis in an attempt to quantify the benefits of endovascular therapy over the entire mRS. They then decided to further adjust these outcomes using multivariable logistic regression in an attempt “adjust for chance imbalances in main prognostic variables between intervention and control group”. The specific variables they chose to adjust for were age, stroke severity (NIHSS), time since onset, previous stroke, atrial fibrillation, carotid top occlusion and diabetes mellitus (11). This is the same statistical wizardry used in IST-3 to magically transform a decidedly negative trial into a statistically positive one (17). MR CLEAN marks the first time this type of statistical analysis was used as a trial’s primary end point rather than a secondary experimental, trial saving outcome.

To be clear this trial was an overwhelming success and this analysis is in no way intended to take away from these findings. Rather to question whether an adjusted ordinal analysis is the appropriate outcome to assess efficacy. We have discussed the problems with ordinal analyses in depth in a prior post, but briefly it is an attempt to granularize the data so as to detect smaller changes in outcomes than the more tradition dichotomous cutoff (mRS 0,1, or 2 vs 3,4,5,or 6) is capable of detecting. On face value this seems like a noble pursuit, but logistically presents a number of problems when employed in a trial. Most importantly, is an ordinal analysis an appropriate measure of functional outcomes? Ordinal analysis is an attempt to examine shifts across an entire functional scale. Minute changes in outcomes that would be missed by a dichotomous measurement. To do so one has to assume flawlessness of the collection process and intrinsic reliability of the functional assessment tool. We know that the reliability of the mRS is questionable at best. In fact when two neurologists assess the same patient their results will often differ by up to 2 points (19).

The MR CLEAN 90 day mRS data was assessed using a structured phone interview conducted by a trained research nurse. This trial employed an open design where the patients were not blinded to their group assignments, using an outcome scale of questionable reliability, collected by a phone interview. Authors then utilized a secondary adjustment for variables that should have been controlled by the randomization process. (11) This data is far from flawless. To think you can granularize such data and then extract meaningful outcomes is certainly an error in judgment. Such analysis should be reserved for secondary measures only after a more robust means of appraisal has proven fruitful.

Like all the stroke literature this leaves us trying to compare the soft endpoints of functional neurological outcomes to the hard endpoints of mortality and intracranial hemorrhage (ICH). Despite its success, like NINDS before it, MR CLEAN failed to demonstrate a mortality benefit for endovascular therapies in acute ischemic stroke. The mortality at 90 days was 21% and 22% respectively (15). Add to that a 5% increase in the rate of serious adverse events (47% vs 42%) in the endovascular therapy group. Despite the claim that the newer endovascular devices were safer and caused less bleeds the rate of clinically relevant ICH was statistically equivalent to the patients who received IV tPA alone (6.0% vs 5.2%) (15). This is the same rate of ICH seen in both the IMS-3 and Synthesis trials in which the MERCI retrieval devices were the primary means of clot retrieval (12,13). Furthermore there was a concerning increase in the rate of secondary ischemic strokes in a different vascular territory (5.6% vs 0.4%) and the number of hemicraniectomies performed (6.0 vs 4.9%) in the endovascular treatment group, though given the overall functional outcomes at 90 days were markedly improved in the endovascular therapy group these strokes may not be clinically relevant (15).

So why did endovascular interventions perform so much better in MR CLEAN than in any of the 3 trials that came before it? Was it the modern devices that create superior reperfusion with fewer complications? Interestingly the rate of recanulization in the intervention group at 24 hours was approximately 80% compared to 32% in the IV tPA group alone (15). When compared to the 24 hour recanulization rates in IMS-3 the intervention group were found to have approximately 80% with similar recanulizations rates in the IV tPA group as MR CLEAN (35%) (12). Furthermore the rates of ICH and secondary ischemic infarction seem to be no less than what was observed during IMS-3 and SYNTHESIS. Seemingly these newer devices add little as far as objective effectiveness. Was it time to reperfusion? Patients in MR CLEAN received both IV tPA administration as well as endovascular therapy incredibly fast. So fast that some may question the trial’s external validity. Despite the fact that patients in MR CLEAN underwent both IV and mechanical reperfusion significantly earlier than patients in IMS-3 and Synthesis, earlier treatment with endovascular therapy did not appear to improve outcomes (15). In fact in MR CLEAN, patients who received IV tPA therapy greater than 120 minutes after their symptom onset did better when randomized to the endovascular intervention arm. Conversely when patients received IV tPA prior than 120 minutes after symptom onset, endovascular therapy demonstrated no added benefit. In both IMS-3 and SYNTHESIS no temporal benefit could be demonstrated for patients receiving endovascular therapy (12,12). Was it the patients MR CLEAN selected that made a difference? Though MR CLEAN required CT angiographic proof of a large vessel occlusion, the resulting population seems very similar to the patients in IMS-3. The median age and presenting NIHSS was fairly similar (65-66 vs 68-69 and 17-18 vs 16-17 respectively) (12,15). Even the variation in stroke location was similar with the large majority of the clots located in the M1 segment of the middle cerebral artery (MCA), followed by a third found in the carotid artery terminus and a small minority found in the M2 segment of the MCA.

A few final thoughts of interest, the authors measured change in NIHSS at 24-hour and 1-week intervals. It will be interesting if these findings are expanded upon in the published document, but as far as I can tell from the data presented at the conference, the difference in NIHSS scores between the groups was 2.3 points at 24 hours and 2.9 points at 1 week. I cannot tell if this difference reached statistical significance but seemingly it is under the threshold of a 4-point improvement on the NIHSS that was deemed clinically relevant by the authors of NINDS in their original publication(1). If this data does prove to be accurate than it means that the anecdotal stories of patients rising from the cath lab table shortly after clot removal, was just that, anecdote. Finally it is important to point out that this trial compared endovascular treatment to standard care, which for all intents and purposes was IV tPA (91% of the control group received IV tPA). It is by no means certain that IV tPA provides any added benefit over placebo alone and some skeptics, such as myself, think there is a suggestion of harm. An additional control group, comparing placebo to both IV tPA and endovascular therapy is needed. In the subgroup analysis though endovascular therapy performed better than standard care in patients who received tPA, these benefits were not seen when IV tPA was withheld.

Surely we are left with more uncertainty than when we started this line of investigation. Thankfully there are a number of studies currently underway that may provide us the clarity we require. MR CLEAN is the first trial to demonstrate the potential benefits endovascular therapy may provide, but one trial should not define the standard of care, especially when multiple trials have concluded quite the opposite. The cost and resources needed to create an infrastructure capable of delivering patients to the endovascular suite with the swiftness seen in this cohort would be extraordinary. We should require more than an ambiguous odds ratio, bolstered by further needless statistical adjustments to justify these costs.

5

Comment Here

One further possibility: It was a Dutch trial. Perhaps we should face the reality that European interventional stroke therapy is quite different (in terms of qualification, 24/7 coverage, time-to-treatment, efficacy of treatment) from, well, IMS-3.
I would also add that the “anecdotal reports” are more than that – in MR CLEAN 33% achieved mRS ? 2 – these are the patients that get up and walk…

Hi Felix,
Thanks for the reply. I agree, European stroke therapy may be very different and that in of itself may be the reason why MR ClEAN succeeded where the 3 previous trials failed. We need more data to say for sure. As far as MR. CLEAN demonstrating that 33% of the patients in the treatment group achieved an mRS of less than 2, this was at 90 days. The change in NIHSS at 24 hours was very little casting some doubt on the immediate miraculous affects described by many interventional apologists. Again hard to say for certain just thought it was an interesting finding.

[…] was positive when the three trials that came before were negative is still unclear. As discussed in my previous post (as well as far more elegant posts on emlitofnote.com and stemlynsblog.org) it may be due to better […]