Abstract

Objectives The aim of this meta-analysis was to determine the accuracy of transcranial Doppler (TCD) compared with transesophageal echocardiography (TEE) as the reference.

Background Right-to-left shunting (RLS), usually through a patent foramen ovale (PFO), has been associated with migraine, cryptogenic stroke, and hypoxemia. With emerging observational studies and clinical trials on the subject of PFO, there is a need for accurate diagnosis of PFO in patients with these conditions, and those being considered for transcatheter closure. Although a TEE bubble study is the current standard reference for diagnosing PFO, the TCD bubble study may be a preferable alternative test for RLS because of its high sensitivity and specificity, noninvasive nature, and low cost.

Methods A systematic review of Medline, the Cochrane Library, and Embase was done to look for all the prospective studies assessing intracardiac RLS using TCD compared with TEE as the reference; both tests were performed with a contrast agent and a maneuver to provoke RLS in all studies.

Results A total of 27 studies (29 comparisons) with 1,968 patients (mean age 47.8 ± 5.7 years; 51% male) fulfilled the inclusion criteria. The weighted mean sensitivity and specificity for TCD were 97% and 93%, respectively. Likewise, the positive and negative likelihood ratios were 13.51 and 0.04, respectively. When 10 microbubbles was used as the embolic cutoff for a positive TCD study, TCD produced a higher specificity compared with when 1 microbubble was used as the cutoff (p = 0.04); there was, however, no significant change in sensitivity (p = 0.29).

Conclusions TCD is a reliable, noninvasive test with excellent diagnostic accuracies, making it a proficient test for detecting RLS. TCD can be used as a part of the stroke workup and for patients being considered for PFO closure. If knowledge of the precise anatomy is required, then TEE can be obtained before scheduling a patient for transcatheter PFO closure.

Patent foramen ovale (PFO) is a remnant of the fetal circulation that is present in 20% to 25% of the population (1–3). Transient right-to-left shunting (RLS), usually through a PFO, has been implicated in the pathophysiology of stroke, migraine, and hypoxemia (3–6). A meta-analysis of observational studies and a recent meta-analysis of the CLOSURE 1 (Closure or Medical Therapy for Cryptogenic Stroke with Patent Foramen Ovale), RESPECT (Closure of Patent Foramen Ovale Versus Medical Therapy After Cryptogenic Stroke), and PC (Percutaneous Closure of Patent Foramen Ovale in Cryptogenic Embolism) trials suggest that PFO occluding devices reduce the recurrence of stroke and transient ischemic attack at higher rates than conventional medical treatment alone (pooled hazard ratio: 0.59, 95% confidence interval [CI]: 0.36 to 0.97; p = 0.04) (7,8). These data, along with the evaluation of patients with severe migraines or other PFO-associated conditions, make it essential to accurately diagnose RLS in patients being considered for PFO closure.

Whereas contrast transesophageal echocardiography (TEE) is considered the gold standard for diagnosing PFO (9,10), contrast transcranial Doppler (TCD) is increasingly being used for safe, noninvasive, and cost-effective screening of intracardiac RLS (11–37). The aim of this study was to expand on prior reviews of TCD to provide the first meta-analysis that methodically assesses the diagnostic accuracy of TCD in evaluating for an intracardiac RLS.

Methods

Literature review

Relevant citations were searched for on Medline, the Cochrane Library, and Embase. The search was completed in August 2013, yielding literature since 1913. The terms used in the search were “PFO” OR “patent foramen ovale” OR “right to left shunt” OR “atrial septal defect” AND “TCD” OR “transcranial Doppler” OR “TEE” OR “echo” OR “transesophageal echo” OR “transesophageal echocardiogram” OR “transesophageal echocardiography.”

The references of all primary studies as well as those from known reviews were analyzed to find cited studies that were not found by initial searches. No restrictions were used regarding publication language. Abstracts lacking peer-reviewed manuscripts were omitted because they would not have enough data required for the meta-analysis.

Selection of studies

Studies that were identified were analyzed by 3 independent reviewers (M.K.M., S.C.R., and J.S.W.). Each study was screened for pre-set inclusion criteria:

3. Studies were selected if they included at least 20 patients with suspected intracardiac RLS who were screened by TCD and confirmed by TEE as a reference. If a study conducted both TCD and TEE, but did not consider TEE as the gold standard, we calculated the appropriate parameters assuming TEE as the reference comparison.

6. If a study compared different TCD protocols (such as comparing accuracy of different contrast injection sites or different types of contrast) and also provided the variables to calculate the different accuracies (i.e., the TP, FP, FN, and TN), then each methodology was considered a separate comparison in the final analysis. A sensitivity analysis was then conducted to demonstrate the effect of varying methodologies on accuracy of TCD.

Quality assessment

The quality of each study was assessed by evaluating items considered relevant to the review topic, on the basis of the Quality Assessment of Diagnostic Accuracy Studies (version 2) instrument (QUADAS-2) (38). Three reviewers (M.K.M., S.C.R., and J.R.) independently assessed the quality items, and discrepancies were resolved by consensus.

Statistical analysis

Sensitivities and specificities were calculated for every study using a more recently developed bivariate random effects model (39). The bivariate approach assumed that logit transforms of sensitivity and specificity from individual studies are from a bivariate normal distribution. The bivariate approach is considered superior to the standard summary receiver-operating characteristic (ROC) approach (40) because: 1) it assesses heterogeneity across studies, providing a summary estimate of sensitivity and specificity; 2) it models sensitivity and specificity jointly so that a 95% confidence ellipse around the summary estimate can be calculated; 3) it allows calculation of positive and negative likelihood ratios; 4) it allows one to directly compare sensitivity and specificity between methods; and 5) several choices are available to obtain a summary ROC curve (39). In this study, the summary ROC curve was obtained by transforming the regression line of logit sensitivity on logit specificity into ROC space. Publication bias was assessed for each analysis using Deeks's method (41). Post-test probabilities were calculated using Bayes' normogram, which requires converting pre-test probabilities to odds and backtracking post-test odds to probabilities.

We assessed between-study heterogeneity visually, by plotting sensitivity and specificity in the ROC curves (42). The analyses were conducted using STATA 12 (Metandi Syntax, StataCorp LP, College Station, Texas), and the figures were generated using STATA graph editor.

Sensitivity Analysis

We further evaluated whether the performance of each technique depends on features of the technique and patient characteristics. A logistic regression for each technique was used to model the sensitivity of these factors.

Results

Study selection

We identified 174,961 reports, of which 53 studies were considered for detailed evaluation and 27 studies met the inclusion criteria (11–37). Two studies demonstrated 2 different accuracies for TCD by comparing different protocols; we therefore included 29 comparisons in our final analysis consisting of 1,968 patients (mean age 47.8 ± 5.7 years; 51% male).

Twenty-six studies were excluded from the final analysis because they did not meet the inclusion criteria; the excluded studies, along with the reasons for exclusion, are provided in the Online Appendix. Figure 1 describes the study selection method used for this analysis.

Quality assessment

Using the items for evaluating diagnostic studies with QUADAS-2, the risk of bias and applicability concerns for all the studies were assessed (Fig. 3). Most studies had high methodological quality with very minimal concerns regarding applicability of the test in clinical practice (Fig. 3). Uninterpretable results and withdrawals that represented the “flow and timing” section were unclear in 86% (25 of 29) of the comparisons. Data on these 2 parameters are often not reported in diagnostic accuracy studies, with the uninterpretable results and withdrawals simply removed from the analysis. This may lead to a biased assessment of test characteristics. Whether or not bias will arise depends on the possible correlation between uninterpretable test results and the true disease status. Uninterpretable results frequently occur randomly and are unrelated to the true disease status of the individual. Therefore, in theory, these should not have any effect on test performance. Likewise, 3 of 29 comparisons (10%) were not conducted in a blinded fashion; this may have led to review bias in these particular studies. Some did not clearly specify whether blinding occurred in the index or reference tests. A review bias may potentially lead to inflated measures of diagnostic accuracy.

(A) Risk of bias and applicability concerns graph: review authors' judgments about each domain presented as percentages across included studies. (B) Risk of bias and applicability concerns summary: review authors' judgments about each domain for each included study. QUADAS-2 = Quality Assessment of Diagnostic Accuracy Studies version 2.

Transcranial Doppler

A total of 29 comparisons met all inclusion criteria and were used for further meta-analytic calculations. Table 1 describes the characteristics of the included studies, and Table 2 describes the diagnostic accuracies of the studies. The major clinical indication for performing a TCD in most of the studies was stroke followed by migraine headache. Of the 29 comparisons that performed TCD and TEE with contrast, 12 (41%) used agitated saline as the contrast agent, 10 (35%) used Echovist (Schering, Berlin, Germany), 4 (14%) used a gelatin-based solution, and 3 (10%) used 2 different contrast agents. The Valsalva maneuver was used as the provocation method in 86% (25 of 29) of the comparisons, Valsalva with cough was used in 10% (3 of 29), and the provocation method used was unknown in 3% (1 of 29) of the comparisons. The majority of the comparisons used ≥1 microbubble as the embolic cutoff for a positive TCD (62%; 18 of 29) or TEE (52%; 15 of 29).

Forest plots of sensitivity (A) and specificity (B). Size of the square plotting symbol is proportional to the sample size for each study. Horizontal lines are the 95% confidence intervals (CI), and the summary sensitivity and specificity are calculated on the basis of the bivariate approach.

Forest Plots of the Positive and Negative Likelihood Ratio of Each Study

Forest plots of positive (A) and negative (B) likelihood ratios. Size of the square plotting symbol is proportional to the sample size for each study. Horizontal lines are the 95% confidence intervals (CI), and the summary sensitivity and specificity are calculated on the basis of the bivariate approach. DLR = diagnostic likelihood ratio.

On the basis of combined sensitivity and specificity weighted for sample size of each dataset reflected by the size of the open circles, showing average sensitivity and specificity estimate of the study results (solid square) and 95% confidence region around it. The 95% prediction region represents the confidence region for a forecast of the true sensitivity and specificity in a future study. HSROC = hierarchical summary receiver-operating characteristic.

We performed a sensitivity analysis for different contrast agents used, different provocation maneuvers, different microembolic cutoffs for a positive index (TCD) and reference test (TEE), different timings of provocation maneuver, and unilateral versus bilateral middle cerebral artery (MCA) insonation (Table 3). There was no significant difference in sensitivity or specificity when different contrast agents (agitated saline, Echovist, and gelatin-based solutions) were utilized (p > 0.05). However, there was a trend towards Echovist producing a higher sensitivity (95% sensitivity) compared with when gelatin-based solutions (94% sensitivity) were used (p = 0.06). Studies that used Valsalva with cough did not produce a higher sensitivity or specificity compared with studies that only used Valsalva as their provocation maneuver (p > 0.7). When 10 microbubbles was used as the embolic cutoff for a positive TCD study, TCD produced a higher specificity compared with when 1 microbubble was used as the cutoff (p = 0.04); there was, however, no significant change in sensitivity (p = 0.29). There was no significant difference in sensitivity or specificity between studies that used 1 microbubble compared with studies that used 3 microbubbles as the cutoff for a positive TEE (p > 0.1). There were no significant differences in sensitivity or specificity in studies that performed the provocation maneuver for ≤5 s compared with >5 s (p > 0.50). Lastly, there was a trend towards insonation of the unilateral MCA producing a higher specificity (95% specificity) compared with when bilateral MCA insonation (89% specificity) was used (p = 0.09). There was no significant difference in sensitivity between unilateral and bilateral MCA insonation (p = 0.15).

Figure 7 demonstrates the pre- and post-test probabilities of detecting an intracardiac RLS with TCD in the general population, and in our study cohort consisting mainly of patients with stroke or migraine. Because PFO is present in 20% to 25% of the adult population and in approximately 50% to 55% of patients with migraine or cryptogenic stroke (1–3,14), these respective prevalences were assumed to demonstrate the likelihood of detecting a RLS by TCD in the 2 populations.

Pre- and Post-Test Probabilities of Detecting an Intracardiac RLS With Transcranial Doppler

Shown are the probabilities of detecting a right-to-left shunt (RLS) in the general population assuming a 20% prevalence of RLS (A), and in a population of patients with cryptogenic stroke or migraine assuming a 50% (B) to 55% (C) prevalence of RLS. A line from the pre-test probability (Prior Prob) (first axis) through the likelihood ratio (LR) (middle axis) to the post-test probability (last axis) was drawn. A person testing positive with a pre-test probability (prevalence) of 20%, 50%, and 55% and a positive likelihood ratio of 14, will have a post-test probability of 77%, 93%, and 94%, respectively; whereas a person testing negative, with the same pre-test probability and a negative likelihood ratio of 0.04, would determine a post-test probability of 1%, 4%, and 4%, respectively.

With a LR+ of 14 and LR− of 0.04, a TCD performed in the general population consisting of a 20% RLS prevalence will have 77% probability of a positive result being a TP and a 1% probability of a negative result being a FN (Fig. 7A). These probabilities significantly change in a population of patients with stroke or migraine who undergo TCD. With a LR+ of 14 and LR− of 0.04, a TCD performed in patients with migraine or stroke consisting of a 50% to 55%% RLS prevalence will have 93% to 94% probability of a positive result being a TP and a 4% probability of a negative result being a FN (Figs. 7B and 7C).

Discussion

Our study demonstrates that TCD detects intracardiac RLS with a sensitivity of 97% and a specificity of 93% when TEE is used as the reference. TCD has an excellent LR+ of 14 and LR− of 0.04, making it a proficient test to rule out or rule in RLS in the stroke or migraine population (Fig. 7). Increasing the microembolic threshold for a positive TCD from 1 to 10 microbubbles increases the specificity of TCD without compromising sensitivity. This is the first meta-analysis that assesses the accuracy of TCD for detecting intracardiac RLS compared with TEE as the reference. It is also the first meta-analysis that compares different protocols of TCD for detecting intracardiac RLS.

In the evaluation of patients who may have a PFO, several methods are available to determine whether a RLS is present: TCD, transthoracic echocardiography, TEE, or intracardiac echo. Although TEE is considered the gold standard for diagnosing PFO (9,10), studies that compared TEE with autopsy or intraoperative detection of PFO demonstrated that the diagnosis is sometimes missed by TEE (43,44). Studies that compared the accuracy of TEE in the detection of PFO with that of catheterization and/or surgery have demonstrated a sensitivity of 91% to 100% and accuracy of 88% to 97% (45,46). Thus, our results may have underestimated the sensitivity of TCD; Spencer et al. (46) demonstrated a sensitivity of 98% and accuracy of 94% when TCD was compared with PFO detection during catheterization as the reference. In addition, some stroke patients may have dysfunctional swallowing or poor cooperation, making TEE difficult to perform, affecting the test accuracy because of an inadequate Valsalva maneuver. FN TEE tests may also be explained by ineffective Valsalva secondary to sedation or by the presence of the TEE probe in the patient's esophagus (37,47). Thus, the decreased specificity of TCD may reflect some true shunts that are not recognized by TEE.

Currently, TEE is often utilized when routine diagnostics cannot identify a stroke etiology, especially in young patients (48). However, TEE may be uncomfortable and time-consuming for some patients. Although unusual, severe complications such as esophageal bleeding or perforation may occur. Contraindications of TEE such as esophageal varices, Barrett's esophagus, Zenker's diverticulum, esophageal or pharyngeal carcinoma, strictures, Mallory-Weiss tears, or patients with a serious bleeding risk make it important to have a reliable alternative in contemporary clinical practice (49).

TCD is an alternative method for indirectly diagnosing PFO by assessing the presence of a RLS. It employs the functional assessment of the shunt using insonation of at least 1 MCA during a venous injection bubble study and Valsalva maneuver. Although intracardiac shunting through a PFO is usually directed from the left atrium to the right atrium, the release of the Valsalva maneuver allows right atrial pressure to briefly exceed left atrial pressure, resulting in transient reversal of flow. After contrast injection, bubbles enter the systemic circulation during this transient RLS, resulting in microembolic signals in the cerebral arteries that are detected by TCD. TCD utilizes a pulsed Doppler transducer that detects the velocity and intensity of cerebral arterial blood flow by spectral analysis. Although less common, color duplex TCD may also be used in addition to spectral TCD to confirm the positivity of an exam; 4 of the included studies also utilized color TCD (26,29,31,34). TCD provides good patient tolerance and excellent accuracies, making it a useful alternative for detecting RLS in patients with stroke or migraine, or in patients being considered for PFO closure.

It is thought that the accuracy of TCD can vary, depending on the effectiveness of the provocative maneuver, skill of the operator, location of the intravenous needle, positioning of the ultrasound probes, type of contrast, and the TCD model (Power M-mode vs. single-gated TCDs). The Consensus Conference of Venice outlined some key standards for performing a TCD including the use of an 18-gauge needle in the cubital vein, preferable utilization of agitated saline as the contrast, and application of the Valsalva maneuver as the provocation maneuver for more than 10 s (50). Although these guidelines are based on data derived from observational studies before the year 2000, some differences in methodology exist even now, depending on the institution. In addition, the Consensus Conference of Venice did not establish any microbubble number cutoff to define a positive TCD study. According to our sensitivity analysis, using a different contrast agent, provocation maneuver, insonation of unilateral versus bilateral MCAs, and a different microbubble cutoff for a positive shunt with TEE do not significantly affect the accuracy of TCD. However, increasing the TCD threshold for a positive shunt from 1 microbubble to 10 microbubbles significantly increased the specificity of TCD from 89% to 100% (p = 0.04) without affecting sensitivity (from 98% to 97%; p = 0.29). Our data support use of a higher microembolic threshold for a positive shunt by TCD; this finding is supported by previously published literature that demonstrates that using a higher cutoff for a positive TCD, such as with the Spencer Logarithmic Scale, which uses 30 microbubbles as a cutoff instead of 1 microbubble, can decrease the number of FP TCDs that occur because of clinically insignificant shunts (46). TCD has a higher sensitivity than specificity as it is difficult for a TCD to differentiate between pulmonary and intracardiac shunts, which may sometimes be misinterpreted on a TEE as well. TCD does not show the operator the anatomic position of the RLS, nor can it distinguish between a PFO, an atrial septal defect, or a pulmonary arteriovenous fistula. Thus, patients who have a clinical indication for PFO closure and have a positive TCD may be sent to the catheterization laboratory, where a TEE, or alternatively, intracardiac echocardiography can be performed before transcutaneous closure. For a suspected pulmonary arteriovenous fistula, a pulmonary artery injection of echo contrast or a pulmonary angiogram may confirm the diagnosis.

Study limitations

Of 29 included comparisons, 3 (10%) were not conducted in a blinded fashion; this may have led to review bias in these particular studies.

Another limitation of our review was the lack of studies that utilized Power M-mode TCD. The sensitivity of Power M-mode TCD has been demonstrated to be higher than older single-gated TCDs for the diagnosis of intracardiac RLS when catheterization was used as the reference (46).

This study was also limited by the heterogeneity of the included studies. In this meta-analysis, we attempted to perform a sensitivity analysis on different protocols, where possible, to assess the effect of changing the TCD protocol on accuracy of the test. However, Hamann et al. (20) described 2 different accuracies of TCD by comparing contrast injection through the antecubital versus the femoral veins. We were unable to compare antecubital versus femoral injection in this meta-analysis because of the lack of other studies utilizing the femoral injection site. It has been reported that femoral injection of agitated saline produces a higher sensitivity for the detection of RLS using TCD (51).

Conclusions

TCD is a reliable, noninvasive alternative to TEE for the diagnosis of RLS, with an excellent sensitivity and specificity of 97% and 93%, respectively. Increasing the microembolic threshold for a positive TCD from 1 microbubble to 10 microbubbles significantly improves the specificity of TCD without compromising sensitivity. With a LR+ of 14 and LR− of 0.04, TCD is the test of choice for detecting RLS through a PFO in patients with cryptogenic stroke or migraine, having a post-test probability of 93% to 94% when testing positive, and a post-test probability of 4% when testing negative.

(2002) Contrast transcranial Doppler ultrasound in the detection of right-to-left shunts: comparison of Echovist-200 and Echovist-300, timing of the Valsalva maneuver, and general recommendations for the performance of the test. Cerebrovasc Dis13:235–241.