Wolters Kluwer Health
may email you for journal alerts and information, but is committed
to maintaining your privacy and will not share your personal information without
your express consent. For more information, please refer to our Privacy Policy.

Objective The aim of this study was to evaluate the reproducibility of computed tomography (CT)–derived fractional flow reserve (FFR) determined on site by inexperienced observers using a postprocessing software based on structural and fluid analysis.

Methods Using 21 coronary vessels in 7 patients who underwent 320-row coronary CT angiography and catheter-FFR, 2 independent inexperienced observers (A: a student radiation technologist; B: a nonmedical staff) determined the CT-FFR using a postprocessing software. After a 20-minute training session, both observers postprocessed all vessels and readjusted their settings after another training/feedback. These CT-FFRs were compared with values determined by an expert analyst.

Results The mean processing times were 23 ± 4 minutes (automatic), 71 ± 5 minutes (observer A), and 57 ± 7 minutes (observer B) per patient. The initial correlations with expert data were r = 0.92 (observer A) and 0.73 (observer B) and increased to 0.83 for observer B after additional training. The final absolute difference with the expert data was 0.000 to 0.020. The correlation between catheter-FFR and expert CT-FFR was r = 0.76.

N.Y. is an employee of Toshiba Medical Systems Corporation. All data were entirely under the control of the corresponding author, and the role of N.Y. was technical support for the computed tomography postprocessing. H.D. has a research agreement with Toshiba Medical Systems Corporation. The other authors declare no conflict of interest.

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Although 20% of coronary artery lesions with greater than 70% stenosis do not cause myocardial ischemia, coronary flow may be functionally limited in segments with stenosis of 40%.1 Therefore, it is important to determine the pathophysiologic significance of the stenosis when planning revascularization. Invasive fractional flow reserve (FFR) measured by coronary catheter angiography is the current standard procedure for assessing pathophysiologic cardiac ischemia.2 The FFR represents the ratio of distal coronary pressure to aortic pressure, and revascularization is not recommended if the FFR is greater than 0.8.3,4 Invasive FFR has been incorporated into standard clinical guidelines for evaluating the necessity of revascularizing vessels with intermediate stenosis.5,6

FFRCT, developed by HeartFlow, Inc (Redwood City, Calif), is a computed tomography (CT)–based simulation of FFR by applying the principles of computational fluid dynamics.7,8 FFRCT has high diagnostic accuracy for the detection of physiologically significant lesions and is considered to be more cost-effective than catheter angiography.9 An alternative method is CT-FFR, developed by Toshiba Medical Systems Corporation (Tokyo, Japan), which calculates coronary flow and pressures by accounting for structural changes in the coronary artery lumen and aorta during the diastolic phase of the cardiac cycle.10 The phantom study demonstrated its feasibility,10 and a recent multicenter study showed a higher diagnostic accuracy of CT-FFR compared with the CT angiography alone, although not statistically significant, for the detection of significant lesions defined by the invasive FFR.11 A benefit of CT-FFR is its timely calculation using an on-site workstation; by contrast, FFRCT currently requires off-site image transfer and a turnaround time of 24 hours.12 The earlier multicenter trial showed good reproducibility and agreement of CT-FFR values between 2 experienced postprocessing analysts.11 However, the person calculating the CT-FFR will not necessarily be experienced in postprocessing analysis, especially once the software is commercially available.

Therefore, the purpose of this study was to test whether CT-FFR values determined by software using structural and fluid analysis are reproducible, even if the postprocessing is performed by an observer without sufficient experience in CT postprocessing.

METHODS

Study Population

The study was approved by the institutional human research ethics committee, and all participants gave written informed consent. We enrolled 7 consecutive symptomatic patients who underwent clinically indicated coronary CT angiography (CCTA) using a CT-FFR protocol and invasive FFR within 90 days of CCTA acquisition at a single institution.

CCTA Protocol

Patients underwent cardiac CT using a 320-detector-row CT scanner (Aquilion ONE ViSION; Toshiba Medical Systems Corporation). The CT protocol consisted of coronary artery calcium scoring followed by CCTA. All patients received sublingual nitroglycerine, and patients with a prescan heart rate of 60 beats/min or greater were given 20 to 40 mg of metoprolol (Lopressor; Tanabe Seiyaku, Osaka, Japan) orally, and if the heart rate remained 61 beats/min or greater after 1 hour, landiolol (Core-beta; Ono Pharmaceutical, Osaka, Japan) (0.125 mg/kg) was intravenously administered.

Images for coronary artery calcium scoring were obtained with a tube voltage of 120 kV, tube current of 150 mA, and slice thickness of 3 mm. Coronary artery calcium scoring was calculated using the standard Agatston method.13

The CCTA scanning parameters were tube voltage of 100 kV (except for 1 patient whose body mass index was >30 kg/m2, who was scanned at 120 kV), detector collimation of 320 × 0.5 mm, gantry rotation time of 275 milliseconds, and a tube current of 570 to 590 mA, which was calculated by an automatic exposure control with an SD of 19. The craniocaudal range was selected from 200 rows (10 cm) to 320 rows (16 cm) to include the entire coronary tree. The contrast agent iohexol (Omnipaque 350 mg I/mL; Daiichi Sankyo Company, Tokyo, Japan) was injected for 12 seconds at 18 mg I/kg per second (Dual Shot GX 7; Nemoto Kyorindo Co, Ltd, Tokyo, Japan) followed by 30 mL of saline at the same injection rate. Using the bolus-tracking technique, scanning was started when the CT attenuation value reached 300 Hounsfield units in the ascending aorta. A single-heartbeat scan with prospective electrocardiogram gating (model 7800; Chronos Medical Devices Inc, Tokyo, Japan) was performed and covered 70% to 99% of the R-R interval.

The effective radiation dose was calculated by multiplying the dose-length product by 0.014 mSv/mGy per centimeter.14

CT-FFR Analysis

All 7 CCTA data sets were reconstructed at a slice thickness of 0.5 mm with an overlap of 0.25 mm using an iterative (AIDR3D [adaptive iterative dose reduction in 3 dimensions]) algorithm15 with an SD of 20 (standard) and a standard cardiac reconstruction kernel (FC03). These data sets were anonymized and transferred to an image postprocessing workstation (Vitrea version V7.2; Vital Images Inc, Minnetonka, Minn) with a dedicated software (research version) to calculate CT-FFR. Calculation of CT-FFR using the structural and fluid analysis software involves 7 steps: (1) the CCTA data set is retrieved for postprocessing; (2) the best phase is selected from the multiple phases available (eg, at 70%, 80%, 90%, and 99% of the R-R interval); (3) the 3 major coronary arteries are selected; (4) the centerline is determined automatically and adjusted manually as required for each vessel; (5) the luminal contour is determined automatically and adjusted manually as required for each vessel (manual adjustment of the centerline and contour is performed using multiplanar reconstruction images and images orthogonal to the centerline of the vessel); (6) the adjusted luminal contour is applied to other cardiac phases; and (7) the CT-FFR is calculated using the structural and fluid analysis software. The structural and fluid analysis is described in more detail elsewhere.10,11

In this study, postprocessing was done by 2 inexperienced observers. Observer A (S.Y.) was a student medical radiation technologist with some knowledge of CT postprocessing, but who had never been involved in clinical CT imaging. Observer B (K.R.) was a nonmedical staff member with limited knowledge of the coronary arteries and who had never performed CT postprocessing. Both observers plus a radiologist (K.K.K.) with 10 years' experience in CT postprocessing received a basic 20-minute training session. Using a handout describing how to use the software (covering steps 1–7), an experienced postprocessing technician (N.Y.) from the software developer demonstrated the flow of the postprocessing, especially how to manually adjust the centerline and luminal contour. The technician emphasized that special caution is needed for segments with severe calcification to avoid including calcification in the lumen, although this is a universal caution for postprocessing of vascular CT.

Three different postprocessing methods were applied for each case. Method 1 involved fully automated calculation of CT-FFR by the software without manual adjustment. Method 2 involved manual adjustment of the centerline of the vessel, as required, by the independent observers A and B. Method 3-i involved manual adjustment of the centerline and vessel contours as required, by observers A and B, independently. After methods 2 and 3-i were completed, the radiologist (K.K.K.) reviewed the data processed by observers A and B and gave them each another 20-minute training session, independently, during which the radiologist highlighted the segments with imperfect centerline or contour adjustment. After the additional training, both observers independently revised the centerline and contour adjustments (method 3-ii). As the reference standard CT-FFR, the expert analyst from the company (N.Y.), who was blinded to the observers' analyses, postprocessed the anonymized CT data sets for all 7 patients using the same software, with manual adjustment of the centerline and contour as required. Observers A and B and the radiologist were all blinded to these expert data when they performed the analyses.

The CT-FFR was computed for the proximal, middle, and distal segments of each vessel, yielding a total of 63 data points (7 patients × 3 vessels × 3 segments) in each method (ie, methods 1, 2, 3-i, and 3-ii and reference expert data). The locations of the CT-FFR measurements in each method were identified using anatomical landmarks such as calcium deposits, side branches, and/or stents.

CCTA Stenosis Evaluation

The severity of stenosis on CCTA was interpreted using a dedicated clinical workstation (Zio M900; Ziosoft Inc, Tokyo, Japan) by consensus between 2 experienced CT imagers (S.F. and Y. Kawaguchi), who were blinded to the CT-FFR data. The severity of stenosis was evaluated using the 18-coronary-segment model.16 A vessel was considered to have significant stenosis if 1 or more segment was considered nonevaluable or if luminal stenosis was 50% or greater.

Invasive Coronary Angiography, FFR

Coronary angiography was performed for all 7 patients using 5F to 7F guide catheters without side holes by a femoral or radial approach. For a total of 9 vessels in 7 patients, FFR was measured. Pressure measurements were performed using a 0.014-inch pressure guide wire (Verrata Pressure Guide Wire; Volcano Corp, San Diego, Calif). The pressure wire was calibrated and equalized with the aortic pressure before being placed distal to the stenosis and in the distal third of the coronary artery being interrogated. Fractional flow reserve was calculated as the mean distal coronary pressure (Pd) divided by the mean aortic pressure (Pa) during maximal hyperemia (ATP was administered at 140 μg/kg per minute for at least 2 minutes through a large forearm vein using an infusion pump until the heart rate began increasing and the Pd/Pa ratio remained steady).

Statistical Analysis

Continuous variables are presented as the mean ± SD. Categorical variables are displayed as percentages.

The study outcomes measured were the levels of agreements (correlation coefficients and mean absolute differences) between observers and between each method, evaluated by using linear regression models and Bland-Altman analysis. The bootstrap method was used to account for within-individual correlations. We also evaluated the correlation between CT-FFR expert data and invasive FFR at respective points; the corresponding locations on CCTA were identified using anatomic landmarks, such as calcium deposits or side branches that were determined from the fluoroscopy images captured during invasive FFR.

Statistical analysis was performed with STATA version 14.1 (STATA Corp, College Station, Tex). P < 0.05 was considered statistically significant.

RESULTS

All 7 cardiac CT studies were performed without any complication. The baseline patient and imaging characteristics are listed in Tables 1 and 2. Coronary artery calcium scoring was not calculated for 1 patient with a history of percutaneous intervention. The mean calcium score was 646.0 ± 550.4 in the remaining 6 patients. Coronary CT angiography revealed significant stenosis in 6 segments in 6 vessels (5 left anterior descending arteries, 1 left circumflex artery) among 7 patients.

Methods 2 and 3: Analysis With Manual Adjustment

The mean analysis times, which included automatic calculation plus manual adjustment, for methods 2 (centerline adjustment) and 3-i (centerline and contour adjustments) were 43 ± 6 and 71 ± 5 minutes, respectively, for observer A, and were 42 ± 5 and 57 ± 7 minutes, respectively, for observer B. For observer A, the correlation coefficients for methods 2 and 3-i, ii with the expert data were 0.88 and 0.92, respectively (Table 3, Figs. 2A–D), and were greater than the correlation between method 1 and the expert data (r = 0.81). The correlation coefficients decreased to 0.59 and 0.73 for methods 2 and 3-i with manual adjustment by observer B (Table 3, Figs. 2E, F). However, after observer B completed the additional 20-minute training and feedback session, the correlation coefficient between method 3-ii and the expert data increased to 0.83 (Fig. 2G). For method 3-ii, the mean difference with the expert data was 0.000 for observer A and 0.020 for observer B (Table 4, Figs. 2D, H). A representative case showing good reproducibility is shown in Figure 3. Finally, in vessels with limited calcification (calcium score of a single vessel <100, number of segments = 24), the correlation coefficient between automatic calculation without any manual adjustment (ie, method 1) and expert data was 0.97.

Comparison of Invasive FFR Versus CT-FFR Expert Data

Among 9 vessels (7 left anterior descending arteries, 2 left circumflex arteries) in 7 patients in which invasive FFR was evaluated, the correlation coefficient between invasive FFR and expert CT-FFR was r = 0.76.

DISCUSSION

On-site calculation of CT-FFR is expected to be beneficial in terms of the timeliness of decisions on the treatment and management of coronary artery disease. The current study showed good reproducibility of CT-FFR values calculated using the novel software based on structural and fluid analysis, even when the postprocessing was performed on site by inexperienced observers given a brief training session. Experienced technicians who engage solely in CT image postprocessing are not always available in clinical institutions. Thus, it is reasonable to train other medical staff to use the software. However, if the training requires several hours before these staff can reliably determine CT-FFR, timely on-site analysis would not be realistic. The current study suggests that a 20-minute training session followed by a 20-minute training and feedback session delivered by someone with experience in CT postprocessing is sufficient to train inexperienced people to use the software and to compute CT-FFR values in a reproducible manner.

Like 3-dimensional computational fluid dynamics–based techniques,7,17 determination of the patient's coronary anatomy is one of the most important steps in computing reliable CT-derived FFR values. Automatic detection of the centerline and contours by the CT-FFR software is based mainly on the difference in the CT values between the lumen and wall and takes into account the direction and curvature of the vessel. Interestingly, the correlation between the expert data and data obtained by observer B with manual adjustment of the centerline and contour for the first time in method 3-i (ie, just after the first training session) was lower than the correlation between the expert data and the fully automated values (method 1). The main reason for the lower performance was incorrect identification of the vessel's centerline, which resulted in significantly different vessel lengths and thus incorrect CT-FFR values. During the second training session, observer B was given feedback on this point, and the observer readjusted the centerline, which improved the correlation coefficient with the expert data to 0.83. The data obtained by observer A, who already had fundamental knowledge of CT acquisition and the coronary arterial tree, showed a high correlation (r = 0.92) with the expert data after the first training session. The final mean absolute differences between the expert data and observers A and B were 0.000 and 0.020, respectively, and these values are comparable to the variability of FFRCT and invasive FFR.18,19 The invasive FFR values show some variation owing to measurement error and biological factors, such as the heart rate, blood pressure, local neurohumoral responses, and responses to adenosine.20

The current data suggest that automatic segmentation works quite well, and minimal manual adjustment is necessary in vessels with limited calcification. However, more attention is needed in vessels with diffuse calcification to avoid including luminal calcification in analysis. We propose that, when the software is commercially distributed, an institute's staff should receive a training session from the developer prior to the software being implemented into the institution's clinical workflow. The training session should include how to correctly identify the vessel's centerline and how to define the lumen in calcified segments. Such training is especially important for the staff who are likely to perform the analyses but are unfamiliar with CT postprocessing. Moreover, the first few cases of CT-FFR determination should be reviewed by a person with experience in CT postprocessing to check whether the manual adjustment is reasonable. In this context, the reviewer does not necessarily need to be a technician from the software developer, because this guidance is based on routine vessel segmentation on CT images.21–23 If a person who manipulates the software is a CT technician or a radiologist with good experience in CT postprocessing, brief training on how to use the software should be sufficient to allow the person to determine CT-FFR values reliably.

In the current study, the mean analysis time was approximately 1 hour, which included manual adjustment of the centerlines and contours. This is longer than the previously reported value of 30 minutes, when CT-FFR was performed by experienced technologists.11 The longer analysis time in our study is mainly because the observers were unfamiliar with the software. We expect the analysis time to decrease with accumulating experience of the procedures. In fact, the mean analysis time decreased substantially between the first 5 cases and the last 5 cases, decreasing from 72 ± 6 minutes to 46 ± 5 minutes (36% reduction) for observer A and from 57 ± 5 minutes to 49 ± 2 minutes (14% reduction) for observer B.

Although the purpose of the study was to evaluate the reproducibility between an expert and inexperienced people, not the “validity” of the technique, we included invasive FFR data in the analysis as the reference standard method. The correlation coefficient was better than the previous data (r = 0.57).11 Severe calcification led to false-positive CT-FFR, in keeping with the results of studies of other CT-FFR algorithms,11,12,17 probably due to overestimation of the stenosis. As the high reproducibility was suggested in the current study, correction of the CT-FFR calculation at the calcified segment should be considered as a next step.

The strength of the current study is that we are the first to evaluate the reproducibility of CT-FFR determined using software based on a novel structural and fluid analysis method, and the analyses were performed by inexperienced observers simulating clinical practice after commercial distribution of the software. However, this study was a single-center study with a small number of subjects and vessels, and thus detailed subgroup analysis was not possible. Although patient population was small, it should be noted that a total of 63 data points were repeatedly evaluated.

CONCLUSIONS

The automated CT-FFR measurement software using structural and fluid analysis displayed good reproducibility even when the postprocessing was performed by inexperienced observers given brief training. Minimal manual adjustment may be required for vessels with limited calcification.