Imaging contract research organizations (iCROs) conduct independent standardized assessments of imaging studies over time, to measure the severity of disease and how it responds to treatment in clinical trials. Quantitative imaging is thus of critical interest to us. We want to provide our clients—pharmaceutical companies—with high-quality data to support decision-making in early drug development and regulatory (e.g., FDA) submission in later phases. Our role gives us perspectives on quantitative imaging and a set of priorities distinct from equipment makers, regulators, pharmaceutical companies, and academic centers, although we share concerns with each of these.

Our clients are cost-conscious. If site compliance with QIBA's scanning recommendations or performing quantitative image analysis incurs extra costs, we must be able to document the value of these changes. iCROs and our clients are keenly interested in QIBA interactions with regulatory authorities and the progress of imaging biomarkers towards qualification for use in late-phase trials. In early development, we must be able to demonstrate to clients that rigorous quantitative imaging can inform "go"/"no-go" decisions more accurately than traditional approaches. Thus, we participate in both QIBA's scientific activities and QIBA's regulatory engagement and must remain flexible in the face of regulatory changes.

iCROs receive scans from numerous sites globally, so we deal with a wider range of acquisition methods and data quality than many other stakeholders. Part of what we do for our clients is to develop site scanning guidelines. To apply QIBA-recommended guidelines in a trial, the acquisition recommendations in QIBA profiles need to be as clear and explicit as possible. We also perform quality checks as we receive data to make sure sites are complying with the guidelines. Therefore, the compliance testing portions being developed for various QIBA profiles are particularly important to us, so that we can define our process for ensuring that measurements achieve the precision claimed in the profile.

For image analysis, iCROs tend either to develop software tools internally or choose one commercial solution for each task and use that for all trials. Because we conduct a large volume of analysis, accuracy is not our only consideration. We must evaluate the efficiency and ease of use of such tools, as well as their robustness in the face of variable data.

In ongoing scientific work on quantitative imaging, iCROs are something of an untapped resource. Our records contain an enormous amount of image data that is owned by our clients, so we cannot use it for scientific work or validation efforts. However, if pharmaceutical companies decide to allow the use of data for additional analyses, iCROs could be valuable partners due to our experience with anonymizing, processing, and storing large volumes of data. General biomarker qualification would require a dataset of a scale currently only achievable through consortia and recruited in the course of ordinary clinical trials. This is exactly the type of data that iCROs possess, and its analysis could be extremely fruitful.

Once a CT, MR or other scanner generates a medical image file in DICOM format, this file is typically viewed on multiple display stations with different viewing software packages [See Figure 1].

Figure 1. Data flow for PET/CT DICOM images. The proposed role of the PET/CT digital reference object (DRO) as a reference standard is also shown.

In PET/CT imaging, the standardized uptake value (SUV) is the most common quantitative imaging biomarker for diagnosis, staging, treatment planning, and assessing therapy response. When examining a region of interest (ROI) in a PET image volume, either the maximum or mean SUV voxel value within the ROI is reported. The DICOM standard lists hundreds of modality-specific data fields that are stored as part of the digital medical image files [www.dicom.org]. PET/CT viewing software should be capable of reading and interpreting the fields as needed to ensure the fidelity of the image display and to calculate SUVs correctly.

Based on these concepts, we identified the following knowledge gap: For an identical physical test object, do different PET/CT image display systems produce the same results? This is illustrated in Figure 1 by the proposed insertion of a PET/CT digital reference object (DRO) as a reference standard with known truth.

We created a 3D PET/CT DRO from a set of DICOM PET and CT image files with known voxel values and DICOM data fields. The PET/CT DRO was then viewed on 22 different combinations of medical image viewing workstations and software packages by QIBA FDG-PET/CT Technical Committee members. Interestingly, there was no identical combination of the software, version, or display station base system in the 22 tests. Six circular ROIs were specified for use with the DRO and the maximum, minimum, mean, and standard deviation of the SUV values within each ROI was recorded. The color-coded map [See Figure 2] classifies the reported SUV values as either acceptable, borderline, or outside the acceptable range.

Figure 2. Top: Results from user-reported SUV values from the PET/CT DRO. Each column represents a single site/system and each row corresponds to a metric from one of the six ROIs indicated. Bottom: ROIs overlaid on the primary slice of the PET component of the DRO and an illustration of the threshold definitions for one ROI metric.

Results generally indicate that common metrics (e.g., SUVmax for ROI 1 and SUVmean for ROI 2) are correctly reported. But it is also evident that there are multiple failure modes for relatively basic metrics. In one case, the maximum SUV value was under-reported by 38% when analyzing the single hot voxel (ROI 3). In the checkerboard region, four software packages over-reported the max SUV values by 11% and over-reported the mean SUV values for that region by as much as 100%. Other anomalies included misalignment of the PET and CT images, artifacts appearing on zoomed images, and the inability to see the checkerboard regions on the monitor.

These results illustrate the potential role of the PET/CT DRO to help ensure that SUV values are computed correctly. The DRO has been included as a component of the QIBA FDG-PET/CT Profile [1] and results have been presented at scientific meetings and discussed with vendors.

The authors acknowledge the contributions of David Clunie, MBBS, Dennis Nelson, PhD, and Brian Elston to this project as well as the DRO testing and comments provided by QIBA FDG-PET/CT Technical Committee members.

Larry Pierce, PhD, is a research scientist in the Imaging Research Laboratory of the Department of Radiology at the University of Washington in Seattle. He has concentrated on PET image reconstruction and analysis for several years.

Paul Kinahan, PhD, is a professor of radiology at the University of Washington. He is chair of the American Association of Physicists in Medicine (AAPM)/SNMMI Task Group on Quantitative PET/CT Imaging and participates in SNMMI, AAPM, and RSNA initiatives on quantitative medical imaging as a biomarker.

More than 100 people attended the QIBA Working Meeting at RSNA 2013 featuring presentations and discussions on how the value of imaging is being defined in radiology and how the value of quantitative imaging can be estimated before and after implementation. The meeting also provided opportunities for Technical Committee members to further develop groundwork projects to identify technology gaps in supporting development of their respective Profiles.

QIBA ACTIVITIES

The ongoing work of the Technical Committees is posted on the QIBA wiki page: http://qibawiki.rsna.org. New participants in QIBA Technical Committees are always welcome; please contact QIBA@rsna.org for more information.

Articles that are generated by Quantitative Imaging Biomarkers Alliance (QIBA) research teams

Articles that reference QIBA

These are articles published by QIBA members, or ones that relate to a research project undertaken by QIBA members that may have received special recognition. New submissions are welcome and may be directed to QIBA@rsna.org.