Digging Into Pixels: Radiogenomics Extracts Meaning

By Amber Dance

In a radiological image, a tumor’s edges might appear fuzzy or crisp; its shape could range from oval to many-lobed; and its density and texture might vary across the tumor. To determine whether and how those characteristics matter, researchers in the new field of radiogenomics are extracting as much information as possible from every pixel and correlating it with gene expression and cancer survival rates.

The work has the potential to increase our understanding of tumor biology and offer patients personalized medicine based on both imaging and genetics.

The field now seems poised to take off, says Robert Gillies, PhD, chair of cancer imaging and metabolism at the Moffitt Cancer Center in Tampa, Florida. “The whole idea of extracting large amounts of quantitative data from images has been around for a long time, but it’s taken a while for the computational power to catch up with us,” he says.

Shades of Gray

In radiogenomics (also known as radiomics), researchers convert images to mineable data in high throughput, Gillies says. So the radiologist’s “fuzzy edge” becomes a numerical descriptor, reproducible between physicians using the same rating scale or calculated by a computer based on the arrangement of pixels in shades of gray.

Radiomics data come in a few flavors, such as semantic and computational, says Sandy Napel, PhD, professor of radiology at Stanford University. Semantic features are word descriptors, such as round, oval or star-shaped, assigned by a human. Computational features are numerical values calculated from pixel patterns.

For example, one way to describe a tumor’s “texture”—whether it’s mostly uniform or mottled in density—is for computer software to compare the intensity of each pixel with those of its neighbors. The more alike the intensities are, the less textured and more homogeneous the tumor. By changing the parameters of the analysis, a computer can describe texture in numerous different ways, Gillies says.

He’s using imaging features, such as density and contrast, to study the tumor microenvironment. Tumors can have a strong or weak blood supply from the vasculature, which shows up as a high or low perfusion of injected contrast agents on an MRI. And they can be more or less dense, which appears as high or low contrast with surrounding tissues. Gillies wants to classify tumors—in brain, breast, lung and connective tissue—based on which microenvironments are present, and correlate that score with patient data such as treatment and survival.

Eventually, Gillies hopes radiologists will build databases of tens of thousands of patient images. A radiologist could log on with a picture from today’s patient; compare it to images from patients past; and predict, based on those past patients’ clinical histories, the best treatment options.

Images Plus

Beyond simply correlating images and clinical data, radiogenomicists can mix in molecular information, such as a tumor’s genetic mutations or gene expression profile. Normally if physicians want to know about the biochemistry and gene expression in a tumor they need a physical piece of it. But that same information may be buried in the medical images obtained noninvasively—if researchers can figure out how the cancer’s molecular biology influences the pictures they see.

In a 2012 paper in the journal Radiology, Sylvia Plevritis, PhD, associate professor of radiology at Stanford, together with Napel and other colleagues, showed it could be done. The team examined CT scans and gene expression profiles from 26 people with lung cancer. They compared the gene expression data with 180 different image descriptors, both semantic and computational. Because the participants were newly diagnosed, with no data available on survival or relapse, the researchers looked for correlations between gene expression and prognosis in a public database. Combining those two analyses, they found that tumor size, shape and edge sharpness were most strongly linked to gene expression that correlated with prognosis.

For example, tumors that include air-filled structures, called internal air bronchograms, often upregulated a gene called KRAS. And KRAS overexpression, according to public databases, is an indicator that a tumor will likely recur. Other studies have offered conflicting results as to whether internal air bronchogram is a positive or negative sign, so more research is necessary. However, this and other correlations in the paper suggest the potential value of imaging in making prognoses.

Plevritis cautions that this was a small proof-of-principle study. “It just says that we should do more, and we are,” she says. The team has recruited around 75 new lung cancer participants so far, and is also looking into similar studies with breast and liver cancer.

With large enough datasets, computers might pull out tumor features that humans would never notice, Napel says. “The technology is here; it could be implemented today,” he adds. But there are logistical challenges. For one, data acquisition is not standardized across different institutions: Some use different slice thicknesses for 3-D images, or apply different filters. Similarly, no standardized methods exist for how people or computers convert those images into comparable, numerical data.

Other obstacles include time and money. Radiologists’ schedules are already full simply reading scans; they have no time to develop a new way of doing their business, Gillies notes. What’s needed, he says, is a “sandbox” where radiologists can experiment with information technology to build annotated image databases. He estimates the price tag for a single center like that at $17 million—and there would need to be many such centers. Storing and moving the petabytes of data would cost a fortune, Napel adds.

But, Gillies points out, misdiagnosis due to incorrect scan interpretation is also costly. If computers could assist in diagnoses and prognoses, and catch human errors, he predicts the new methods would be “more than cost-effective.” And similar techniques could be useful in radiology and pathology image analysis beyond cancer, Napel says.

“I certainly don’t propose replacing radiologists with computers, but we need to incorporate them as allies,” Gillies says.