Not proper ROC curves as new tool for the analysis of differentially expressed genes in microarray experiments.

Abstract

Most microarray experiments are carried out with the purpose of identifying genes whose expression varies in relation with specific conditions or in response to environmental stimuli. In such studies, genes showing similar mean expression values between two or more groups are considered as not differentially expressed, even if hidden subclasses with different expression values may exist. In this paper we propose a new method for identifying differentially expressed genes, based on the area between the ROC curve and the rising diagonal (ABCR). ABCR represents a more general approach than the standard area under the ROC curve (AUC), because it can identify both proper (i.e., concave) and not proper ROC curves (NPRC). In particular, NPRC may correspond to those genes that tend to escape standard selection methods.

RESULTS:

We assessed the performance of our method using data from a publicly available database of 4026 genes, including 14 normal B cell samples (NBC) and 20 heterogeneous lymphomas (namely: 9 follicular lymphomas and 11 chronic lymphocytic leukemias). Moreover, NBC also included two sub-classes, i.e., 6 heavily stimulated and 8 slightly or not stimulated samples. We identified 1607 differentially expressed genes with an estimated False Discovery Rate of 15%. Among them, 16 corresponded to NPRC and all escaped standard selection procedures based on AUC and t statistics. Moreover, a simple inspection to the shape of such plots allowed to identify the two subclasses in either one class in 13 cases (81%).

TNRC value for the 1607 top genes selected by ABCR at FDR = 15%, as a function of AUC (Panel A) and t statistics (Panel B). Area I includes genes corresponding to not proper ROC curves (blue circles); Area II includes genes under-expressed in malignant cells (green circles); Area III includes genes over-expressed in malignant cells (red circles); empty circles correspond to unselected genes. Solid lines represent the thresholds corresponding to p = 0.05 for TNRC (horizontal line in Panel A and in Panel B), for AUC (vertical lines in Panel A) and for t statistic (vertical lines in Panel B). Broken lines represent the expected value under the null hypothesis for AUC (Panel A) and for t statistics (Panel B).

Mean and variance estimates for ABCR and TNRC under the null hypothesis as a function of the number of samples in each class (equal sample size). Each estimate was obtained from 104 random permutations.