Bottom Line:
We examined the concordance of compound-induced transcriptional changes using data from several sources: rat liver and rat primary hepatocytes (RPH) from Drug Matrix (DM) and open TG-GATEs (TG), human primary hepatocytes (HPH) from TG, and mouse liver/HepG2 results from the Gene Expression Omnibus (GEO) repository.Co-expression networks performed better than genes or GSA when comparing treatment effects within rat liver and rat vs. mouse liver.We observe that the baseline state of untreated cultured cells relative to untreated rat liver shows striking similarity with toxicant-exposed cells in vivo, indicating that gross systems level perturbation in the underlying networks in culture may contribute to the low concordance.

Affiliation: Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana, United States of America.

ABSTRACTThe effect of drugs, disease and other perturbations on mRNA levels are studied using gene expression microarrays or RNA-seq, with the goal of understanding molecular effects arising from the perturbation. Previous comparisons of reproducibility across laboratories have been limited in scale and focused on a single model. The use of model systems, such as cultured primary cells or cancer cell lines, assumes that mechanistic insights derived from the models would have been observed via in vivo studies. We examined the concordance of compound-induced transcriptional changes using data from several sources: rat liver and rat primary hepatocytes (RPH) from Drug Matrix (DM) and open TG-GATEs (TG), human primary hepatocytes (HPH) from TG, and mouse liver/HepG2 results from the Gene Expression Omnibus (GEO) repository. Gene expression changes for treatments were normalized to controls and analyzed with three methods: 1) gene level for 9071 high expression genes in rat liver, 2) gene set analysis (GSA) using canonical pathways and gene ontology sets, 3) weighted gene co-expression network analysis (WGCNA). Co-expression networks performed better than genes or GSA when comparing treatment effects within rat liver and rat vs. mouse liver. Genes and modules performed similarly at Connectivity Map-style analyses, where success at identifying similar treatments among a collection of reference profiles is the goal. Comparisons between rat liver and RPH, and those between RPH, HPH and HepG2 cells reveal lower concordance for all methods. We observe that the baseline state of untreated cultured cells relative to untreated rat liver shows striking similarity with toxicant-exposed cells in vivo, indicating that gross systems level perturbation in the underlying networks in culture may contribute to the low concordance.

pcbi.1004847.g003: Comparison of in vitro vs. in vivo treatment effects for azathioprine.Transcriptional effects after treatment of rat primary hepatocytes with 4 μM azathioprine for 24 hours were compared to 24 different in vivo azathioprine experiments in rat liver. The low, medium and high in vivo doses are denoted with separate lines, and concordance is assessed vs. time using 6 analysis methods (two concordance metrics and genes / GSA / module analysis). The circled point denotes the most concordant in vivo dose / time condition for each of the 6 analysis methods.

Mentions:
The approach is illustrated for 4 μM azathioprine treatment in RPH 24 hours after drug administration. When using the Pearson metric, the most similar in vivo profile is the high dose at 6 hours for gene and module analysis, vs. the 9 hour time point when using GSA (Fig 3). Using the overlap metric selects a similar in vivo condition for GSA and modules, but the 8 day condition for gene-level analysis. We selected azathioprine as a case where the correlation (using the Pearson metric) oscillates from positive to negative to minimal, a behavior that reflects the dynamic nature of biological response patterns in vivo for acute vs. chronic dosing. This example serves to illustrate a key difference between the metrics: Pearson correlation can be negative, conveying a reversal of states (which has been successfully used for drug repurposing[31]), while percent overlap is limited to a range between 0 and 100 and does not distinguish a situation where no genes overlap between two experiments from one where they overlap strongly but change in different directions.

pcbi.1004847.g003: Comparison of in vitro vs. in vivo treatment effects for azathioprine.Transcriptional effects after treatment of rat primary hepatocytes with 4 μM azathioprine for 24 hours were compared to 24 different in vivo azathioprine experiments in rat liver. The low, medium and high in vivo doses are denoted with separate lines, and concordance is assessed vs. time using 6 analysis methods (two concordance metrics and genes / GSA / module analysis). The circled point denotes the most concordant in vivo dose / time condition for each of the 6 analysis methods.

Mentions:
The approach is illustrated for 4 μM azathioprine treatment in RPH 24 hours after drug administration. When using the Pearson metric, the most similar in vivo profile is the high dose at 6 hours for gene and module analysis, vs. the 9 hour time point when using GSA (Fig 3). Using the overlap metric selects a similar in vivo condition for GSA and modules, but the 8 day condition for gene-level analysis. We selected azathioprine as a case where the correlation (using the Pearson metric) oscillates from positive to negative to minimal, a behavior that reflects the dynamic nature of biological response patterns in vivo for acute vs. chronic dosing. This example serves to illustrate a key difference between the metrics: Pearson correlation can be negative, conveying a reversal of states (which has been successfully used for drug repurposing[31]), while percent overlap is limited to a range between 0 and 100 and does not distinguish a situation where no genes overlap between two experiments from one where they overlap strongly but change in different directions.

Bottom Line:
We examined the concordance of compound-induced transcriptional changes using data from several sources: rat liver and rat primary hepatocytes (RPH) from Drug Matrix (DM) and open TG-GATEs (TG), human primary hepatocytes (HPH) from TG, and mouse liver/HepG2 results from the Gene Expression Omnibus (GEO) repository.Co-expression networks performed better than genes or GSA when comparing treatment effects within rat liver and rat vs. mouse liver.We observe that the baseline state of untreated cultured cells relative to untreated rat liver shows striking similarity with toxicant-exposed cells in vivo, indicating that gross systems level perturbation in the underlying networks in culture may contribute to the low concordance.

Affiliation:
Lilly Research Laboratories, Eli Lilly and Company, Indianapolis, Indiana, United States of America.

ABSTRACTThe effect of drugs, disease and other perturbations on mRNA levels are studied using gene expression microarrays or RNA-seq, with the goal of understanding molecular effects arising from the perturbation. Previous comparisons of reproducibility across laboratories have been limited in scale and focused on a single model. The use of model systems, such as cultured primary cells or cancer cell lines, assumes that mechanistic insights derived from the models would have been observed via in vivo studies. We examined the concordance of compound-induced transcriptional changes using data from several sources: rat liver and rat primary hepatocytes (RPH) from Drug Matrix (DM) and open TG-GATEs (TG), human primary hepatocytes (HPH) from TG, and mouse liver/HepG2 results from the Gene Expression Omnibus (GEO) repository. Gene expression changes for treatments were normalized to controls and analyzed with three methods: 1) gene level for 9071 high expression genes in rat liver, 2) gene set analysis (GSA) using canonical pathways and gene ontology sets, 3) weighted gene co-expression network analysis (WGCNA). Co-expression networks performed better than genes or GSA when comparing treatment effects within rat liver and rat vs. mouse liver. Genes and modules performed similarly at Connectivity Map-style analyses, where success at identifying similar treatments among a collection of reference profiles is the goal. Comparisons between rat liver and RPH, and those between RPH, HPH and HepG2 cells reveal lower concordance for all methods. We observe that the baseline state of untreated cultured cells relative to untreated rat liver shows striking similarity with toxicant-exposed cells in vivo, indicating that gross systems level perturbation in the underlying networks in culture may contribute to the low concordance.