This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

For the XIV QTLMAS workshop, a dataset for traits with complex genetic architecture
has been simulated and released for analyses by participants. One of the tasks was
to estimate direct genomic values for individuals without phenotypes. The aim of this
paper was to compare results of different approaches used by the participants to calculate
direct genomic values for quantitative trait (QT) and binary trait (BT).

Results

Participants applied 26 approaches for QT and 15 approaches for BT. Accuracy for QT
was between 0.26 and 0.89 for males and between 0.31 and 0.89 for females, and for
BT ranged from 0.27 to 0.85. For QT, percentage of lost response to selection varied
from 8% to 83%, whereas for BT the loss was between 15% and 71%.

Conclusions

Bayesian model averaging methods predicted breeding values slightly better than GBLUP
in a simulated data set. The methods utilizing genomic information performed better
than traditional pedigree based BLUP analyses. Bivariate analyses was slightly advantageous
over single trait for the same method. None of the methods estimated the non-additivity
of QTL affecting the QT, which may be one of the constrains in accuracy observed in
real data.

Background

An idea of genomic selection (GS) has been presented nearly a decade ago [1] and since that time it has been applied to plant [2] and animal breeding [3]. Together with an increased availability of dense marker assays, implementation of
GS in breeding programs has become more popular [4,5] stimulating development of methods to estimate genomic breeding values.

Genetic basis of a phenotypic trait – its genetic architecture - is often complex.
A particular trait may be, for example, controlled by many genes with small effects
or by several major genes. Genes that control one trait may also control other trait(s),
i.e. they are pleiotropic and the traits are genetically correlated. A gene variant
may have an effect when it is inherited from a parent of one sex but not from the
other (i.e. imprinting) or its effect will be present only when several alleles are
in a particular combination (i.e. epistasis or haplotype effect). Genomic selection
opens new opportunities in the analyses of complex traits.

A number of approaches have been developed to obtain direct genomic values (DGV) or
genomic-enhanced breeding values (GEBV) [1,6]. Because number of markers is usually greater than number of genotyped individuals,
predictions of individual genes are based on Bayesian model averaging, penalized regression,
dimension reduction methods and algorithmic machine learning methods. .Several Bayesian
models have been developed to model effects of individual loci. They differ in number
of characteristics, including a prior distribution of number of QTL, their effects
and assumption of homogeneity or heterogeneity of QTL variance (Table 1). Ridge regression (RR) and spatial regression are two types of panelized estimation,
which assume homogenous variance across all markers. Double hierarchical generalized
linear models (DHGLM) estimates marker-specific variances and can be solved by the
iteratively weighted least squares. Partial Least Square Regression (PLSR) is an extension
of the principal component analyses (PCA): the most systematic variations in marker
data are decomposed into a small number of latent variables (principal components).
This method reduces the dimensionality of the problem utilizing existing correlations
between SNP [7]. GBLUP is an alternative, which treats the markers as a source of information on
relatedness among individuals and models the sum of all QTL instead of individual
loci. Some variants of GBLUP use preselected SNP to build relationship matrix for
particular trait [1]. Some authors apply machine learning approaches (boosting, support vector), with
hope that these methods better account for interaction between QTL [8].

For the XIV QTLMAS workshop, a dataset for traits with complex genetic architecture
has been simulated and released for analyses by participants [9]. One of the tasks was to estimate DGV for individuals without phenotypes. The aim
of this paper was to evaluate and compare results of different approaches used by
the participants to calculate DGV.

Methods

Simulated data

Simulated, four-generation pedigree consisted of 3,226 individuals, descended from
20 founders, each mating resulted in 30 offspring. The last generation consisted of
900 young individuals with no progeny and no phenotypes. All 3,226 individuals had
100 Mb long genomes consisting of 5 chromosomes. In total, 37 out of 10,1031 single
nucleotide polymorphism (SNP) markers were assumed to be QTL of which two had major
effects. One of the simulated traits was a quantitative trait (QT) and the other one
was a binary trait (BT). Heritability for QT, due to imprinting, was higher for males
(0.52) than for females (0.39). True breeding values (TBV) for QT were calculated
as a summation of effects of 30 additive QTL, haplotype effects (QTL pairs 31-32 and
33-34) and effects of imprinted QTL (for males only). Heritability for BT was 0.48.
TBV for BT were calculated as a summation of effects of 22 additive QTL. Simulated
pedigree, genome, marker and phenotypic data were made available for analyses. More
detailed description of simulation can be found in [9] and the simulated dataset is available at http://jay.up.poznan.pl/qtlmas2010/dataset.htmlwebcite.

Methods used by participants to estimate genomic breeding values

Eleven groups submitted their estimates of DGV. Participants applied several methods
and often different variants of same method [8,10-17]. In total, they applied 26 approaches for QT and 15 approaches for BT (Table 2 and 3). The QT was analyzed by 11 groups, whereas BT by 6 groups. Ten groups used univariate
models and two groups applied bivariate models. Bayesian models were used by five
groups, machine learning was applied by a single group, and eight groups tested other
methods.

Table 2. The comparison of the applied approaches used by participants for estimation of genomic
breeding value of quantitative trait.

Table 3. The comparison of the applied approaches used by participants for estimation of genomic
breeding value of binary trait.

Comparison criteria

Five criteria were used to compare the applied genomic selection methods: (1) Accuracy
being the Pearson correlation between true breeding values (TBV) and DGV. (2) Bias
of estimates calculated as the linear regression coefficient (TBV = M + b*DGV +E)
(unbiased estimates are expected to have regression coefficient of 1), (3) mean square
difference (MSD) between TBV and DGV, (4) % of shared ID when selecting top 10% (45
males and 45 females) based on DGV vs. TBV, and (5) selection loss from selecting
on DGV instead of TBV as a proportion of response using TBV. Due to a presence of
imprinting, the average genetic values for males and females were different, and therefore,
accuracies and regression coefficients for these groups were calculated separately.

Results

Accuracy

For QT, the accuracy was between 0.26 and 0.89 for males and between 0.31 and 0.89
for females (Table 2). Most of the approaches using Bayesian model averaging methods performed slightly
better (average accuracy 0.68) than other methods. GBLUP yielded an average accuracy
of 0.61 (after exclusion of the least accurate case of GBLUP). Traditional pedigree
BLUP ignoring genomic data was only about half as accurate as the best approaches.
No substantial differences between bivariate and univariate analyses were found. For
BT, accuracy level was, generally, higher than for QT and ranged from 0.27 to 0.85
(Table 3). Similarly to the QT, Bayesian approaches were somewhat superior to other methods.
The highest accuracy was reached by bivariate BayesC approach. Unlike QT, for BT bivariate
analyses were considerably more accurate than univariate ones.

Regression coefficient

For QT, regression coefficients ranged from 0.19 to 9.05 for males and from 0.15 to
7.31 for females (Table 2). Similarly to the previous criterion, the best performing approaches were Bayesian
methods. For BT, regression coefficients ranged from 0.61 for some of GBLUP applications
to 0.77 for bivariate BayesC (Table 3). Again for this trait bivariate analyses appeared to be better than univariate,
which was not the case for QT.

Mean square difference (MSD)

For QT, MSD for most of the approaches ranged from 42 to 63 (Table 2). Higher MSD were observed for one of the machine learning techniques - boosting
– (280.7), one case of GBLUP analyses (87.0) and PLSR (76.4). These approaches were
inferior in comparison to pedigree BLUP that yielded MSD of 66.4-66.8. For BT, MSD
ranged from 0.26 for BayesC bivariate to 1.20 for PLSR. These results indicate that
BayesC bivariate was superior to remaining methods (Table 3).

Shared

For QT, percentage of ID shared between the groups of young individuals selected on
TBV and DGV varied substantially and ranged from 16% (PLSR) to 77% (Table 2). The best three methods were: BayesB (74%-77%), BayesCpi (77%) and TA-BLUP (72%-73%).
For pedigree BLUP only 17% (univariate) or 23% (bivariate) ID were shared with true
top individuals. For BT, similar range of variation of shared ID was observed (from
20% with PLSR to 64% with bivariate BayesC) (Table 3). The best three approaches were: bivariate BayesC (64%), bivariate GBLUP (60%),
and bivariate BayesA (60%). For pedigree BLUP only 30% (univariate) or 28% (bivariate)
ID were shared.

Loss

High percentage of shared ID, generally, was associated with low level of loss in
genetic gain. For QT, percentage of lost response to selection when 10% are selected
based on DGV instead of TBV varied from low (8%) to very high (83%). Pedigree BLUP
resulted in 62% to 79% of loss, whereas approaches using genomic information, in general,
resulted in smaller loss. Most of analyses using BayesB as well as BayesCpi and TA-BLUP
appeared to be superior to other methods and caused only 8% to 9% of loss. Percentage
of loss for most of the GBLUP approaches was close to 20%. For BT, loss in response
to selection was, usually, at higher level. The smallest observed loss was 15% for
bivariate BayesB and the biggest was 71% for PLSR. Pedigree BLUP caused from 74% to
79% of loss. Bivariate analyses were superior to univariate for both traits.

Discussion

When phenotypes for young individuals are not available, the approaches that use genomic
information had superior performance compared to the methods that were based solely
on pedigree information. Use of genomic information, therefore, led to improved breeding
value estimation, which was also found by others [1,18-24]. The traits simulated for the XIV QTL-MAS workshop differed with respect to complexity
and a number of QTL. Effects of simulated QTL were unequal, some QTL had large effects
and most of the other QTL had small effects on the simulated traits. All Bayesian
model averaging methods had similar accuracy. These models were, furthermore, expected
to achieve higher accuracy than GBLUP because of relatively small number of QTL [25]. GBLUP, however, was expected to capitalize on genetic relationships between training
and validation sets [26]. We have found that these two groups of methods yielded similar accuracies. Bastiaansen
et al. [23] who analyzed results of the previous QTLMAS workshop also reached a similar conclusion.
Lack of apparent advantage in terms of accuracy of a single method across a range
of traits was also shown in other simulation studies [27] and in real data, e.g. [3] and [28].

When the same approach is used by different researchers, similar results are expected,
which was not always the case in our comparison. One GBLUP implementation, for example,
was about 0.20 less accurate than other GBLUP analyses. This suggests that the methods
may be very sensitive to data preparation and that their implementations may vary
in performance.

Bivariate analyses, in general, performed better than univariate analyses for the
same approach. This was expected as the two simulated traits were indeed genetically
correlated. Differences between univariate and bivariate analyses were especially
apparent for BT, for which phenotypes carry less information. More complex approaches,
requiring initial estimation of marker effects or use of machine learning techniques
applied to QT provided similar or inferior results to simpler methods.

Conclusions

Bayesian model averaging methods predicted breeding values slightly better than GBLUP
in a simulated data set, where traits had complex genetic architecture (epitasis,
pleiotropy, and imprinting) and were affected by relatively small number of QTL. The
methods utilizing genomic information performed better than traditional pedigree based
BLUP analyses. Bivariate analyses were slightly advantageous over single trait for
the same method. None of the methods estimated the non-additivity of QTL affecting
the QT, which may be one of the constrains in accuracy observed in real data.