Title:Balancing Type I Error and Power in Linear Mixed Models

Abstract: Linear mixed-effects models have increasingly replaced mixed-model analyses
of variance for statistical inference in factorial psycholinguistic
experiments. The advantages of LMMs over ANOVAs, however, come at a cost:
Setting up an LMM is not as straightforward as running an ANOVA. One simple
option, when numerically possible, is to fit the full variance-covariance
structure of random effects (the maximal model; Barr et al., 2013), presumably
to keep Type I error down to the nominal $\alpha$ in the presence of random
effects. Although it is true that fitting a model with only random intercepts
may lead to higher Type I error, fitting a maximal model also has a cost: it
can lead to a significant loss of power. We demonstrate this with simulations
and suggest that for typical psychological and psycholinguistic data, models
with a random effect structure that is supported by the data have optimal Type
I error and power properties.