Confidence Interval Estimators of the Intraclass Correlation Coefficient in Longitudinal Data with Missingness

Background: Intraclass Correlation Coefficient (ICC) is an important statistic when we analyze correlated data from longitudinal studies, family studies or group randomized trials (GRT). ICC is used to assess consistency of repeated measurements and is also used to measure the amount of unexplained variation in measurements taken from the same cluster. Despite its wide usage there is no clear guidance on what confidence interval estimator is the most appropriate for a given setting. Objective: To evaluate the magnitude of the bias and to characterize the performance of different confidence interval estimators of ICC under the generalized linear mixed model (GLMM) framework when data are missing-at-random (MAR) and missing-not-at-random (MNAR). Methods: We simulated repeated measured data from a normal linear mixed model with a random intercept and with treatment as covariate of interest. We considered a total of 36 scenarios (three missing data, four sample sizes and three true ICC). The sample size varied from 50 to 1000 subjects and the true ICC values were 0.25(low), 0.5 (medium) and 0.75 (high). For MAR, missingness was associated with treatment and for MNAR missingness was associated with the outcome. For each scenario, we simulated 1000 iterations and made comparisons among four estimators (maximum likelihood (ML), restricted maximum likelihood (REML) and Fisher transformation under ML (F-ML) and REML (F-REML)) in terms of relative bias, coverage and width of the 95%confidence interval. Results: In the analysis of the complete data, all ICC estimators showed some bias (a maximum of 6%) when n=50 but the bias approached zero as n increase to 1000. As expected, the magnitude of bias did not change for the MAR scenario. However, the bias increased dramatically for MNAR (up to 42%). While, the bias was relatively higher for high values of true ICC, the estimators based on REML method were less biased than the ML based methods in all scenarios. The95% confidence interval for ML and REML performed better than Fisher transformation which gave more false negatives (coverage almost 1 for all sample sizes) under MAR. However, for MNAR scenario, all methods showed a general trend toward false positives. The coverage was below 90% in most cases with some exceptions for n=50 and dropped down to 0% coverage for the n= 1000. Under MNAR, across different sample sizes all methods performed poorly in estimating the 95% confidence interval especially for high values of ICC. We used real data a study of continuation of treatment beyond hospitalization in patients with substance abuse problems, where GMI (Group Motivational Interviewing) and IHMD (In-Home- Messaging Device) were the two treatment groups. The two main outcomes were standard ethanol content unit and (SEC)and peak SEC that were measured at baseline, at 1 and 3 months after discharge. There is approximately 17% missing data for SEC as well as for peak SEC. We got an REML estimate of .37(0.25, 0.50) and .28 (0.17, 0.44) for the ICC in SEC and peak SEC respectively. Conclusion: As the ICC increases the bias goes down and the estimators have nominal 95% coverage for MAR. But bias was higher and 95% coverage very poor with MNAR especially at higher ICC values. Impact: Given that missing data is common in repeated measures studies (where ICC is typically high) as well as GRTs (where ICC is typically small), the results will be useful to clinicians and researchers.