Trial arm outcome variance difference after dropout as an indicator of missing-not-at-random bias in randomized controlled trials

Abstract

Randomized controlled trials (RCTs) are vulnerable to bias from missing data. When outcomes are missing not at random (MNAR), estimates from complete case analysis (CCA) and multiple imputation (MI) may be biased. There is no statistical test for distinguishing between outcomes missing at random (MAR) and MNAR. Current strategies rely on comparing dropout proportions and covariate distributions, and using auxiliary information to assess the likelihood of dropout being associated with the outcome. We propose using the observed variance difference across trial arms as a tool for assessing the risk of dropout being MNAR in RCTs with continuous outcomes. In an RCT, at randomization, the distributions of all covariates should be equal in the populations randomized to the intervention and control arms. Under the assumption of homogeneous treatment effects and homoskedastic outcome errors, the variance of the outcome will also be equal in the two populations over the course of follow-up. We show that under MAR dropout, the observed outcome variances, conditional on the variables included in the model, are equal across trial arms, whereas MNAR dropout may result in unequal variances. Consequently, unequal observed conditional trial arm variances are an indicator of MNAR dropout and possible bias of the estimated treatment effect. Heterogeneous treatment effects or heteroskedastic outcome errors are another potential cause of observing different outcome variances. We show that for longitudinal data, we can isolate the effect of MNAR outcome-dependent dropout by considering the variance difference at baseline in the same set of patients who are observed at final follow-up. We illustrate our method in simulation for CCA and MI, and in applications using individual-level data and summary data.

Keywords: bias; dropout; missing-not-at-random; randomized controlled trials; risk-of-bias.