Sample size importantly limits the usefulness of instrumental variable methods, depending on instrument strength and level of confounding

Abstract

Objectives: Instrumental variable (IV) analysis is promising for estimation of therapeutic effects from observational data as it can circumvent unmeasured confounding. However, even if IV assumptions hold, IV analyses will not necessarily provide an estimate closer to the true effect than conventional analyses as this depends on the estimates' bias and variance. We investigated how estimates from standard regression (ordinary least squares [OLS]) and IV (two-stage least squares) regression compare on mean squared error (MSE).

Study design: We derived an equation for approximation of the threshold sample size, above which IV estimates have a smaller MSE than OLS estimates. Next, we performed simulations, varying sample size, instrument strength, and level of unmeasured confounding. IV assumptions were fulfilled by design.

Results: Although biased, OLS estimates were closer on average to the true effect than IV estimates at small sample sizes because of their smaller variance. The threshold sample size above which IV analysis outperforms OLS regression depends on instrument strength and strength of unmeasured confounding but will usually be large given the typical moderate instrument strength in medical research.

Conclusion: IV methods are of most value in large studies if considerable unmeasured confounding is likely and a strong and plausible instrument is available.

Keywords: Confounding; Instrumental variable; Observational studies; Simulation study; Therapeutic effects; Variance.