Partial misspecification of survey design features sufficed to severely bias estimates of health-related outcomes

Abstract

Objective: Surveys frequently deviate from simple random sampling through the use of unequal probability sampling, stratified sampling, and multistage sampling. This work uses a survey of public health to systematically illustrate the effects of incompletely accounting for strata, clustering, and weights.

Study design and setting: Data analysis was based on the Study of Health in Pomerania (n=4,308, 20-79 years), a two-stage regional survey with high sampling fractions at the first stage. Effects of survey design features comprising weights, stratification, clustering, and finite population correction on point and variance estimates of lifestyle indicators and clinical parameters were assessed.

Results: Misspecifications of the survey design substantially affected both the point estimates of health characteristics and their standard errors (SEs). The strongest bias in SEs concerned the omission of the second sampling stage. Ignoring the sampling design led to minor differences in variance estimates from the complete setup. Weighting predominantly affected point estimates of lifestyle factors.

Conclusion: A partial misspecification of survey design elements may bias variance estimates severely and is sometimes even more harmful compared with completely neglecting design elements. If subgroups are sampled at different rates, weighting is of particular relevance with regard to prevalence estimates of lifestyle indicators.