Objective: To identify any differences in response and completion rates across two versions of a questionnaire, in order to determine the trade-off between a potentially higher response rate (from a short questionnaire) and a greater level of information from each respondent (from a long questionnaire).
Methods: This was a randomised trial to determine whether response rates and/or results differ between questionnaires containing different numbers of choices: a short version capable of estimating main effects only and a longer version capable of estimating two-way interactions, provided certain assumptions hold. Best-worst scaling was the form of discrete choice experimentation used. Data were collected by post and analysed in terms of response rates, completion rates and differences in mean utilities.
Results: Fifty-three percent of individuals approached agreed to take part. From these, the response to the long questionnaire was 83.2% and the short questionnaire was 85.1% (difference 1.9%, 95% CI -7.3, 11.2; p = 0.68). The two versions of the questionnaire provided similar inferences.
Discussion/conclusion: This trial indicates that, in a healthcare setting, for this complexity of questionnaire (i.e. four attributes and the best-worst scaling design), the use of 16 scenarios obtained very similar response rates to those obtained using half this number.