Caveat emptor: the combined effects of multiplicity and selective reporting

Trials. 2018 Sep 17;19(1):497. doi: 10.1186/s13063-018-2888-9.

Abstract

Clinical trials and systematic reviews of clinical trials inform healthcare decisions. There is growing concern, however, about results from clinical trials that cannot be reproduced. Reasons for nonreproducibility include that outcomes are defined in multiple ways, results can be obtained using multiple methods of analysis, and trial findings are reported in multiple sources ("multiplicity"). Multiplicity combined with selective reporting can influence dissemination of trial findings and decision-making. In particular, users of evidence might be misled by exposure to selected sources and overly optimistic representations of intervention effects. In this commentary, drawing from our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study and evidence from previous research, we offer practical recommendations to enhance the reproducibility of clinical trials and systematic reviews.

Keywords: Multiplicity; Reproducibility; Selective reporting.

Publication types

  • Letter

MeSH terms

  • Clinical Decision-Making
  • Data Accuracy*
  • Evidence-Based Medicine / methods*
  • Humans
  • Randomized Controlled Trials as Topic / methods*
  • Reproducibility of Results
  • Research Design*
  • Systematic Reviews as Topic*