How to understand and teach P values: a diagnostic test framework

Abstract

Objectives: The aim of the tutorial is to help educators address misconceptions about P values and provide a tool that can be used to teach a more contemporary interpretation.

Study design and setting: A scripted tutorial using problem-based learning and a diagnostic test analogy to deconstruct the misunderstandings about P values and develop a more Bayesian approach to study interpretation.

Results: A diagnostic test analogy is an effective teaching tool. Learners' understanding of Bayes' theorem in diagnostic testing can be used as a bridge to the realization that the prestudy probability of a true difference is crucial for study interpretation. The analogy has several caveats and shortcomings. The limitations of this analogy and the conceptual difficulties with the Bayesian study analyses are addressed.

Conclusion: P values do not provide the information many assume they do-they are not equivalent to a probability of a chance finding. This tutorial helps move learners from these incorrect notions to new insights.

Keywords: Bayes' theorem; Evidence-based medicine; Hypothesis testing; Medical education; P-values; Problem-based learning.