Dunning-Kruger Effect Between Self-Peer Ratings of Surgical Performance During a MASCAL Event and Pre-Event Assessed Trauma Procedural Capabilities

Pamela B Andreatta; Jigarkumar A Patel; Mark D Buzzelli; Kenneth J Nelson; John Christopher Graybill; Shane D Jensen; Kyle N Remick; Mark W Bowyer; Jennifer M Gurney

doi:10.1097/AS9.0000000000000180

Dunning-Kruger Effect Between Self-Peer Ratings of Surgical Performance During a MASCAL Event and Pre-Event Assessed Trauma Procedural Capabilities

Ann Surg Open. 2022 Jun 29;3(3):e180. doi: 10.1097/AS9.0000000000000180. eCollection 2022 Sep.

Authors

Affiliations

¹ From the Department of Surgery, Uniformed Services University of the Health Science and the Walter Reed National Military Medical Center, Bethesda, MD.
² Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD.
³ Womack Army Medical Center, Fort Bragg, NC.
⁴ Army Trauma Training Center, Ryder Trauma Center, University of Miami, Miami, FL.
⁵ Department of Trauma, San Antonio Military Medical Center, San Antonio, TX.
⁶ Joint Trauma System, DHA Combat Support, San Antonio, TX.
⁷ U.S. Army Institute of Surgical Research, Houston, TX.

Abstract

Objectives: The research question asked to what extent do self-rated performance scores of individual surgeons correspond to assessed procedural performance abilities and to peer ratings of procedural performance during a mass casualty (MASCAL) event?

Background: Self-assessment using performance rating scales is ubiquitous in surgical education as a proxy for direct measurement of competence. The validity and reliability of self-ratings as competency measures are susceptible to cognitive biases such as Dunning-Kruger effects, which describe how individuals over/underestimate their own performance compared to assessments from independent sources. The ability of surgeons to accurately self-assess their procedural performance remains undetermined.

Methods: A purposive sample of military surgeons (N = 13) who collectively cared for trauma patients during a MASCAL event participated in the study. Pre-event performance assessment scores for 32 trauma procedures were compared with post-event self and peer performance ratings using F tests (P < 0.05) and effect sizes (Cohen's d).

Results: There were no significant differences between peer ratings and performance assessment scores. There were significant differences between self-ratings and both peer ratings (P < 0.001) and performance assessment scores (P < 0.001). Effect sizes were very large for self to peer rating comparison (Cohen's d = 2.34) and self to performance assessment comparison (Cohen's d = 2.77).

Conclusions: The outcomes demonstrate that self-ratings were significantly lower than the independently determined assessment scores for each surgeon, revealing a Dunning-Kruger effect for highly skilled individuals underestimating their abilities. These outcomes underscore the limitations of self-assessment for measuring competence.

Keywords: Dunning-Kruger effect; peer-assessment; performance assessment; procedural assessment; self-assessment; self-rating; surgeons; surgical assessment.