Assessments of Physicians' Electrocardiogram Interpretation Skill: A Systematic Review

David A Cook; So-Young Oh; Martin V Pusic

doi:10.1097/ACM.0000000000004140

Assessments of Physicians' Electrocardiogram Interpretation Skill: A Systematic Review

Acad Med. 2022 Apr 1;97(4):603-615. doi: 10.1097/ACM.0000000000004140.

Authors

David A Cook¹, So-Young Oh², Martin V Pusic³

Affiliations

¹ D.A. Cook is professor of medicine and medical education, director of education science, Office of Applied Scholarship and Education Science, research chair, Mayo Clinic Rochester Multidisciplinary Simulation Center, and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine and Science, Rochester, Minnesota; ORCID: https://orcid.org/0000-0003-2383-4633 .
² S.-Y. Oh is assistant director, Program for Digital Learning, Institute for Innovations in Medical Education, NYU Grossman School of Medicine, NYU Langone Health, New York, New York; ORCID: https://orcid.org/0000-0002-4640-3695 .
³ M.V. Pusic is associate professor of emergency medicine and pediatrics, Department of Emergency Medicine, NYU Grossman School of Medicine, New York, New York; ORCID: https://orcid.org/0000-0001-5236-6598 .

PMID: 33913438
DOI: 10.1097/ACM.0000000000004140

Abstract

Purpose: To identify features of instruments, test procedures, study design, and validity evidence in published studies of electrocardiogram (ECG) skill assessments.

Method: The authors conducted a systematic review, searching MEDLINE, Embase, Cochrane CENTRAL, PsycINFO, CINAHL, ERIC, and Web of Science databases in February 2020 for studies that assessed the ECG interpretation skill of physicians or medical students. Two authors independently screened articles for inclusion and extracted information on test features, study design, risk of bias, and validity evidence.

Results: The authors found 85 eligible studies. Participants included medical students (42 studies), postgraduate physicians (48 studies), and practicing physicians (13 studies). ECG selection criteria were infrequently reported: 25 studies (29%) selected single-diagnosis or straightforward ECGs; 5 (6%) selected complex cases. ECGs were selected by generalists (15 studies [18%]), cardiologists (10 studies [12%]), or unspecified experts (4 studies [5%]). The median number of ECGs per test was 10. The scoring rubric was defined by 2 or more experts in 32 studies (38%), by 1 expert in 5 (6%), and using clinical data in 5 (6%). Scoring was performed by a human rater in 34 studies (40%) and by computer in 7 (8%). Study methods were appraised as low risk of selection bias in 16 studies (19%), participant flow bias in 59 (69%), instrument conduct and scoring bias in 20 (24%), and applicability problems in 56 (66%). Evidence of test score validity was reported infrequently, namely evidence of content (39 studies [46%]), internal structure (11 [13%]), relations with other variables (10 [12%]), response process (2 [2%]), and consequences (3 [4%]).

Conclusions: ECG interpretation skill assessments consist of idiosyncratic instruments that are too short, composed of items of obscure provenance, with incompletely specified answers, graded by individuals with underreported credentials, yielding scores with limited interpretability. The authors suggest several best practices.

Publication types

Systematic Review
Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

Delivery of Health Care
Electrocardiography
Humans
Physicians*
Research Personnel
Students, Medical*