Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education

Claudio Violato; Michael J Cullen; Robert Englander; Katherine E Murray; Patricia M Hobday; Emily Borman-Shoap; Ozge Ersan

doi:10.1097/ACM.0000000000004090

Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education

Acad Med. 2021 Jul 1;96(7S):S70-S75. doi: 10.1097/ACM.0000000000004090.

Authors

Claudio Violato¹, Michael J Cullen², Robert Englander³, Katherine E Murray⁴, Patricia M Hobday⁵, Emily Borman-Shoap⁶, Ozge Ersan⁷

Affiliations

¹ C. Violato is assistant dean for assessment and evaluation and professor of internal medicine, University of Minnesota Medical School, Minneapolis, Minnesota.
² M.J. Cullen is director of evaluation, Graduate Medical Education, University of Minnesota Medical School, Minneapolis, Minnesota.
³ R. Englander is associate dean for undergraduate medical education and professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁴ K.E. Murray is assistant dean for curriculum and assistant professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁵ P.M. Hobday is director, Education in Pediatrics Across the Continuum pilot and assistant professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁶ E. Borman-Shoap is vice chair of education, director, pediatric residency program, and associate professor of pediatrics, University of Minnesota Medical School, Minneapolis, Minnesota.
⁷ O. Ersan is a PhD candidate, Department of Educational Psychology, University of Minnesota, Minneapolis, Minnesota.

PMID: 34183605
DOI: 10.1097/ACM.0000000000004090

Abstract

Purpose: To explore validity evidence for the use of entrustable professional activities (EPAs) as an assessment framework in medical education.

Method: Formative assessments on the 13 Core EPAs for entering residency were collected for 4 cohorts of students over a 9- to 12-month longitudinal integrated clerkship as part of the Education in Pediatrics Across the Continuum pilot at the University of Minnesota Medical School. The students requested assessments from clinical supervisors based on direct observation while engaging in patient care together. Based on each observation, the faculty member rated the student on a 9-point scale corresponding to levels of supervision required. Six EPAs were included in the present analyses. Student ratings were depicted as curves describing their performance over time; regression models were employed to fit the curves. The unit of analyses for the learning curves was observations rather than individual students.

Results: (1) Frequent assessments on EPAs provided a developmental picture of competence consistent with the negative exponential learning curve theory; (2) This finding was true across a variety of EPAs and across students; and (3) The time to attain the threshold level of performance on the EPA for entrustment varied by student and EPA.

Conclusions: The results provide validity evidence for an EPA-based program of assessment. Students assessed using multiple observations performing the Core EPAs for entering residency demonstrate classic developmental progression toward the desired level of competence resulting in entrustment decisions. Future work with larger data samples will allow further psychometric analyses of assessment of EPAs.

MeSH terms

Clinical Competence*
Competency-Based Education*
Education, Medical, Undergraduate*
Educational Measurement / methods
Humans
Pediatrics / education*
Reproducibility of Results