Validation of a Web-Based Curriculum for Resident Education in Orthopedic Surgery

Abstract

Background: The Orthopedic In-Training Examination is an annual standardized examination with multiple-choice questions focused on application of orthopedic surgery core knowledge and principles. The outcome of this test can be used to both predict how residents are progressing in their orthopedic knowledge as well as assess their likelihood to pass the Orthopedic Board Examinations, that is the American Boards of Orthopedic Surgery Part 1, following completion of residency. Preparation for the examination can be difficult, as residents commonly have limited study time to review the vast amounts of available published literature. The objective of our study is to evaluate the effectiveness of the Orthopaedic In Training Examination (OITE) scores and the participants' perceived utility of the curriculum for OITE preparation.

Methods: Residents from 5 US Orthopedic residencies (4 M.D. and 1 D.O.) were included in a pilot program of the Orthobullets PASS curriculum in the academic year 2013 to 2014. Only residents enrolled in the PASS curriculum who completed both the 2013 and 2014 OITEs were included in the final analysis (n = 71). We used the OITE 2013 and 2014 rank postgraduate year (RPGY) reported scores to assess for efficacy of the PASS curriculum, as the RPGY score provides postgraduate year-of-training matched analysis to control for expected increased levels of knowledge with subsequent retesting.

Results: While OITE scores incrementally increased for the group as a whole (n = 71, RPGY mean improvement = +2.5%, p = 0.406), the junior resident subgroup (postgraduate year 1-2) produced a statistically significant increase in scores (n = 28, RPGY mean increase = 10.1%, p = 0.0260). Nearly 90% (38/42) of curriculum participants surveyed reported a preference to complete a similar review curriculum for future OITE preparation. The participants completing greater than 150 PASS questions (n = 57) were analyzed for OITE predictive capacity of the PASS curriculum. Pearson analysis with PASS questions percent answered correctly and 2014 OITE raw score (independent and dependent variables, respectively) suggests moderate correlation of the variables (R = 0.682, p < 0.05).

Conclusion: We propose that the Orthobullets PASS curriculum is a useful tool for OITE preparation, especially for junior residents, with a focus on achieving competency for a broad fund of orthopedic knowledge, whereas mastery of these topics would be better suited through using complementary sources.

Keywords: Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; in-training examination; orthopedic surgery; resident self-assessment; web-based curriculum.