Objectives: To evaluate the reliability and validity of the assessment tool for endoscopic sinus surgery when used in the operating room for in-training assessment of operative competency; to identify the tasks that may serve as the best indicators for overall surgical performance.
Study design: Cross-sectional validation study.
Methods: We implemented the global and checklist parts of the endoscopic sinus surgery (ESS) assessment tool to evaluate the surgical skills of 13 Otolaryngology-Head & Neck Surgery residents (PGY 1-5) in the operating room over a period of 15 months. Rhinology faculty scored residents' performance of every step of ESS at the end of each procedure using a previously validated tool. Construct validity was calculated by comparing scores across training levels (using analysis of variance [ANOVA]). Regression analysis was performed to identify tasks on the ESS checklist that most strongly correlated to the overall surgical performance.
Results: Construct validity was demonstrated with senior residents performing better than junior residents. Average checklist and global scores improved with the number of days on rhinology rotation. "Identification of uncinate and boundaries" was found to be the strongest predictor of overall surgical performance.
Conclusions: The results indicate that this evaluation tool is a reliable and valid instrument for the assessment of surgical competency in the operating room. It can be used to identify weak areas of performance for which additional training may be required early in the rotation/training.