Development and evaluation of a decision-based simulation for assessment of team skills

Abstract

Background: There is a need to train and evaluate a wide variety of nontechnical surgical skills. The goal of this project was to develop and evaluate a decision-based simulation to assess team skills.

Methods: The decision-based exercise used our previously validated Laparoscopic Ventral Hernia simulator and a newly developed team evaluation survey. Five teams of 3 surgical residents (N = 15) were tasked with repairing a 10 × 10-cm right upper quadrant hernia. During the simulation, independent observers (N = 6) completed a 6-item survey assessing: (1) work quality; (2) communication; and (3) team effectiveness. After the simulation, team members self-rated their performance by using the same survey.

Results: Survey reliability revealed a Cronbach's alpha of r = .811. Significant differences were found when we compared team members' (T) and observers' (O) ratings for communication (T = 4.33/5.00 vs O = 3.00/5.00, P < .01) and work quality (T = 4.33/5.00 vs O = 3.33/5.00, P < .05). The team with the greatest survey ratings was the only group to successfully complete the task.

Conclusion: The team evaluation survey had good reliability and correlated with task performance on the simulator. Our current and previous work provides strong evidence that nontechnical and team related skills can be assessed without simulating a crisis situation.