The sequential organ failure assessment (SOFA) score is a scoring system commonly used in critical care to assess severity of illness. Automated calculation of the SOFA score using existing electronic health record data would broaden its applicability. We performed a manual validation of an automated SOFA score previously developed at our institution. A retrospective analysis of a random subset of 300 patients from a previously published randomized trial of critically ill adults was performed, with manual validation of SOFA scores from the date of initial intensive care unit admission. Spearman's rank correlation coefficient, weighted Cohen's kappa, and Bland-Altman plots were used to assess agreement between manual and electronic versions of SOFA scores and between manual and electronic versions of their individual components. There was high agreement between manual and electronic SOFA scores (Spearman's rank correlation coefficient = 0.90, 95% CI 0.87-0.93). Renal and respiratory components had lower agreement (weighted Cohen's kappa = 0.63, 95% CI 0.53-0.73 for renal; weighted Cohen's kappa = 0.77, 95% CI 0.70-0.84 for respiratory). The area under the receiver operating characteristic curve (AUC) for 30-day in-hospital mortality was 0.77 (95% CI 0.68-0.84) for manual SOFA scores and 0.75 (95% CI 0.66-0.83) for automated SOFA scores. Automatic calculation of SOFA scores from the electronic health record is feasible and correlates highly with manually calculated SOFA scores. Both have similar predictive value for 30-day in-hospital mortality.
Keywords: Automation; Critical care; Decision support techniques; Sepsis.