External Validation of Predictive Models for Failed Medical Management of Spinal Epidural Abscess

Abstract

Objective: There is limited consensus regarding management of spinal epidural abscesses (SEA), particularly in patients without neurologic deficits. Several models have been created to predict failure of medical management in patients with SEA. We evaluate the external validity of five predictive models in an independent cohort of patients with SEA.

Methods: 176 patients with SEA between 2010 and 2019 at our institution were identified, and variables relevant to each predictive model were collected. Published prediction models were used to assign probability of medical management failure to each patient. Predicted probabilities of medical failure and actual patient outcomes were used to create Receiver Operating Characteristic (ROC) curves, with the area under the ROC curve (AUROC) used to quantify a model's discriminative ability. Calibration curves were plotted using predicted probabilities and actual outcomes. The Spiegelhalter Z-test was used to determine adequate model calibration.

Results: One model (Kim et al.) demonstrated good discriminative ability and adequate model calibration in our cohort (ROC = 0.831, p-value = 0.83). Parameters included in the model were age >65, diabetes, MRSA infection, and neurologic impairment. Four additional models did not perform well for discrimination or calibration metrics (Patel et al., ROC=0.580, p=<0.0001; Shah et al., ROC=0.653, p=<0.0001; Baum et al., ROC=0.498, p=<0.0001; Page et al., ROC=0.534, p=<0.0001).

Conclusion: Only one published predictive model demonstrated acceptable discrimination and calibration in our cohort, suggesting limited generalizability of the evaluated models. Multi-institutional data may facilitate the development of widely applicable models to predict medical management failure in patients with SEA.

Keywords: machine learning; prediction model; spinal epidural abscess; spinal infection.