Interpretable Machine Learning for Early Prediction of Prognosis in Sepsis: A Discovery and Validation Study

Chang Hu; Lu Li; Weipeng Huang; Tong Wu; Qiancheng Xu; Juan Liu; Bo Hu

doi:10.1007/s40121-022-00628-6

Interpretable Machine Learning for Early Prediction of Prognosis in Sepsis: A Discovery and Validation Study

Infect Dis Ther. 2022 Jun;11(3):1117-1132. doi: 10.1007/s40121-022-00628-6. Epub 2022 Apr 10.

Authors

Chang Hu^#^{1

2}, Lu Li^#^{1

2}, Weipeng Huang^{1

2}, Tong Wu^{1

2}, Qiancheng Xu^{1

2}, Juan Liu³, Bo Hu^{4

5}

Affiliations

¹ Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan, 430071, Hubei, China.
² Clinical Research Center of Hubei Critical Care Medicine, Wuhan, 430071, Hubei, China.
³ School of Computer Science, Wuhan University, Wuhan, 430072, Hubei, China. liujuan@whu.edu.cn.
⁴ Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, Wuhan, 430071, Hubei, China. hobbier1979@163.com.
⁵ Clinical Research Center of Hubei Critical Care Medicine, Wuhan, 430071, Hubei, China. hobbier1979@163.com.

^# Contributed equally.

Abstract

Introduction: This study aimed to develop and validate an interpretable machine-learning model based on clinical features for early predicting in-hospital mortality in critically ill patients with sepsis.

Methods: We enrolled all patients with sepsis in the Medical Information Mart for Intensive Care IV (MIMIC-IV, v.1.0) database from 2008 to 2019. Lasso regression was used for feature selection. Seven machine-learning methods were applied to develop the models. The best model was selected based on its accuracy and area under curve (AUC) in the validation cohort. Furthermore, we employed the SHapley Additive exPlanations (SHAP) method to illustrate the effects of the features attributed to the model, and to analyze how the individual features affect the output of the model, and to visualize the Shapley value for a single individual.

Results: In total, 8,817 patients with sepsis were eligible for participation, the median age was 66.8 years (IQR, 55.9-77.1 years), and 3361 of 8817 participants (38.1%) were women. After selection, 25 of a total 57 clinical parameters collected on day 1 after ICU admission remained associated with prognosis and were used for developing the machine-learning models. Among seven constructed models, the eXtreme Gradient Boosting (XGBoost) model achieved the best performance with an AUC of 0.884 and an accuracy of 89.5% in the validation cohort. Feature importance analysis showed that Glasgow Coma Scale (GCS) score, blood urea nitrogen, respiratory rate, urine output, and age were the top 5 features of the XGBoost model with the greatest impact. Furthermore, SHAP force analysis illustrated how the constructed model visualized the individualized prediction of death.

Conclusions: We have demonstrated the potential of machine-learning approaches for predicting outcome early in patients with sepsis. The SHAP method could improve the interpretability of machine-learning models and help clinicians better understand the reasoning behind the outcome.

Keywords: Algorithm; Critically ill; Machine learning; Mortality; Sepsis.

Abstract

Grants and funding