Study objectives: To examine the applicability of a previously developed intensive care prognostic measure to a community-based sample of hospitals, and assess variations in severity-adjusted mortality across a major metropolitan region.
Design: Retrospective cohort study.
Setting: Twenty-eight hospitals with 38 ICUs participating in a community-wide initiative to measure performance supported by the business community, hospitals, and physicians.
Patients: Included in the study were 116,340 consecutive eligible patients admitted to medical, surgical, neurologic, and mixed medical/surgical ICUs between March 1, 1991, and March 31, 1995.
Main outcome measures: The risk of hospital mortality was assessed using a previous risk prediction equation that was developed in a national sample, and a reestimated logistic regression model fit to the current sample. The standardized mortality ratio (SMR) (actual/predicted mortality) was used to describe hospital performance.
Results: Although discrimination of the previous national risk equation in the current sample was high (receiver operating characteristic [ROC] curve area = 0.90), the equation systematically overestimated the risk of death and was not as well calibrated (Hosmer-Lemeshow statistic, 2407.6, 8 df, p < 0.001). The locally derived equation had similar discrimination (ROC curve area = 0.91), but had improved calibration across all ranges of severity (Hosmer-Lemeshow statistic = 13.5, 8 df, p = 0.10). Hospital SMRs ranged from 0.85 to 1.21, and four hospitals had SMRs that were higher or lower (p < 0.01) than 1.0. Variation in SMRs tended to be greatest during the first year of data collection. SMRs also tended to decline over the 4 years (1.06, 1.02, 0.98, and 0.94 in years 1 to 4, respectively), as did mean hospital length of stay (13.0, 12.4, 11.6, and 11.1 days in years 1 to 4; p < 0.001). However, excluding the increasing (p < 0.001) number of patients discharged to skilled nursing facilities attenuated much of the decline in standardized mortality over time.
Conclusions: A previously validated physiologically based prognostic measure successfully stratified patients in a large community-based sample by their risk of death. However, such methods may require recalibration when applied to new samples and to reflect changes in practice over time. Moreover, although significant variations in hospital standardized mortality were observed, changing hospital discharge practices suggest that in-hospital mortality may no longer be an adequate measure of ICU performance. Community-wide efforts with broad-based support from business, hospitals, and physicians can be sustained over time to assess outcomes associated with ICU care. Such efforts may provide important information about variations in patient outcomes and changes in practice patterns over time. Future efforts should assess the impact of such community-wide initiatives on health-care purchasing and institutional quality improvement programs.