Radiology order decision support: examination-indication appropriateness assessed using 2 electronic systems

Erika Schneider; Stacy Zelenka; Paul Grooff; Dan Alexa; Jennifer Bullen; Nancy A Obuchowski

doi:10.1016/j.jacr.2014.12.005

Radiology order decision support: examination-indication appropriateness assessed using 2 electronic systems

J Am Coll Radiol. 2015 Apr;12(4):349-57. doi: 10.1016/j.jacr.2014.12.005.

Authors

Erika Schneider¹, Stacy Zelenka², Paul Grooff², Dan Alexa², Jennifer Bullen³, Nancy A Obuchowski⁴

Affiliations

¹ Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio. Electronic address: schneie1@ccf.org.
² Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio.
³ Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, Ohio.
⁴ Imaging Institute, Cleveland Clinic Foundation, Cleveland, Ohio; Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, Ohio.

PMID: 25842015
DOI: 10.1016/j.jacr.2014.12.005

Abstract

Purpose: The goal of the study was to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems.

Methods: The appropriateness and insurance dispositions of MRI and CT orders were evaluated using the Medicalis SmartReq and Nuance RadPort CDS systems during 2 different 3-month periods. Logistic regression was used to compare these outcomes between the 2 systems, after adjusting for patient-mix differences.

Results: Approximately 2,000 consecutive outpatient MRI and CT orders were evaluated over 2 periods of 3 months each. Medicalis scored 60% of exams as "indeterminate" (insufficient information) or "not validated" (no guidelines). Excluding these cases, Nuance scored significantly more exams as appropriate than did Medicalis (80% versus 51%, P < .001) and predicted insurance outcome significantly more often (76% versus 58%, P < .001). Only when the Medicalis "indeterminate" and "not validated" categories were combined with the high- or moderate-utility categories did the 2 CDS systems have similar performance. Overall, 19% of examinations with low-utility ratings were reimbursed. Conversely, 0.8% of examinations with high- or moderate-utility ratings were denied reimbursement.

Conclusions: The chief difference between the 2 CDS systems, and the strongest influence on outcomes, was how exams without relevant guidelines or with insufficient information were handled. Nuance augmented published guidelines with clinical best practice; Medicalis requested additional information utilizing pop-up windows. Thus, guideline implementation choices contributed to decision making and outcomes. User interface, specifically, the number of screens and completeness of indication choices, controlled CDS interactions and, coupled with guidance implementation, influenced willingness to use the CDS system.

Keywords: CPOE; decision support; exam appropriateness; utilization management.

Publication types

Comparative Study

MeSH terms

Decision Support Systems, Clinical / standards*
Decision Support Systems, Clinical / statistics & numerical data*
Guideline Adherence / statistics & numerical data*
Ohio
Practice Guidelines as Topic*
Radiology / standards*
Unnecessary Procedures / statistics & numerical data*