High-Alert Medication Stratification Tool-Revised Phase II: A Multisite Study Examining the Validity, Interrater Reliability, and Ease of Use of the High-Alert Medication Stratification Tool-Revised

Abstract

Objectives: The primary objective of this multisite study, High-Alert Medication Stratification Tool-Revised (HAMST-R) phase II, was to assess the content validity of HAMST-R. Secondary outcomes included interrater reliability and ease of use.

Methods: HAMST-R was designed as an objective tool to evaluate high-alert medications (HAMs) at a single site during HAMST-R Phase I. Medication safety experts from 7 health systems across the United States volunteered to participate in this phase II study. Participants completed a demographic survey, oversaw evaluation of 47 HAMs and 35 non-HAMs using HAMST-R, and submitted scores for each medication evaluated. In addition, participants rated each question of HAMST-R on its relevance to assess a medication's safety risk, measured as scale-content validity index. Positive and negative predictive values were evaluated in a post hoc analysis. Interrater reliability was evaluated using the Kendall coefficient of concordance (K), and ease of use was assessed using a mixed-methods approach.

Results: Scale-content validity index was 0.80, indicating that the tool was valid. Positive predictive value was 90.5% (95% confidence interval, 87.2%-93.0%), and negative predictive value was 98.2% (95% confidence interval, 95.4%-99.3%). A score of 4 or more differentiated between HAMs and non-HAMs, confirming phase I findings. K was 0.56, indicating moderate agreement. Participants confirmed that the tool was easy to use and plan to incorporate the tool into HAM policies and procedures, formulary review, and safety strategy implementation.

Conclusions: HAMST-R is a valid, objective, and easy to use method that institutions may implement to evaluate a medication's potential safety risk.