The authors used a real-time survey instrument and subsequent focus group among primary care clinicians at a large healthcare system to assess usefulness of automated drug alerts. Of 108 alerts encountered, 0.9% (n = 1) represented critical alerts, and 16% (n = 17) were significant drug interaction alerts. Sixty-one percent (n = 66) involved duplication of a medication or medication class. The rest (n = 24) involved topical medications, inhalers, or vaccines. Of the 84 potentially relevant alerts, providers classified 11% (9/84), or about 1 in 9, as useful. Drug interaction alerts were more often deemed useful than drug duplication alerts (44.4% versus 1.5%, P < .001). Focus group participants generally echoed these results when ranking the relevance of 15 selected alerts, although there was wide variance in ratings for individual alerts. Hence, a "smarter" system that utilizes a set of mandatory alerts while allowing providers to tailor use of other automated warnings may improve clinical relevance of drug alert systems.