Objective: Reliability of diagnostic criterion items for psychoactive substance dependence and the impact of each on the reliability of the diagnosis were analyzed.
Method: As part of a reliability study for a new interview developed for the multisite Collaborative Study on the Genetics of Alcoholism (COGA), data were collected from both within-center and across centers. The impact of each diagnostic item on the reliability of the substance dependence diagnosis was studied by forcing each item to be reliable one at a time and recomputing the kappa statistic for the diagnosis.
Results: Findings indicated that the majority of individual diagnostic criterion items were reliable; 87% and 81% were in the fair or better range of reliability for the within- and cross-center studies, respectively. Individual kappa estimates were statistically similar for the two studies. Reliability findings for two classes of substance, alcohol and cocaine, were good, while those for stimulants were less satisfactory.
Conclusions: Forcing items one at a time to be reliable did not affect reliability of the overall substance dependence diagnosis, because more than one criterion item changed from Time 1 to Time 2. Because no single item was influential, weighting criteria equally, as is done in the DSM and ICD classification systems, appears to be a reasonable approach.