The main question addressed by this paper is whether DSM-IV substance dependence diagnoses obtained from two different instruments (the semi-structured WHO Schedules for Clinical Assessment in Neuropsychiatry, SCAN and the highly structured WHO Composite International Diagnostic Interview--Substance Abuse Module, SAM) are as consistent as diagnoses obtained from a single instrument (SAM) administered twice. Such comparisons of results from the two different instruments provide some measure of validity of the lay-administered SAM and of the underlying diagnostic concepts. Chance-corrected concordance was estimated using the kappa coefficient for SAM/SCAN (test/validation) and SAM/SAM (test/retest) comparisons. Analyses of agreement between SAM and SCAN for DSM-IV dependence diagnoses indicated good agreement for alcohol and cocaine, and fair agreement for opiates and cannabis. SAM/SAM (test/retest) agreement was excellent for alcohol and opiate dependence, good for cocaine dependence, and fair for cannabis dependence. Agreement on individual dependence criteria was generally consistent with overall diagnostic agreement though more variable. Notable was the poor agreement for cannabis criteria in the SAM/SCAN protocol. This may indicate that the dependence syndrome is less applicable to cannabis, while the consistency of agreement for alcohol, opiate, and cocaine dependence criteria supports the validity of these dependence syndromes. Finally, these data indicate that both the clinical (SCAN) and non-clinical (SAM) interviews can be used effectively for a variety of substances and dependence diagnoses.