Purpose: Anomia, or word-finding difficulty, is a prevalent and persistent feature of aphasia, a neurogenic language disorder affecting millions of people in the United States. Anomia assessments are essential for measuring performance and monitoring outcomes in clinical settings. This study aims to evaluate the reliability of response time (RT) annotation based on spectrograms and assess the predictive utility of proxy RTs collected during computerized naming tests.
Method: Archival data from 10 people with aphasia were used. Trained research assistants phonemically transcribed participants' responses, and RTs were generated from the onset of picture stimulus to the initial phoneme of the first complete attempt. RTs were measured in two ways: hand-generated RTs (from spectrograms) and proxy RTs (automatically extracted online). Interrater agreement was evaluated based on interclass correlation coefficients and generalizability theory tools including variance partitioning and the φ-coefficient. The predictive utility of proxy RTs was evaluated within a linear mixed-effects framework.
Results: RT annotation reliability showed near-perfect agreement across research assistants (φ-coefficient = .93), and the variance accounted for by raters was negligible. Furthermore, proxy RTs significantly and strongly predicted hand-annotated RTs (R2 = ~0.82), suggesting their utility as an alternative measure.
Conclusions: The study confirms the reliability of RT annotation and demonstrates the predictive utility of proxy RTs in estimating RTs during computerized naming tests. Incorporating proxy RTs can enhance clinical assessments, providing additional information for cognitive measurement. Further research with larger samples and exploring the impact of using proxy RTs in different psychometric models could optimize clinical protocols and improve communication interventions for individuals with aphasia.