Machine-learning assisted swallowing assessment: a deep learning-based quality improvement tool to screen for post-stroke dysphagia

Rami Saab; Arjun Balachandar; Hamza Mahdi; Eptehal Nashnoush; Lucas X Perri; Ashley L Waldron; Alireza Sadeghian; Gordon Rubenfeld; Mark Crowley; Mark I Boulos; Brian J Murray; Houman Khosravani

doi:10.3389/fnins.2023.1302132

Machine-learning assisted swallowing assessment: a deep learning-based quality improvement tool to screen for post-stroke dysphagia

Front Neurosci. 2023 Nov 24:17:1302132. doi: 10.3389/fnins.2023.1302132. eCollection 2023.

Authors

Rami Saab^#¹, Arjun Balachandar^#¹, Hamza Mahdi^#¹, Eptehal Nashnoush^#¹, Lucas X Perri², Ashley L Waldron², Alireza Sadeghian³, Gordon Rubenfeld^{4

5}, Mark Crowley⁶, Mark I Boulos^{1

4}, Brian J Murray¹, Houman Khosravani^{1

2}

Affiliations

¹ Hurvitz Brain Sciences Program, Division of Neurology, Department of Medicine, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada.
² Goodfellow-Waldron Initiative in Stroke Innovation and Recovery, Division of Neurology, Neurology Quality and Innovation Lab, University of Toronto, Toronto, ON, Canada.
³ Department of Computer Science, Faculty of Science, Toronto Metropolitan University, Toronto, ON, Canada.
⁴ Institute of Medical Science, University of Toronto, Toronto, ON, Canada.
⁵ Interdepartmental Division of Critical Care, Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
⁶ Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, Canada.

^# Contributed equally.

Abstract

Introduction: Post-stroke dysphagia is common and associated with significant morbidity and mortality, rendering bedside screening of significant clinical importance. Using voice as a biomarker coupled with deep learning has the potential to improve patient access to screening and mitigate the subjectivity associated with detecting voice change, a component of several validated screening protocols.

Methods: In this single-center study, we developed a proof-of-concept model for automated dysphagia screening and evaluated the performance of this model on training and testing cohorts. Patients were admitted to a comprehensive stroke center, where primary English speakers could follow commands without significant aphasia and participated on a rolling basis. The primary outcome was classification either as a pass or fail equivalent using a dysphagia screening test as a label. Voice data was recorded from patients who spoke a standardized set of vowels, words, and sentences from the National Institute of Health Stroke Scale. Seventy patients were recruited and 68 were included in the analysis, with 40 in training and 28 in testing cohorts, respectively. Speech from patients was segmented into 1,579 audio clips, from which 6,655 Mel-spectrogram images were computed and used as inputs for deep-learning models (DenseNet and ConvNext, separately and together). Clip-level and participant-level swallowing status predictions were obtained through a voting method.

Results: The models demonstrated clip-level dysphagia screening sensitivity of 71% and specificity of 77% (F1 = 0.73, AUC = 0.80 [95% CI: 0.78-0.82]). At the participant level, the sensitivity and specificity were 89 and 79%, respectively (F1 = 0.81, AUC = 0.91 [95% CI: 0.77-1.05]).

Discussion: This study is the first to demonstrate the feasibility of applying deep learning to classify vocalizations to detect post-stroke dysphagia. Our findings suggest potential for enhancing dysphagia screening in clinical settings. https://github.com/UofTNeurology/masa-open-source.

Keywords: Artificial Intelligence; dysphagia; machine learning; neural technology; original research stroke; quality improvement; stroke; swallowing.

Grants and funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. We acknowledge local institutional funding from the Sunnybrook AFP Innovation fund, in addition to Summer Research Studentships from TCAIREM (Temerty Centre for Artificial Intelligence Research and Education in Medicine).