Collaborative strategies for deploying artificial intelligence to complement physician diagnoses of acute respiratory distress syndrome

Negar Farzaneh; Sardar Ansari; Elizabeth Lee; Kevin R Ward; Michael W Sjoding

doi:10.1038/s41746-023-00797-9

Collaborative strategies for deploying artificial intelligence to complement physician diagnoses of acute respiratory distress syndrome

NPJ Digit Med. 2023 Apr 8;6(1):62. doi: 10.1038/s41746-023-00797-9.

Authors

Negar Farzaneh^{1

2}, Sardar Ansari^{3

4}, Elizabeth Lee⁵, Kevin R Ward^{3

4

6}, Michael W Sjoding^{3

7}

Affiliations

¹ The Max Harry Weil Institute for Critical Care Research & Innovation, University of Michigan, Ann Arbor, MI, USA. negarf@umich.edu.
² Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, MI, USA. negarf@umich.edu.
³ The Max Harry Weil Institute for Critical Care Research & Innovation, University of Michigan, Ann Arbor, MI, USA.
⁴ Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, MI, USA.
⁵ Department of Radiology, University of Michigan, Ann Arbor, MI, USA.
⁶ Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, USA.
⁷ Department of Internal Medicine, Division of Pulmonary and Critical Care, University of Michigan Medical School, Ann Arbor, MI, USA.

Abstract

There is a growing gap between studies describing the capabilities of artificial intelligence (AI) diagnostic systems using deep learning versus efforts to investigate how or when to integrate AI systems into a real-world clinical practice to support physicians and improve diagnosis. To address this gap, we investigate four potential strategies for AI model deployment and physician collaboration to determine their potential impact on diagnostic accuracy. As a case study, we examine an AI model trained to identify findings of the acute respiratory distress syndrome (ARDS) on chest X-ray images. While this model outperforms physicians at identifying findings of ARDS, there are several reasons why fully automated ARDS detection may not be optimal nor feasible in practice. Among several collaboration strategies tested, we find that if the AI model first reviews the chest X-ray and defers to a physician if it is uncertain, this strategy achieves a higher diagnostic accuracy (0.869, 95% CI 0.835-0.903) compared to a strategy where a physician reviews a chest X-ray first and defers to an AI model if uncertain (0.824, 95% CI 0.781-0.862), or strategies where the physician reviews the chest X-ray alone (0.808, 95% CI 0.767-0.85) or the AI model reviews the chest X-ray alone (0.847, 95% CI 0.806-0.887). If the AI model reviews a chest X-ray first, this allows the AI system to make decisions for up to 79% of cases, letting physicians focus on the most challenging subsets of chest X-rays.

Grants and funding

R01 HL158626/HL/NHLBI NIH HHS/United States