A foundation for evaluating the surgical artificial intelligence literature

Daniel A Hashimoto; Sai Koushik Sambasastry; Vivek Singh; Sruthi Kurada; Maria Altieri; Takuto Yoshida; Amin Madani; Matjaz Jogan

doi:10.1016/j.ejso.2024.108014

A foundation for evaluating the surgical artificial intelligence literature

Eur J Surg Oncol. 2024 Feb 10:108014. doi: 10.1016/j.ejso.2024.108014. Online ahead of print.

Authors

Daniel A Hashimoto¹, Sai Koushik Sambasastry², Vivek Singh³, Sruthi Kurada², Maria Altieri⁴, Takuto Yoshida⁵, Amin Madani⁶, Matjaz Jogan³

Affiliations

¹ Penn Computer Assisted Surgery and Outcomes Laboratory, Department of Surgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA; Global Surgical AI Collaborative, Toronto, ON, USA. Electronic address: daniel.hashimoto@pennmedicine.upenn.edu.
² Penn Computer Assisted Surgery and Outcomes Laboratory, Department of Surgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA.
³ Penn Computer Assisted Surgery and Outcomes Laboratory, Department of Surgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
⁴ Penn Computer Assisted Surgery and Outcomes Laboratory, Department of Surgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA; Global Surgical AI Collaborative, Toronto, ON, USA.
⁵ Surgical AI Research Academy, Department of Surgery, University Health Network, Toronto, ON, USA.
⁶ Global Surgical AI Collaborative, Toronto, ON, USA; Surgical AI Research Academy, Department of Surgery, University Health Network, Toronto, ON, USA.

PMID: 38360498
DOI: 10.1016/j.ejso.2024.108014

Abstract

With increasing growth in applications of artificial intelligence (AI) in surgery, it has become essential for surgeons to gain a foundation of knowledge to critically appraise the scientific literature, commercial claims regarding products, and regulatory and legal frameworks that govern the development and use of AI. This guide offers surgeons a framework with which to evaluate manuscripts that incorporate the use of AI. It provides a glossary of common terms, an overview of prerequisite knowledge to maximize understanding of methodology, and recommendations on how to carefully consider each element of a manuscript to assess the quality of the data on which an algorithm was trained, the appropriateness of the methodological approach, the potential for reproducibility of the experiment, and the applicability to surgical practice, including considerations on generalizability and scalability.

Keywords: Algorithms; Artificial intelligence; Computer vision; Data science; Deep learning; Machine learning.