Challenges of implementing computer-aided diagnostic models for neuroimages in a clinical setting

www.nature.com/npjdigitalmed REVIEW ARTICLE OPEN Challenges of implementing computer-aided diagnostic models for neuroimages in a clinical setting 1234567890():,; Matthew J. Leming 1,2 ✉, Esther E. Bron Hyungsoon Im 1,2,11 ✉ 3 , Rose Bruffaerts4,5, Yangming Ou 6 , Juan Eugenio Iglesias7,8,9, Randy L. Gollub 10 and Advances in artificial intelligence have cultivated a strong interest in developing and validating the clinical utilities of computeraided diagnostic models. Machine learning for diagnostic neuroimaging has often been applied to detect psychological and neurological disorders, typically on small-scale datasets or data collected in a research setting. With the collection and collation of an ever-growing number of public datasets that researchers can freely access, much work has been done in adapting machine learning models to classify these neuroimages by diseases such as Alzheimer’s, ADHD, autism, bipolar disorder, and so on. These studies often come with the promise of being implemented clinically, but despite intense interest in this topic in the laboratory, limited progress has been made in clinical implementation. In this review, we analyze challenges specific to the clinical implementation of diagnostic AI models for neuroimaging data, looking at the differences between laboratory and clinical settings, the inherent limitations of diagnostic AI, and the different incentives and skill sets between research institutions, technology companies, and hospitals. These complexities need to be recognized in the translation of diagnostic AI for neuroimaging from the laboratory to the clinic. npj Digital Medicine (2023)6:129 ; https://doi.org/10.1038/s41746-023-00868-x INTRODUCTION Computer-aided diagnostic (CAD) models are computer algorithms capable of making a prognosis or diagnosis about the health of a patient, given available data. CAD models for radiological images have been widely applied in breast cancer screening in mammograms1,2, largely to automate repetitive tasks, and, more recently, AI tools for the detection of intracranial hemorrhages (ICH) and large vessel occlusion (LVO) in CT images have been approved by the FDA and validated in further studies3–5. The eventual, widespread clinical application of CAD models6 to brain images routinely collected in hospitals, such as CT and MRI, holds promise to automate the diagnostic process, reduce rates of misdiagnosis of brain-related disorders7–10, reduce diagnostic wait times11,12, cut costs, increase diagnostic objectivity13, and inform doctors in their assessment of patients14 for a wide range of brain disorders. Decades of research in machine learning—accelerated in recent years by the surge of interest in deep learning—has led to developments in the research world of CAD models for brain images across a wide range of psychological and neurological disorders15–17. In spite of this, however, very little systemic, real-world, clinical translation has thus far occurred18. This is not entirely unexpected, given historic trends. OakdenRayner6 describes the history of computer-aided detection in radiology as well as its disappointing results in the initial waves of AI, specifically for mammography diagnosis2,19–21, given the limited ability of early diagnostic models. His article provides, in contrast, a more optimistic light on current CAD models because of deep learning’s unprecedented success in other areas of science. This success, however, does not guarantee that it can be implemented successfully in healthcare because success in healthcare is only partially related to the reported efficacy of CAD models. In this article, we attempt to characterize the ongoing progress and future directions of CAD models in translational neuroimaging. We first review the development of CAD models in the research world, covering the continuum of methods with current clinical applicability, those under active development, and those with potential future applications. We then discuss the general challenges of developing CAD models from a purely technical perspective, including issues both unique to healthcare and those seen in machine learning generally. Finally, we discuss translational pathways for bringing neuroimaging CAD models to the clinic as well as the institutional, cultural, and sociological barriers that affect health AI research more generally. We end by suggesting potential future directions and scenarios for translating diagnostic AI to the clinic. UTILITY OF CAD MODELS FOR BRAIN IMAGES CURRENTLY BEING DEVELOPED IN RESEARCH SETTINGS Several past reviews have focused on the development of CAD models for the diagnosis of different brain-related disorders (such as Alzheimer’s disease, Parkinson’s disease, and multiple sclerosis) based on radiological images15–17,22. This work has shown that these disorders exist on an evolving continuum and vary in terms of CAD models’ ability to detect them in neuroimages, 1 Center for Systems Biology, Massachusetts General Hospital, Boston, MA, USA. 2Massachusetts Alzheimer’s Disease Research Center, Charlestown, MA, USA. 3Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands. 4Computational Neurology, Experimental Neurobiology Unit (ENU), Department of Biomedical Sciences, University of Antwerp, Antwerp, Belgium. 5Biomedical Research Institute, Hasselt University, Diepenbeek, Belgium. 6Boston Children’s Hospital, 300 Longwood Ave, Boston, MA, USA. 7Center for Medical Image Computing, University College London, London, UK. 8Martinos Center for Biomedical Imaging, Harvard Medical School, Boston, MA, USA. 9Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA. 10Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. 11Department of Radiology, Massachusetts General Hospital, Boston, MA, USA. ✉email: ; Published in partnership with Seoul National University Bundang Hospital M.J. Leming et al. 1234567890():,; 2 Fig. 1 Current development of different types of neuroimaging CAD models. Neuroimaging CAD models and analysis methods exist on a continuum of development and clinical applicability. Models that use diagnostic segmentation can be applied to brain disorders characterized by focal structural anomalies, and they are in a better position today to be applied clinically. CAD models that output a label directly can help in diagnosing neurodegenerative disorders, which have an explicit, though diffuse, structural basis, and thus CAD models can be used to detect and inform their diagnosis. However, they have yet to see widespread clinical use or a specific clinical need. Brain disorders characterized by both diffuse structural and functional qualities have been analyzed by CAD models, but specific biomarkers are elusive and their clinical implementation would require further development. Neuroimaging CAD models with current clinical a (...truncated)