Domain-guided data augmentation for deep learning on medical imaging

PLOS ONE RESEARCH ARTICLE Domain-guided data augmentation for deep learning on medical imaging Chinmayee Athalye ID1¤, Rima Arnaout ID2* 1 Division of Cardiology, Department of Medicine, Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, California, United States of America, 2 Division of Cardiology, Department of Medicine, Department of Radiology, Bakar Computational Health Sciences Institute, Computational Precision Health Graduate Program, Center for Intelligent Imaging, Biological and Medical Informatics Graduate Program, Chan Zuckerberg Biohub Intercampus Research Award Investigator, University of California San Francisco, San Francisco, California, United States of America a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 ¤ Current address: Department of Bioengineering, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America * Abstract OPEN ACCESS Citation: Athalye C, Arnaout R (2023) Domainguided data augmentation for deep learning on medical imaging. PLoS ONE 18(3): e0282532. https://doi.org/10.1371/journal.pone.0282532 Editor: Kathiravan Srinivasan, Vellore Institute of Technology: VIT University, INDIA Received: June 15, 2022 Accepted: February 16, 2023 Published: March 23, 2023 Copyright: © 2023 Athalye, Arnaout. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All relevant data are within the paper and its Supporting information files. Minimal dataset is additionally available from the Zenodo database (doi: https://zenodo.org/ record/7533030#.Y8DuLezMJQI). Funding: R.A. and C.A. are supported by the National Institutes of Health (R01HL150394, https://www.nih.gov/) and the Department of Defense (PR181763, https://www.defense.gov/). R.A. is additionally supported by the Gordon and Betty Moore Foundation (https://www.moore.org/), and is a Chan Zuckerberg Intercampus Research Awardee (https://chanzuckerberg.com/). The While domain-specific data augmentation can be useful in training neural networks for medical imaging tasks, such techniques have not been widely used to date. Our objective was to test whether domain-specific data augmentation is useful for medical imaging using a wellbenchmarked task: view classification on fetal ultrasound FETAL-125 and OB-125 datasets. We found that using a context-preserving cut-paste strategy, we could create valid training data as measured by performance of the resulting trained model on the benchmark test dataset. When used in an online fashion, models trained on this hybrid data performed similarly to those trained using traditional data augmentation (FETAL-125 F-score 85.33 ± 0.24 vs 86.89 ± 0.60, p-value 0.014; OB-125 F-score 74.60 ± 0.11 vs 72.43 ± 0.62, p-value 0.004). Furthermore, the ability to perform augmentations during training time, as well as the ability to apply chosen augmentations equally across data classes, are important considerations in designing a bespoke data augmentation. Finally, we provide open-source code to facilitate running bespoke data augmentations in an online fashion. Taken together, this work expands the ability to design and apply domain-guided data augmentations for medical imaging tasks. Introduction First demonstrated for non-medical tasks, deep learning has shown remarkable utility for medical imaging in recent years [1–3]–with little to no need to adapt neural network architectures for medical domains. The use of augmentation techniques to enhance the diversity of available training data critical to training robust and generalizable deep learning models [4]. Traditional data augmentations for images include shear, rotation, flipping, blurring, contrast stretching, and other operations [5] performed in an online manner during training (i.e., computing slightly different data augmentations on each image on-the-fly during training, so that each time the image is used, it looks slightly different). PLOS ONE | https://doi.org/10.1371/journal.pone.0282532 March 23, 2023 1 / 12 PLOS ONE funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist. Domain-guided data augmentation for deep learning on medical imaging While neural network architectures can be applied off-the-shelf to medical imaging tasks, the same is not always true for data augmentation techniques for two reasons. First, domain expertise is needed to apply traditional data augmentations correctly. For example, flipping Xrays can change the laterality of the image; rotating images significantly can change the training labels in adult echocardiography [6] but can be very helpful in fetal ultrasound [1]; and a high degree of distortion can affect image quality in ways that are not clinically relevant or can obscure important anatomic structures. The spectrum of traditional augmentations/hyperparameters that can be used in a given medical imaging task is therefore constrained, and model training can face data starvation. Second, traditional data augmentations do not fully exploit the domain-specific traits of medical images. This means that opportunities to further expand the training dataset may not be leveraged. Furthermore, failing to apply domain-specific data augmentation can lead to fundamental bias in trained models, such as relying on fiducial markings associated with skin biopsies predict skin cancer [7], or relying on presence of endotracheal tubes or other equipment to predict severity of disease from chest X-rays [8]. To fully exploit domain-specific traits, domain-guided data augmentation can be useful for medical imaging. However, it has not been widely used to date due to complexity of implementation compared to traditional methods. Our objectives were to test whether domain-guided data augmentation can be used to successfully train deep learning tasks in medical imaging, to compare performance of domainguided vs. traditional data augmentation in benchmarked medical imaging tasks, and to highlight important design and implementation considerations when using domain-guided data augmentation. We present a custom, context-preserving, anatomy-aware way of combining two images in an online fashion to create new training data for medical imaging. To illustrate the development and evaluation of domain-guided data augmentation, we used a well-benchmarked [1] task of fetal cardiac view classification, where five screening views of the fetal heart—called 3-vessel trachea (3VT), 3-vessel view (3VV), left-ventricular outflow tract (LVOT), axial 4-chamber (A4C), or Abdomen (ABDO)—must be distinguished from non-target (NT) images. Both class imbalance (see Methods) and the labo (...truncated)