Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese

PLOS ONE RESEARCH ARTICLE Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese Thao Nguyen1☯, Tam M. Vo1☯, Thang V. Nguyen1, Hieu H. Pham ID1,2,3*, Ha Q. Nguyen1,2 1 Smart Health Center, VinBigData JSC, Hanoi, Vietnam, 2 College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam, 3 VinUni-Illinois Smart Health Center, Hanoi, Vietnam a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Nguyen T, Vo TM, Nguyen TV, Pham HH, Nguyen HQ (2022) Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese. PLoS ONE 17(10): e0276545. https://doi.org/10.1371/journal. pone.0276545 Editor: Tarik A. Rashid, University of Kurdistan Hewler, IRAQ Received: May 12, 2022 Accepted: October 7, 2022 Published: October 31, 2022 Copyright: © 2022 Nguyen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Data are available from the Institutional Review Board (IRB) of the Phu Tho General Hospital. Data access may be requested from Dr. Luc Quang Nguyen, Head of Radiology Department, Phu Tho General Hospital, at "," for researchers who meet the criteria for access to confidential data. ☯ These authors contributed equally to this work. * Abstract Deep learning, in recent times, has made remarkable strides when it comes to impressive performance for many tasks, including medical image processing. One of the contributing factors to these advancements is the emergence of large medical image datasets. However, it is exceedingly expensive and time-consuming to construct a large and trustworthy medical dataset; hence, there has been multiple research leveraging medical reports to automatically extract labels for data. The majority of this labor, however, is performed in English. In this work, we propose a data collecting and annotation pipeline that extracts information from Vietnamese radiology reports to provide accurate labels for chest X-ray (CXR) images. This can benefit Vietnamese radiologists and clinicians by annotating data that closely match their endemic diagnosis categories which may vary from country to country. To assess the efficacy of the proposed labeling technique, we built a CXR dataset containing 9,752 studies and evaluated our pipeline using a subset of this dataset. With an F1-score of at least 0.9923, the evaluation demonstrates that our labeling tool performs precisely and consistently across all classes. After building the dataset, we train deep learning models that leverage knowledge transferred from large public CXR datasets. We employ a variety of loss functions to overcome the curse of imbalanced multi-label datasets and conduct experiments with various model architectures to select the one that delivers the best performance. Our best model (CheXpert-pretrained EfficientNet-B2) yields an F1-score of 0.6989 (95% CI 0.6740, 0.7240), AUC of 0.7912, sensitivity of 0.7064 and specificity of 0.8760 for the abnormal diagnosis in general. Finally, we demonstrate that our coarse classification (based on five specific locations of abnormalities) yields comparable results to fine classification (twelve pathologies) on the benchmark CheXpert dataset for general anomaly detection while delivering better performance in terms of the average performance of all classes. Funding: The author(s) received no specific funding for this work. PLOS ONE | https://doi.org/10.1371/journal.pone.0276545 October 31, 2022 1 / 17 PLOS ONE Competing interests: The authors have declared that no competing interests exist. Learning to diagnose common thorax diseases on chest radiographs from radiology reports in Vietnamese Introduction Radiography has always been one of the most ubiquitous diagnostic imaging modalities so far, while chest X-ray (CXR) is the most commonly performed diagnostic X-ray examination [1]. CXRs has an important role in clinical practice, effectively assisting radiologists to detect pathologies related to the airways, pulmonary parenchyma, vessels, mediastinum, heart, pleura and chest wall [2]. In recent years, great advances in GPU computing and research in the fields of machine learning have led to the trend of automating CXR image diagnostics [3–9] and many other X-ray modalities [10–13]. In addition, the availability of large-scale public datasets [14–19] has sparked interest in study and application, with some of them already being used and integrated into the Computer-Aided Diagnosis (CAD) system to reduce the rate of CXR misdiagnosis. Several datasets, including CheXpert [14], MIMIC-CXR [15], PadChest [16], Chest-xray8, Chest-xray14 [17] and VinDr-CXR [19, 20], VinDr-PCXR [21, 22], had a significant impact on increasing labeling methods and model quality. Building a reliable CXR dataset for a specific project, on the other hand, remains a difficult and challenging task because medical data is difficult to obtain due to numerous restrictions on patient information confidentiality, and label quality is heavily influenced by the doctors’ experience and subjective opinion [1]. This is costly and time-consuming but essential, especially for a task that tackles specific challenges, such as focusing on a certain set of patients or illnesses. In such a way that adopting the aforesaid large-scale datasets is sometimes ineffective, possibly because the image quality, labeling, or data characteristics are no longer appropriate. Additionally, CXR images and medical reports corresponding to each examination are also stored in hospital storage systems such as Picture Archiving and Communication System (PACS) and Hospital Information System (HIS) during the radiology process. This is a tremendous available resource to build largescale CXR datasets in which the annotation can be automatically interpolated from the free text report without any involvement of radiologists. Therefore, pipelines or methods to create datasets from available resources are always valuable. Some previous works also developed methods to relabel public large datasets or constructed a new one. Wang et al. [17] proposed a method for extracting a hospital-scale CXR dataset from the PACS via an unified weakly-supervised multi-label image classification and disease localization formulation by applying natural language processing (NLP) techniques. NegBio [23], a rule-based algorithm that utilizes universal dependencies and subgraph matching, known as providing regular expression infrastructure for negation and uncertain detection in radiology reports. Filice et al. [24] investigated the benefit of utilizing AI models to create annotations for review before adjudication in order to speed up the annotation process while sacrificing specificity. Johnson et al. [15] e (...truncated)