Assessing Agreement between Radiomic Features Computed for Multiple CT Imaging Settings

RESEARCH ARTICLE Assessing Agreement between Radiomic Features Computed for Multiple CT Imaging Settings Lin Lu1, Ross C. Ehmke2, Lawrence H. Schwartz1, Binsheng Zhao1* 1 Department of Radiology, Columbia University Medical Center, New York, NY, United States of America, 2 Department of Medicine, Columbia University Medical Center, New York, NY, United States of America a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 * Abstract Objectives OPEN ACCESS Citation: Lu L, Ehmke RC, Schwartz LH, Zhao B (2016) Assessing Agreement between Radiomic Features Computed for Multiple CT Imaging Settings. PLoS ONE 11(12): e0166550. doi:10.1371/journal.pone.0166550 Editor: Jie Tian, Institute of Automation Chinese Academy of Sciences, CHINA Received: July 5, 2016 Accepted: October 31, 2016 Published: December 29, 2016 Copyright: © 2016 Lu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All image data are publicly available through the National Cancer Institute’s Reference Image Database to Evaluate Therapy Response project. The definitions of these QIFs along with relevant references can be found in S2 File, and the numerical values of these QIFs can be found in S1 Table. Radiomics utilizes quantitative image features (QIFs) to characterize tumor phenotype. In practice, radiological images are obtained from different vendors’ equipment using various imaging acquisition settings. Our objective was to assess the inter-setting agreement of QIFs computed from CT images by varying two parameters, slice thickness and reconstruction algorithm. Materials and Methods CT images from an IRB-approved/HIPAA-compliant study assessing thirty-two lung cancer patients were included for the analysis. Each scan’s raw data were reconstructed into six imaging series using combinations of two reconstruction algorithms (Lung[L] and Standard[S]) and three slice thicknesses (1.25mm, 2.5mm and 5mm), i.e., 1.25L, 1.25S, 2.5L, 2.5S, 5L and 5S. For each imaging-setting, 89 well-defined QIFs were computed for each of the 32 tumors (one tumor per patient). The six settings led to 15 inter-setting comparisons (combinatorial pairs). To reduce QIF redundancy, hierarchical clustering was done. Concordance correlation coefficients (CCCs) were used to assess inter-setting agreement of the non-redundant feature groups. The CCC of each group was assessed by averaging CCCs of QIFs in the group. Results Twenty-three non-redundant feature groups were created. Across all feature groups, the best inter-setting agreements (CCCs>0.8) were 1.25S vs 2.5S, 1.25L vs 2.5L, and 2.5S vs 5S; the worst (CCCs<0.51) belonged to 1.25L vs 5S and 2.5L vs 5S. Eight of the feature groups related to size, shape, and coarse texture had an average CCC>0.8 across all imaging settings. Funding: This work was supported in part by Grant R01 CA149490 from the National Cancer Institute (NCI). Conclusions Competing Interests: The authors have declared that no competing interests exist. Varying degrees of inter-setting disagreements of QIFs exist when features are computed from CT images reconstructed using different algorithms and slice thicknesses. Our findings PLOS ONE | DOI:10.1371/journal.pone.0166550 December 29, 2016 1 / 12 Assessing Agreement of Radiomic Features highlight the importance of harmonizing imaging acquisition for obtaining consistent QIFs to study tumor imaging phonotype. Introduction Radiomics seeks to use a large number of quantitative image features (QIFs) extracted from non-invasive, routinely acquired radiologic images to characterize tumor phenotypes [1–4]. Radiomics has shown promise in improving cancer diagnosis and prognosis in several tumor types including lung [5–10], brain [11,12], liver [13], kidney [14] and esophageal [15] cancers. While the field of radiomics continues to evolve, a potential limitation in image analysis is the heterogeneous imaging acquisition parameters being used, i.e., there exists a wide range of imaging equipment, acquisition techniques and reconstruction parameters in clinical practice and even in clinical trials. Thus it is important to determine how different imaging acquisition parameters affect computed values of QIFs. Greater understanding of the effect of imaging acquisition choices on radiomic feature variability will lead to increased confidence in the validity of radiomic analyses, inform the standardization of imaging parameters, and enhance generalization and applicability of findings in the rapidly growing field of radiomics. Quantitative image features in radiomics are computed from digital images. They have both spatial resolution (voxel size) and gray-level/density resolution (density bin size) which are determined by imaging acquisition techniques and parameters. To date, our knowledge of the reliability of QIFs is limited to studies of CT and PET test-retest reproducibility [16–19], intra- and inter-observer variability [20,21], segmentation method-induced variability [22,23], variation due to CT acquisition parameters (phantom study) [24], and effects of different CT scanners [25]. There was no in vivo study that assessed how CT imaging acquisition parameters such as slice thickness and reconstruction algorithm affect the computations of QIFs, until the recently published same-day repeat CT study [26]. In that study, Zhao et al. focused on investigating the effect of re-imaging on reproducibility of QIFs under a same imaging setting condition between two repeat scans[26]. In the present study, an urgent unmet need is to assess the agreement between QIFs computed at different imaging settings covering the range of CT imaging reconstruction parameters that are widely used in current clinical practice and oncology clinical trials. Materials and Methods CT Imaging Data A CT imaging data set of 32 non-small cell lung cancer (NSCLC) patients was collected from a previous institutional review board (IRB) approved and Health Insurance Portability and Accountability Act (HIPAA) compliant repeat CT study (ClinicalTrials.gov identifier NCT00579852). In the original study, 32 lung cancer patients underwent two repeat CT scans within 15 minutes. GE scanners were used and the imaging protocol is provided in Table 1. To date, only the repeat imaging series of 1.25L in the protocol was primarily used to study the reproducibility of tumor volume and diameter [21] and became publicly available through the National Cancer Institute’s Reference Image Database to Evaluate Therapy Response (RIDER) project [27]. Since then, researchers around the world have used this publicly accessible data set to validate their computational methods and hypotheses [1–3,16,18,19]. In the present study, the raw data of each patient’s first CT scan were reconstructed in (...truncated)