Predictive modeling in pediatric traumatic brain injury using machine learning

Chong et al. BMC Medical Research Methodology (2015) 15:22 DOI 10.1186/s12874-015-0015-0 RESEARCH ARTICLE Open Access Predictive modeling in pediatric traumatic brain injury using machine learning Shu-Ling Chong1, Nan Liu2,3*, Sylvaine Barbier3 and Marcus Eng Hock Ong2,4 Abstract Background: Pediatric traumatic brain injury (TBI) constitutes a significant burden and diagnostic challenge in the emergency department (ED). While large North American research networks have derived clinical prediction rules for the head injured child, these may not be generalizable to practices in countries with traditionally low rates of computed tomography (CT). We aim to study predictors for moderate to severe TBI in our ED population aged < 16 years. Methods: This was a retrospective case–control study based on data from a prospective surveillance head injury database. Cases were included if patients presented from 2006 to 2014, with moderate to severe TBI. Controls were age-matched head injured children from the registry, obtained in a 4 control: 1 case ratio. These children remained well on diagnosis and follow up. Demographics, history, and physical examination findings were analyzed and patients followed up for the clinical course and outcome measures of death and neurosurgical intervention. To predict moderate to severe TBI, we built a machine learning (ML) model and a multivariable logistic regression model and compared their performances by means of Receiver Operating Characteristic (ROC) analysis. Results: There were 39 cases and 156 age-matched controls. The following 4 predictors remained statistically significant after multivariable analysis: Involvement in road traffic accident, a history of loss of consciousness, vomiting and signs of base of skull fracture. The logistic regression model was created with these 4 variables while the ML model was built with 3 extra variables, namely the presence of seizure, confusion and clinical signs of skull fracture. At the optimal cutoff scores, the ML method improved upon the logistic regression method with respect to the area under the ROC curve (0.98 vs 0.93), sensitivity (94.9% vs 82.1%), specificity (97.4% vs 92.3%), PPV (90.2% vs 72.7%), and NPV (98.7% vs 95.4%). Conclusions: In this study, we demonstrated the feasibility of using machine learning as a tool to predict moderate to severe TBI. If validated on a large scale, the ML method has the potential not only to guide discretionary use of CT, but also a more careful selection of head injured children who warrant closer monitoring in the hospital. Keywords: Brain injuries, Child, Prediction rules, Machine learning Background Head Injury remains an important cause of mortality and morbidity for children, worldwide. Injury-related deaths in the pediatric age group are mostly associated with head injury [1]. Emergency Departments (EDs) worldwide are seeing an increase in pediatric head injury attendance [2]. The admission rates for head injured * Correspondence: 2 Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore 3 Centre for Quantitative Medicine, Duke-NUS Graduate Medical School, Singapore, Singapore Full list of author information is available at the end of the article children are also on the rise [3]. While the majority of these are mild, severe head injuries have potential for mortality and long-term neurological devastation. The prevalence of neurological disability among children and youths admitted for traumatic brain injury approximates 20% [4]. Compared to adults with head injury, children tend to present in a varied way. Younger children are unable to provide a clear history and may be difficult to examine. A matched retrospective cohort study performed to inform an evidence-based triage assessment showed that young age and injuries to the temporoparietal region were more likely to be associated with © 2015 Chong et al.; licensee BioMed Central. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Chong et al. BMC Medical Research Methodology (2015) 15:22 significant closed head injury, as identified on computed tomography (CT) [5]. CT scans are frequently performed in the adult head injured population. In children however, the rapidly developing brain, when exposed to radiation, is at risk of developing malignancies [6,7]. When deciding on whether a CT is warranted in a young child, the physician has to weigh the need to promptly diagnose an intracranial injury against the radiation that the child will be exposed to. Locally, there is great reluctance to order unnecessary CT scans. Clinical prediction rules [8-10] have been published by large North American research networks to guide the ED physician on when to order a CT scan for a headinjured child. The Pediatric Emergency Care Applied Research Network (PECARN) [7] rule specifically, has been reported to be of excellent performance [11]. However, prior to application, it has been encouraged that the question of generalizability and performance to the individual population be addressed [12]. The CT rate in the Singapore population has been maintained at a low level of under 2%, as opposed to the estimated 30-50% reported in the literature. This is because a large majority of our patients comprise of young children presenting with mild head injuries after falls, as well as the availability of inpatient observation in most cases. While most of the published clinical rules [6-8] were derived with recursive partitioning [13], emerging computational methods like machine learning (ML) have potential in solving complex and challenging medical problems [14-17]. ML procedures are capable of discovering interaction, nonlinear, and high-order effects in the predictive variables [14], which are difficult to handle with conventional parametric regression methods. In this study, we aim to (1) select clinical predictors for moderate to severe traumatic brain injury (TBI) in children aged < 16 years, (2) derive a ML model and a logistic regression model (3) Compare the performance of both tools. Methods Study design and patient recruitment This was a retrospective case–control study. Cases were included if patients presented during the period from 2006 to 2014, with moderate to severe TBI. Due to the very low event rate, a case–control design was chosen [18,19] instead of a cohort analysis. Data was collected from KK Women’s and Children’s Hospital, Singapore, the main pediatric emergency department in Singapore, with an annual trauma attendance (of all severities) of about 28,000. The major (...truncated)