Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, Dec 2017

The aim of this Letter to the Editor was to report some methodological shortcomings in a recently published article. Issues regarding missing values and overfitting are mentioned. First, Complete Case (CC) analysis was used instead of an imputation method. Second, there was a high chance of overfitting and lack of model validation. In conclusion, the results of this study should be interpret with caution and further research is necessary.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://sjtrem.biomedcentral.com/track/pdf/10.1186/s13049-017-0454-1?site=sjtrem.biomedcentral.com

Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study

Sewalt et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study" Charlie A. Sewalt 0 Eveline J. A. Wiegers 0 Esmee Venema 0 Hester F. Lingsma 0 0 Center for Medical Decision Making, Department of Public Health, Erasmus University Medical Center , P.O. Box 2040, 3000, CA, Rotterdam , The Netherlands The aim of this Letter to the Editor was to report some methodological shortcomings in a recently published article. Issues regarding missing values and overfitting are mentioned. First, Complete Case (CC) analysis was used instead of an imputation method. Second, there was a high chance of overfitting and lack of model validation. In conclusion, the results of this study should be interpret with caution and further research is necessary. Predictive factors; Prediction modelling; Methodology - With great interest we read the study by Thompson et al. [ 1 ] where they identified pre-hospital factors associated with major trauma outcomes. This study showed that Glasgow Coma Score (GCS), Respiration Rate (RR) and Age are potential predictive triggers for direct transport to a Major Trauma Center (MTC). This is an interesting finding, which might help in the challenging decision which patients will benefit from treatment in MTCs. However, some methodological issues should be taken into consideration. First, the authors used ‘listwise’ exclusion, also known as complete case (CC) analysis, to handle their missing data. This resulted in excluding almost 45% of their entire sample (462 out of 1033 casualties). Obviously, this leads to less efficiency and possibly bias [ 2, 3 ]. In dealing with missing data, the CC analysis could be biased when Missing at Random (MAR) on the outcome variable is present [ 2–4 ], for example when GCS is missing in patients with high GCS who have a high probability to die. Thus imputation methods should have been considered. This could certainly increase efficiency and potentially reduce bias dependent on the mechanism of missing data [ 2–4 ]. Second, the authors stated that their model including GCS, RR and Age correctly predicted 97.4% of the casualties. This high prediction rate could be the result of overfitting and might not be generalizable to the population [ 3 ]. Three strategies could be followed to avoid overfitting. First, the use of a more liberal p-value than 0.050 and preselection of variables based on clinical knowledge could have decreased the chance of testimation bias and overestimation of the effect of the selected predictors, especially with few events [ 3, 5 ]. Two other important steps in prediction modelling are internal and external validation [ 3, 6 ]. Internal validation is about the stability of the selected predictors and the quality of the predictions in the underlying population [ 3, 6 ]. External validation is about the generalizability of the predictors and predictions in comparable populations [ 3, 6 ]. Unfortunately none of these strategies to decrease overfitting and increase the quality of the prediction models have been used and therefore overfitting is likely in this study. In conclusion, results of this study should be interpret with caution and further research is necessary to estimate the predictive ability of pre-hospital factors with special emphasis on model validity and overfitting. Abbreviations CC: Complete case; GCS: Glasgow coma score; MAR: Missing at random; MTC: Major trauma center; RR: Respiration rate Acknowledgements Not applicable. Funding No funding was sought or obtained for this study. Availability of data and materials Not applicable. Authors’ contributions CS and EW were the leading authors, EV: proofreading and intellectual input, HL: proofreading and intellectual input. All authors read and approved the final manuscript. Ethics approval and consent to participate Not applicable. Consent for publication Not applicable. Competing interests The authors declare that they have no competing interests. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 1. Thompson L , Hill M , Davies C , Shaw G , Kiernan MD . Identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study . Scand J Trauma Resusc Emerg Med . 2017 ; 25 ( 1 ): 83 . 2. White IR , Carlin JB . Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values . Stat Med . 2010 ; 29 ( 28 ): 2920 - 31 . 3. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating . New York: Springer; 2008 . 4. Vach W , Blettner M. Missing data in epidemiologic studies. Encyclopedia of biostatistics . New Jersey: Wiley; 2005 . 5 (...truncated)


This is a preview of a remote PDF: https://sjtrem.biomedcentral.com/track/pdf/10.1186/s13049-017-0454-1?site=sjtrem.biomedcentral.com

Charlie A. Sewalt, Eveline J. A. Wiegers, Esmee Venema, Hester F. Lingsma. Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study, Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, pp. 113,