Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review

PLOS ONE, Sep 2016

Traumatic brain injury (TBI) is a leading cause of death worldwide and is increasing exponentially particularly in low and middle income countries (LMIC). To inform the development of a standard Clinical Practice Guideline (CPG) for the acute management of TBI that can be implemented specifically for limited resource settings, we conducted a systematic review to identify and assess the quality of all currently available CPGs on acute TBI using the AGREE II instrument. In accordance with PRISMA guidelines, from April 2013 to December 2015 we searched MEDLINE, EMBASE, Google Scholar and the Duke University Medical Center Library Guidelines for peer-reviewed published Clinical Practice Guidelines on the acute management of TBI (less than 24 hours), for any level of traumatic brain injury in both high and low income settings. A comprehensive reference and citation analysis was performed. CPGs found were assessed using the AGREE II instrument by five independent reviewers and scores were aggregated and reported in percentage of total possible score. An initial 2742 articles were evaluated with an additional 98 articles from the citation and reference analysis, yielding 273 full texts examined. A total of 24 final CPGs were included, of which 23 were from high income countries (HIC) and 1 from LMIC. Based on the AGREE II instrument, the best score on overall assessment was 100.0 for the CPG from the National Institute for Health and Clinical Excellence (NIHCE, 2007), followed by the New Zealand Guidelines Group (NZ, 2006) and the National Clinical Guideline (SIGN, 2009) both with a score of 96.7. The CPG from a LMIC had lower scores than CPGs from higher income settings. Our study identified and evaluated 24 CPGs with the highest scores in clarity and presentation, scope and purpose, and rigor of development. Most of these CPGs were developed in HICs, with limited applicability or utility for resource limited settings. Stakeholder involvement, Applicability, and Editorial independence remain weak and insufficiently described specifically with piloting, addressing potential costs and implementation barriers, and auditing for quality improvement.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0161554&type=printable

Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review

September Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review Anjni Patel 0 1 Mateus Mazorra Coelho Vieira 1 John Abraham 0 1 Nick Reid 0 1 Tu Tran 1 Kevin Tomecsek 0 1 João Ricardo N. Vissoci 1 Stephanie Eucker 0 1 Charles J. Gerardo 0 1 Catherine A. Staton 0 1 0 Division of Emergency Medicine, Department of Surgery, Duke University Medical Center , Durham , North Carolina, United States of America, 2 Centro Universitário Unicesumar, Maringá, Brazil, 3 Duke Global Health Institute, Duke University , Durham , North Carolina, United States of America, 4 Division of Global Neuroscience and Neurosurgery, Department of Neurosurgery, Duke University Medical Center , Durham , North Carolina, United States of America , 5 Faculdade Inga , Medicine Department , Maringá, 87005240 , Brazil 1 Editor: Johannes Boltze, Fraunhofer Research Institution of Marine Biotechnology , GERMANY Traumatic brain injury (TBI) is a leading cause of death worldwide and is increasing exponentially particularly in low and middle income countries (LMIC). To inform the development of a standard Clinical Practice Guideline (CPG) for the acute management of TBI that can be implemented specifically for limited resource settings, we conducted a systematic review to identify and assess the quality of all currently available CPGs on acute TBI using the AGREE II instrument. In accordance with PRISMA guidelines, from April 2013 to December 2015 we searched MEDLINE, EMBASE, Google Scholar and the Duke University Medical Center Library Guidelines for peer-reviewed published Clinical Practice Guidelines on the acute management of TBI (less than 24 hours), for any level of traumatic brain injury in both high and low income settings. A comprehensive reference and citation analysis was performed. CPGs found were assessed using the AGREE II instrument by five independent reviewers and scores were aggregated and reported in percentage of total possible score. An initial 2742 articles were evaluated with an additional 98 articles from the citation and reference analysis, yielding 273 full texts examined. A total of 24 final CPGs were included, of which 23 were from high income countries (HIC) and 1 from LMIC. Based on the AGREE II instrument, the best score on overall assessment was 100.0 for the CPG from the National Institute for Health and Clinical Excellence (NIHCE, 2007), followed by the New Zealand Guidelines Group (NZ, 2006) and the National Clinical Guideline (SIGN, 2009) both with a score of 96.7. The CPG from a LMIC had lower scores than CPGs from higher income settings. Our study identified and evaluated 24 CPGs with the highest scores in clarity and presentation, scope and purpose, and rigor of development. Most of these CPGs were developed in HICs, with limited applicability or utility for resource limited settings. Stakeholder involvement, Applicability, and Editorial independence remain - OPEN ACCESS Data Availability Statement: All relevant data are within the paper and its Supporting Information files. Funding: The authors received no research funding for this work. Dr. Staton would like to acknowledge salary support funding from the Fogarty International Center (Staton, K01 TW010000-01A1). Competing Interests: The authors have declared that no competing interests exist. weak and insufficiently described specifically with piloting, addressing potential costs and implementation barriers, and auditing for quality improvement. Introduction Traumatic brain injury (TBI) is one of the leading causes of death and disability in both developing and developed countries, with the highest incidences among young people less than 30 years of age [ 1, 2 ]. While the current global burden is unknown, previous conservative estimates indicate an annual incidence of over 10 million people sustaining a TBI leading to hospitalization or death, with road traffic injuries causing a preponderance of cases [ 1, 3 ]. TBI incidence is projected to continue to rise worldwide due to the continued increasing rates of road traffic injuries, particularly in low and middle-income countries (LMIC) where the rates are twice as high as in high-income countries [ 1, 4 ]. Furthermore, the World Health Organization suggests that upwards of 90% of road traffic injury deaths occur in LMIC. These trends have been attributed to the rapid economic growth, urbanization, and motorization but limited infrastructure improvements in LMIC [ 1, 4 ]. Unfortunately, as the burden of TBI continues to increase globally, appropriate prevention efforts have been limited, especially in LMIC, and healthcare quality remains poor, resulting in disproportionately higher mortality rates [ 5 ]. One healthcare quality improvement measure that has shown impact is the use of clinical practice guidelines (CPGs) [ 6–9 ]. In fact, in the last several years a significant number of CPGs for acute TBI care has been developed worldwide [5]. However, CPGs vary in quality and comprehensiveness, leading to difficulties with standardization of care, adaptation and implementation, particularly in resource limited settings [ 10 ]. There is limited literature comparing and evaluating the strengths and weaknesses of all available CPGs for the treatment of acute TBI. The Brain Trauma Foundation first developed CPGs for the management of severe TBI in the United States in 1995, with subsequent updated editions published in 2000 and 2007 [ 11– 13 ]. These guidelines have gradually gained acceptance and increased use internationally, with a number of countries adapting them to their individual needs. For example, until recently in Saudi Arabia, patients with severe TBI were managed per individual provider knowledge and experience. By implementing an ICU protocol derived from the Brain Trauma Foundation’s guidelines, Saudi Arabian providers were able to significantly reduce hospital and ICU mortality due to TBI [ 6 ]. Despite this, a major criticism of the Brain Trauma Foundation CPGs is that they may not be appropriate for use in all locales due to differences in available resources. Subsequently, a number of newer CPGs have been developed in many different countries and by various practitioner groups based on data and capabilities specific to their respective practice environments [ 14–16 ]. Applying these CPGs in LMICs with limited resources is challenging. Additionally, these CPGs vary in quality and content, span across multiple disciplines, and are published in disparate literature bases, making effective utilization challenging. Previous studies evaluating quality of existing TBI CPGs, for instance, have focused on subsets of TBI severity such as mild TBI only [ 14, 16 ], or review only a limited number of CPGs [17]. We aimed to assess and summarize the quality of all currently available international acute TBI CPGs by conducting a systematic review using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument [ 17 ]. We also compared the quality of CPGs created in high resource countries with those from low and middle income. We expect that these results 2 / 17 will inform the development of a standard CPG for the acute management of TBI that can be implemented specifically for limited resource settings. Materials and Methods Protocol and Registration This systematic review is reported in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) Statement [ 18 ], and is registered in the PROSPERO database (International Prospective Register of Systematic Reviews) under the number CRD42013006008. Eligibility Criteria Articles mentioning clinical practice guidelines or recommendations for traumatic brain injury, which met the following inclusion criteria were considered: acute management of TBI (less than 24 hours), any level of traumatic brain injury, high and low income countries and publication in English. No CPG population age restrictions were added as we chose to evaluate both pediatric and adult CPGs. For multiple versions of CPGs, only the newest CPGs were included in the analysis and the older versions were excluded. Information Sources We employed an extensive search strategy from April 2013 to December 2015 to identify guidelines from the following electronic databases: MEDLINE, EMBASE and the Duke University Medical Center Guidelines located in the Duke University Medical Center Library and Archives [ 19 ]. The Duke University Medical Center Guidelines is an electronic repository curated by Duke University which gathers several national sources indexing CPGs, including the following: National Guideline Clearinghouse, Center for Disease Control Guidelines, Cochrane Library, AHRQ Evidence reports, American College of Emergency Medicine CPGs, American Academy of Child & Adolescent Psychiatry, American Psychiatric Association Guidelines, Canadian Medical Association, Health Services/Technology Assessment Text Collection, Infectious Disease Society of America, Practice guidelines, Joint National Committee, National Comprehensive Cancer Network Guidelines, National Institute for for Health and Clinical Excellence guidelines, NIH Consensus statements archive, US Preventative Services Task Force, Veterans Affairs Clinical Practice Guidelines. In addition, we manually searched the references and performed a citation analysis of the included studies using Google Scholar to include any potential CPG document that was not included in the initial steps. Literature Search The initial search comprised of the MeSH terms "Brain Injuries", "Guideline [Publication Type]”, “Practice Guideline [Publication Type]”, “Practice Guidelines as Topic”, and their respectives entry terms. Appendix 1 presents the search strategy used in the PubMed database. We did not use limits for languages or date when searching the databases, but added to the final list only documents in English. Study Selection Titles and abstract of the retrieved articles were independently evaluated by two reviewers (A. P. and J.A.). Abstracts that did not provide enough information regarding the eligibility criteria were retrieved for full-text evaluation. Reviewers (A.P. and J.A.) independently evaluated fulltext articles and determined study eligibility. Disagreements were solved by consensus and if 3 / 17 disagreement persisted, a third reviewer’s opinion was sought (C.S). For CPGs with more than one version, we reported only the most recent versions given their large overlap and anticipated improvement in quality. Quality of Clinical Practice Guidelines Five appraisers independently assessed each eligible and selected guideline for quality in accordance with the AGREE II instrument [ 17, 20 ]. AGREE II is an instrument designed to assess quality of clinical practice guidelines. It consists of 23 items divided into six domains: 1) scope and purpose, 2) stakeholder involvement, 3) rigor of development, 4) clarity of presentation, 5) applicability, and 6) editorial independence. Domain one, scope and purpose, evaluates specificity of the overall objective, clinical questions and the patient population described in the CPG. Domain two measures stakeholder involvement, inclusion of relevant professional groups, patients’ views and preferences, and target users. Domain three evaluates the guideline’s systematic methods, stating criteria for selecting evidence and explicit links between evidence and recommendations, strengths and limitations of the evidence, consideration of risks and benefits, external review prior to publication, and procedures for updating the guideline. The fourth domain, clarity of presentation, evaluates for clarity, lack of ambiguity, determines if different management options are presented, and assesses whether key recommendations are easily identifiable. Domain five assesses if the CPGs address recommendations for clinical application, barriers to application, cost or resource implications, and monitoring criteria. Finally, domain six evaluates for editorial independence from funding bodies and addressing of potential conflicts of interest. Each item was scored on a seven-point scale (one = strongly disagree and seven = strongly agree). Scores for each domain were calculated by using the sum of all items within a domain and scaling the score as a percentage of the maximum possible score using the following formula: Obtained Score Maximum Possible Score Minimum Possible Score Minimum Possible Score The results from each guideline were summarized in a heatmap visualization with values for each domain. All 23 items of the AGREE-II instrument were assessed with results reported in percentage form for each of the six domains. A value of 100% indicated a domain in which all items were scored with seven points (strongly agree). A value of 0% corresponded to a domain in which all items were scored with one point (strongly disagree). Each reviewer received a user’s manual of the AGREE II instrument, containing its instructions. The six AGREE domains were reported independently for each included CPG. Additionally, an independent global assessment was conducted from the six domains and reported along with the appraiser's recommendation. Recommendations were measured in a threeoption scale (“Yes”, “Yes with modifications”, and “No”), with a qualitative comment about the CPG [ 17 ]. For each domain and for the global assessment, we report average values with respective standard deviations and range (minimum and maximum values). Data Extraction Two reviewers (A.P. and J.A.) independently conducted the data extraction and disagreements were resolved by a third reviewer (C.S.). Besides the AGREE assessment previously described, the following general characteristics of the studies were collected: year of publication, location where the guideline creation took place, the organization that created the guidelines, the main focus of the guideline, the patient population of the guideline, and the severity of TBI the 4 / 17 guideline was addressing. Additionally, the countries of origin for each guideline were classified into high-, middle-, and low-income based on World Bank definitions [ 21 ]. The senior researcher (C.S.) moderated all discrepancies or doubts during the rating process. Data Analysis Data analysis was performed descriptively and with graphical representation. Overall assessments for each domain were calculated following the methods already described [ 17, 20 ]. Consistency of evaluations of the AGREE II domain and for the overall assessment was calculated with an intra-class correlation coefficient (ICC). Graphical solutions were carried out with R software for statistical language [22]. Results Study Selection The initial search strategy identified 2,742 titles and abstracts 80 of which were removed for duplicates. From these, 2487 were excluded after reviewing abstracts. A reference and citation analysis was performed on the remaining 175 articles yielding an additional 98 abstracts. Full text analysis was then performed on a total of 273 articles of which only 24 [ 23–46 ] met inclusion criteria (Fig 1). CPG Characteristics Overall, there were 24 guidelines that were included in this analysis (Table 1) representing 19 different organizations and spanning several countries on four continents. Of these 24 CPG's, 23 [ 23–45 ] were developed in high-income countries and only one [ 46 ] from a upper middleincome country (Brazil). The CPGs evaluated covered the full scope of adult and pediatric populations with four covering pediatric patients [ 23, 24, 27, 29 ], eight for adults patients [ 25, 28, 33, 38, 40, 41, 43, 44 ], and seven covering both populations [ 26, 30, 34, 36, 37, 39, 42 ]. Five did not specify their population group [ 31, 32, 35, 45, 46 ]. Regarding the severity of TBI, one third of the CPGs were developed for minor or mild TBI [ 23, 24, 25, 29, 31, 32, 34, 33 ], another third covered severe TBI [ 26, 27, 28, 33, 35, 36, 37, 44, 45 ] and the rest were developed for all levels of TBI severity [ 30, 37, 38, 39, 40, 41, 42 ]. The majority (19) of CPGs focused on the early management of TBI [ 23, 24, 25, 27, 29, 30, 31, 32, 34, 38, 39, 40, 41, 42, 43, 44, 46 ], with two focusing specifically on prehospital care [ 27, 36 ], another two on both early management and ICU care [ 28, 45 ], one covered prehospital, early management, and rehabilitation [39], and one covered the entire breadth of management of severe TBI [ 36 ]. Of the 24 assessed CPGs, roughly half (11) were developed by professional organizations [ 23, 24, 25, 30, 31, 32, 35, 36, 37, 43, 44 ], four were developed by non-profit organizations [ 26, 27, 39, 42 ], three by international committees [ 41, 33, 34 ], another three by national institutes or government organizations [ 37, 38, 40 ], one from an academic organization [29], one did not specified the type of organization [ 45 ], and the remainder of the CPGs were developed by mixture of different organizations: a non-profit professional organization [ 28 ] and a professional organization with an academic organization [ 46 ]. CPG Quality Assessment (AGREE) All AGREE II assessments summaries (average per domain and standard deviation) are described per domain and with the overall assessment. Domain specific results are summarized in Fig 2. Overall assessment recommendations and comments are displayed in Table 2. Interrater reliability for each domain is expressed in Table 3. 5 / 17 Fig 1. Study Flow Diagram. Domain One—Scope and Purpose. The lowest score was 55.9 which was from Guidelines for the Pre-hospital Care of Patients with Severe Head Injuries (ESICM, 1998). The highest score was 86.9, from both the Adult Trauma Clinical Practice Guidelines, Initial Management of Closed Head Injury in Adults (NSW MoH, 2011) and the ACEP Clinical Policy: Neuroimaging and Decisionmaking in Adult Mild Traumatic Brain Injury in the Acute Setting (ACEP, 2009). Domain Two—Stakeholder Involvement. The highest score of 86.9 was from the Early Management of Patients with a Head Injury; A National Clinical Guideline. (SIGN, 2009) The lowest score of 41.9 was from The Study Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for minor head injured patients' management in adult age (SINch, 1996). Domain Three—Rigor of Development. The lowest score was 28.3 from The Study Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for minor head injured patients' management in adult age (SINch, 1996). The highest score of 93.3 was from the New Zealand Guidelines Group: Traumatic Brain Injury: Diagnosis, Acute Management and Rehabilitation (NZ, 2006). Domain Four—Clarity of Presentation. The lowest score was 64.4 from the Guidelines for Neurosurgical Trauma in Brazil (USP/BSN, 2001) and the highest value was 95.6 achieved by two guidelines: Early Management of Patients with a Head Injury: A National Clinical Guideline (SIGN, 2009) and the Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and Adolescents (BTF, 2012). Domain Five–Application. Two guidelines shared the lowest score (38.3): the Study Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for Minor Head Injured Patients’ Management in Adult Age (SINch, 1996) and Guidelines for the Pre-hospital care of Patients with Severe Head Injuries (ESICM, 1998). The highest score of 95.0 belonged to the following guideline: Early Management of Patients with a Head Injury. A National Clinical Guideline (SIGN, 2009). Domain Six—Editorial Independence. The lowest score of 40 points came from both the Study Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for minor 6 / 17 (Continued) 7 / 17 (Continued) 8 / 17 head injured patients' management in adult age (SINch, 1996) and the Guidelines for Neurosurgical Trauma in Brazil (USP/BSN, 2001). The highest score was 93.3, from the New Zealand Guidelines Group: Traumatic Brain Injury: Diagnosis, Acute Management and Rehabilitation (NZ, 2006). Overall Assessment. The 2007 CPG from the National Institute for Health and Clinical Excellence (NIHCE, 2007) had the best overall assessment with maximum score from all appraisers, followed by the New Zealand Guidelines Group (NZ, 2006) and the National Clinical Guideline (SIGN, 2009) both with only one appraiser scoring six points each. The worst ovserall assessment was for the CPG from The Study Group on Head Injury of the Italian Society for Neurosurgery (SINch, 1996). Most CPGs would be recommend by the appraisers (EFNS, 2001; AAP, 2001; NZ, 2006; BTF/AANS, 2007; NIHCE, 2007; ACEP, 2009; SIGN, * Clinical Practice Guidelines have older versions, only most recent version of this Clincal Practice Guidelines was included 9 / 17 Fig 2. AGREE Scoring by domain for each Clinical Practice Guidelines. (CPG = Clinical Pratice Guideline; D1 = Domain one, scope and porpouse; D2 = Domain 2, stakeholder involvement; D3 = Domain three, rigor of development; D4 = Domain four, clarity of presentation; D5 = Domain five, applicability; D6 = Domain six, editorial Independence, * = Indicates newest version of a CPG for which multiple versions exist). 2009; NSW MoH, 2011). Three CPGs had 60% of appraisers recommending with modifications (SINch/SIAARTI, 2000; EAST, 2002; CHOP, 2003). The guidelines from the Study Group on Head Injury of the Italian Society for Neurosurgery (SINch, 1996) was the only CPG that 100% of reviewers reported they would not recommend for use. Comments on the strengths and weaknesses of each CPG are summarized in Table 2. Appraisers Consistency. Overall reliability was very good, with three of the quality domains (stakeholder involvement, rigor of development and editorial independence) and the overall assessment with values above 0.70, and two (scope and purpose and applicability) values around 0.65. The domain “clarity and presentation” had the lowest reliability (0.43) (Table 3). Discussion This systematic review is the first to synthesize and collate all published clinical practice guidelines for all types of traumatic brain injury into a single large-scale quality review. Our study included 24 CPGs; across all TBI CPGs, the highest mean scores were achieved in clarity and presentation, scope and purpose and rigour of development, while the main weaknesses across CPGs were stakeholder involvement, applicability and editorial independence. The National Institute for Health and Clinical Excellence (NIHCE, 2007) guidelines, the New Zealand Guidelines Group (NZ, 2006), and the National Clinical Guideline (SIGN, 2009) were the three CPGs with best results. The majority of CPGs evaluated in this study were developed by high income countries (HICs), and are therefore minimally applicable in resource limited settings. CPGs strengths and weaknesses Overall, the strong scores in the clarity and presentation, scope and purpose, and rigor of development domains have been reported in other systematic reviews evaluating TBI CPGs [14, 17, 10 / 17 A1 A2 A3 A4 A5 Yes No 6 100% % of CPG recommendation for use Mod. 0% (CPGs = Clinical Pratice Guidelines; A1, A2, A3, A4, A5 = appraisers 1 to 5 respectively) 11 / 17 0% Well written and comprehensive text with good levels of evidence. Average methods and stakeholder involvement. Merits in use despite population age. Presence of good facilitators for applicability. 20% Well writtten and comprehensive text with good levels of evidence. Average methods and applicabilty. 0% Well written and comprehensive text with good levels of evidence. Average guideline's updating system and stakeholder involvement. 20% Comprehensive text with average levels of evidence. Limited by demography (varying resources in Italy). Good applicabilty. The text has good informations for prehospital care but have average informations about the monitoring criteria. 0% Comprehensive text with good levels of evidence and population focus (pediatrics). Average applicabilty and stakeholder involvement. 0% Well written and comprehensive text with good levels of evidence. Good methods but average applicability. No conflicts of interest but resource-limited enviroment. 20% Bad levels of evidence with unclear methods. Average applicability and stakeholder involviment. 20% Comprehensive text with good levels of evidence and unclear methods. Average guideline's uptodating system and monitoring criteria. The scope is limited though acknowledged and stakeholder involvement. 20% Well written and comprehensive text. Unclear levels of evidence. Poor discussion on implementation and recommendations. Bad methods, applicability and stakeholder involviment. 0% Comprehensive text with good levels of evidence but unclear selection criteria for the evidence. No external review. Good methods and scope. 40% Bad levels of evidence, population remains unclear. Scope is limited by region. Bad applicabilty and methods. 0% Comprehensive text with good levels of evidence. Presence of good facilitators for applicability. Good scope and methods. 60% Comprehensive text with average levels of evidence. Bad methods and applicability. Low content paper. 0% Well written and comprehensive text with average evidence. Good methods. Scope is limited the due age of the population. 0% Well written text. Good levels of evidence, scope and methods. 0% Well written and comprehensive text with good levels of evidence. Good applicability. Average stakeholder involvement and editorial independence. Unclear guideline updating system. 0% Well written and comprehensive text with good levels of evidence. High lenght. 0% Unclear understanding text with unclear levels of evidence. Average guideline updating system. Good scope. Bad stakeholder involvement. 0% Well written and comprehensive text with unclear levels of evidence (do not specify the class of evidence). Average stakeholder involvement. 0% Well written and comprehensive text with good evidence based. Good applicability. No competing interests by contributors. 0% 0% 100% Bad levels of evidence and is outdated. 20% 60% 20% Well writen and comprehensive text 60% 40% 0% Comprehensive text with good levels of eviedence. The paper needs more informations on the monitoring criteria. High lengh text. 20% 20% 60% Well written and comprehensive text. Unclear methods. Bad applicability. 47, 48]. This is likely attributed to the scientific rigor of developing a CPG, which typically involves a highly methodical approach [ 49 ]. In general, the guidelines that were more recently developed or updated, and those that had undergone numerous updates, most consistently demonstrated the highest quality by AGREE II scores. Our analysis indicates an overall improvement in the above domains in the most current CPGs, consistent with other studies [ 14–16, 50 ]. In a 2011 systematic review of CPGs for managing mild TBI by Tavender et al, the NSW 2006 CPGs fared worse in all domains with the exception of domain scope and purpose when compared to our values for the 2011 version of the guidelines [14]. Similarly, another systematic review evaluated an older 2003 version of NIHCE determining an overall assessment score of 66.9 compared to a score of 100.0 on the 2007 version in our study [ 17 ]. It is noteworthy to mention that more recent CPGs have also the advantage of newer and more rigorous evidence-based medicine in addition to the availability of designing guidelines around the AGREE or AGREE II format. Nevertheless, a frequently criticized area in our results, within the rigor of development domain, was the lack of procedures for updating the guidelines for quality improvement (QI). Given the trend toward improved CPG quality with newer revisions, development of a systematic QI procedure may help to ensure quality of future CPGs without the resources required for a full new edition every few years. Older reviews have demonstrated limited Stakeholder involvement in CPG development, a trend that persists in our current review of CPGs [ 14, 15 ]. While there has been progressive improvement in CPG development, the domains of stakeholder involvement, applicability, and editorial independence remain weak, specifically when it comes to piloting interventions, addressing potential costs and barriers to implementation, and auditing for quality improvement. Recent literature suggests that successful implementation of CPGs reduces mortality and morbidity [ 6, 51–54 ], however applicability of guidelines to a given locale based on factors such as availability and cost of resources, provider skills, and population needs and values, are critically important for successful implementation of CPGs in a manner that will improve patient care. Consideration of stakeholder involvement and applicability are imperative considering these domains are intrinsically associated with CPG implementation and translation to other settings such as LMICs. It has been suggested that adaptation of existing CPGs to local situations may be a more valid and cost-effective means of achieving high-quality CPGs worldwide [ 55 ]. However, recent attempts have revealed that this adaptation process remains complex and challenging, requiring careful planning and implementation to avoid additional costly resource utilization [ 56 ]. In particular, when the CPG recommendations require resources not present in a given locale, alternatives to the suggested “optimal” recommendations are required. Given our 12 / 17 findings that most CPGs lacked information on addressing costs and barriers to implementation, such as in resource-poor settings, this demonstrates an important void in existing CPGs. Current attempts at adapting existing CPGs may require local practitioners and CPG developers to return to the primary literature to develop location-specific alternatives that are more practicable in their region or medical center. In addition, further research may be required to develop these potential cost-reducing alternatives and demonstrate efficacy. CPGs and relevance to LMICs As identified in this systematic review most CPGs have been developed in HICs, which makes them of questionable relevance to LMICs, especially for those populations with different costbenefit parameters for medical care. Many hospitals in LMIC, particularly in more rural areas, lack basic intensive or critical care capabilities, specialized staff, or even necessary diagnostic imaging, factors upon which the majority of the highest scoring guidelines rely upon for implementation [ 57 ]. LMICs have potentially greater challenges and barriers to implementation than HICs based on these factors, which need to be addressed to enable a CPG to be useful and beneficial. The few CPGs with high applicability scores were developed by and tailored for HICs [ 34, 35 ], which makes them unlikely to be applicable for use in limited resourced settings and LMIC countries in their current forms. Individual CPGs quality assessment and previous research Comparing our results with other systematic reviews found many similarities. The EAST Evaluation and Management of the Mild Traumatic Brain Injury CPG (EAST, 2012) had similar evaluation of the best and worst domains. Our assessment of the Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and Adolescents (BTF, 2012) was similar to that of Grimmer et al., only differing in the applicability domain, which they assigned a fairly low score of 26.4, compared to our value of 66.7. However, the overall assessment was similar [ 24 ]. We found similar evaluations for domains, overall assessment and recommendations for the CPG from the National Institute for Health and Clinical Excellence (NIHCE, 2007) [ 14, 17, 47, 48, 58 ] and the National Clinical Guideline (SIGN, 2009) [ 14, 58 ] in other studies. We did find some differences in the domains of applicability and editorial independence in the systematic review mild TBI CPGs by Tavender et al. For the ACEP Clinical Policy, we assessed scores of 70 and 91.7 in the fields of applicability and editorial independence, respectively, compared to their findings of 17 and 50 [ 14 ]. The same study, in addition to a 2011 evaluation by Berrigan et al. found respective scores of 36 and 50 in the applicability domain for the New Zealand Guidelines Group, whereas our score was much higher at 82.5 [ 14, 47 ]. Given the limited number of systematic reviews on this topic, we were not able to compare all CPGs included in this study. Limitations. The main limitations of our study are the subjectiveness of the AGREE II tool and the potential bias of the reviewers performing the assessment. The AGREE II tool is a 23 question tool established to evaluate CPG quality. While it is a subjective tool, it is the current gold standard; the AGREE II guidelines suggest using at least two and preferably four appraisers with content specific knowledge [ 17 ]. We utilized five appraisers who all had content specific knowledge; four were emergency medicine physicians with experience in research and evidence-based methods and one was a masters level student with extensive research and content topic experience. Given that the AGREE II assessment requires evaluation of the CPGs based on the descriptions available in the published manuscripts, there is a small chance that 13 / 17 inaccurate assessments would be due to poor descriptions in the manuscript. However, comparison of our AGREE II ratings to those of other researchers found similar scores [ 14 ]. Expanding the reviewers to include other relevant specialties such as neurosurgery, critical care, or neurology would have provided additional input on specialty-specific recommendations. Additionally, due to language limitations, we were only able to review CPGs in English. The language limitation did not provide a significant barrier to most CPGs, given that many of the articles written in different languages were also available in English, or failed to meet inclusion criteria. However, given that many LMICs are non-English speaking, they may have developed CPGs in other languages that we were unable to evaluate and subsequently missed. Finally, we did not search some clinical databases, like the TRIP and GIN repository. However, due to the several diverse repositories included through the Duke library search, it is unlikely that we missed published CPGs. Conclusions. Our review identifies two specific areas for improvement in clinical practice guidelines addressing the acute management of TBI: (a) the domains of stakeholder involvement, applicability, and editorial independence remain weak and insufficiently described specifically when it comes to piloting interventions, addressing potential costs and implementation barriers, and auditing for quality improvement; (b) CPGs created specifically for use in low income settings are non-existent. Most of these CPGs were developed by high-income countries with only one CPG from an upper middle income country, which was found to have a poorer quality across all domains. This will limit the applicability and implementation capacity of CPGs for limited resourced settings. Supporting Information S1 Appendix. Search strategy for PubMed. (DOCX) S2 Appendix. PRISMA 2009 Checklist. (DOCX) Author Contributions Conceived and designed the experiments: JRNV SE CJG CAS. Performed the experiments: AP MMCV JA NR TT KT. Analyzed the data: AP MMCV JA NR TT KT JRNV SE CAS. Contributed reagents/materials/analysis tools: MMCV JRNV. Wrote the paper: AP MMCV JRNV SE CJG CAS. 14 / 17 15 / 17 16 / 17 1. Hyder A , Wunderlich C , Puvanachandra P , Gururaj G , Kobusingye O. The impact of traumatic brain injuries: a global perspective . NeuroRehabilitation . 2007 ; 22 ( 5 ): 341 - 353 . PMID: 18162698 2. Bruns J , Hauser W. The Epidemiology of Traumatic Brain Injury: A Review . Epilepsia . 2003 ; 44 : 2 - 10 . 3. Corrigan J , Selassie A , Orman J . The Epidemiology of Traumatic Brain Injury . Journal of Head Trauma Rehabilitation . 2010 ; 25 ( 2 ): 72 - 80 . doi: 10 .1097/HTR.0b013e3181ccc8b4 PMID: 20234226 4. Global status report on road safety 2013 Supporting a decade of action [Internet] . 1st ed. Geneva: Margaret Chan; 2013 [cited 18 April 2016 ]. Available: http://www.who.int/violence_injury_prevention/road_ safety_status/2013/en/. 5. Bulger E , Nathens A , Rivara F , Moore M , MacKenzie E , Jurkovich G . Management of severe head injury: Institutional variations in care and effect on outcome* . Critical Care Medicine . 2002 ; 30 ( 8 ): 1870 - 1876 . PMID: 12163808 6. Arabi Y , Haddad S , Tamim H , Al-Dawood A , Al-Qahtani S , Ferayan A , et al. Mortality reduction after implementing a clinical practice guidelines-based management protocol for severe traumatic brain injury . Journal of Critical Care . 2010 ; 25 ( 2 ): 190 - 195 . doi: 10 .1016/j.jcrc. 2009 . 05 .004 PMID: 19592201 7. Grol R , Grimshaw J . From best evidence to best practice: effective implementation of change in patients' care . The Lancet . 2003 ; 362 ( 9391 ): 1225 - 1230 . 8. Gupta D , Sharma D , Kannan N , Prapruettham S , Mock C , Wang J , et al. Guideline Adherence and Outcomes in Severe Adult Traumatic Brain Injury for the CHIRAG (Collaborative Head Injury and Guidelines) Study . World Neurosurgery. 2016 ; 89 : 169 - 179 . doi: 10 .1016/j.wneu. 2015 . 12 .097 PMID: 26806065 9. O 'Lynnger T , Shannon C , Le T , Greeno A , Chung D , Lamb F , et al. Standardizing ICU management of pediatric traumatic brain injury is associated with improved outcomes at discharge . Journal of Neurosurgery: Pediatrics . 2016 ; 17 ( 1 ): 19 - 26 . doi: 10 .3171/ 2015 .5. PEDS1544 PMID : 26451717 10. Alexander T , Fuller G , HArgovan P , Clarke D , Muckart D , Thomson S. An audit of the quality of care of traumatic brain injury at a busy regional hospital in South Africa . S Afr J Surg . 2009 ; 47 ( 4 ): 1202 , 124 - 6 . 11. Bullock R , Chesnut R , Clifton G , Marion D , Narayan R , Newell D , et al. Guidelines for the management of severe head injury . Eur J Emerg Med . 1996 ; 3 ( 2 ): 109 - 27 . PMID: 9028756 12. The Brain Trauma Foundation . The American Association of Neurological Surgeons. The Joint Section on Neurotrauma and Critical Care. Initial management . J Neurotrauma . 2000 ; 17 ( 6-7 ): 463 - 469 . PMID: 10937888 13. Carson S , Davis-O'Reilly C , Drexel P , Fu R , Norris S , Pappas M , et al. Guidelines for the Management of Severe Traumatic Brain Injury: Introduction . Journal of Neurotrauma , 2007 . 24 : p. S1 - S106 . J Neurotraum. 2016 ; 24 ( sup1 ): 1 - 116 . 14. Tavender E , Bosch M , Green S , O'Connor D , Pitt V , Phillips K , et al. Quality and Consistency of Guidelines for the Management of Mild Traumatic Brain Injury in the Emergency Department . Academic Emergency Medicine. 2011 ; 18 ( 8 ): 880 - 889 . doi: 10 .1111/j.1553- 2712 . 2011 . 01134 . x PMID : 21843224 15. Alarcon J , Rubiano A , Chirinos M , Valderrama A , Gich I , Bonfill X et al. Clinical practice guidelines for the care of patients with severe traumatic brain injury . Journal of Trauma and Acute Care Surgery . 2013 ; 75 ( 2 ): 311 - 319 . doi: 10 .1097/TA.0b013e3182924bf8 PMID: 23887565 16. Rusnak M , Mauritz W , Lecky F , Kaniasnky M , Brazinova A . Evaluation of traumatic brain injury guidelines using AGREE instrument . Bratisl Lek Listy . 2008 ; 109 ( 8 ): 374 - 380 . PMID: 18837249 17. Brouwers M , Kho M , Browman G , Burgers J , Cluzeau F , Feder G , et al. AGREE II: advancing guideline development, reporting and evaluation in health care . Canadian Medical Association Journal . 2010 ; 182 ( 18 ): E839 - E842 . doi: 10 .1503/cmaj.090449 PMID: 20603348 18. Moher D , Liberati A , Tetzlaff J , Altman D . Preferred Reporting Items for Systematic Reviews and MetaAnalyses: The PRISMA Statement . PLoS Med . 2009 ; 6 ( 7 ):e1000097. doi: 10.1371/journal.pmed. 1000097 PMID: 19621072 19. Search-Clinical Tools-LibGuides at Duke University Medical Center [Internet]. Guides.mclibrary. duke.edu . 2016 [cited 18 April 2016 ]. Available: http://guides.mclibrary.duke.edu/clinicaltools. 20. Vlayen J , Aertgeerts B , Hannes K , Sermeus W , Ramaekers D. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit . International Journal for Quality in Health Care . 2005 ; 17 ( 3 ): 235 - 242 . PMID: 15743883 21. World Bank Country and Lending Groups-World Bank Data Help Desk [Internet]. Datahelpdesk.worldbank.org . 2016 [cited 1 July 2016 ]. Available: https://datahelpdesk.worldbank.org/knowledgebase/ articles/906519. 22. The R Project for Statistical Computing , Version 3.2 .1. Viena , Austria: R Foundation for Statistical Computing; 2015 . 23. Schutzman S , Barnes P , Duhaime A , Greenes D , Homer C , Jaffe D , et al. Evaluation and Management of Children Younger Than Two Years Old With Apparently Minor Head Trauma: Proposed Guidelines . PEDIATRICS . 2001 ; 107 ( 5 ): 983 - 993 . PMID: 11331675 24. The management of minor closed head injury in children . Committee on Quality Improvement , American Academy of Pediatrics. Commission on Clinical Policies and Research , American Academy of Family Physicians. Pediatrics . 1999 ; 104 ( 6 ): 1407 - 1415 . PMID: 10585999 25. Jagoda A , Bazarian J , Bruns J , Cantrill S , Gean A , Howard P , et al. Clinical Policy: Neuroimaging and Decisionmaking in Adult Mild Traumatic Brain Injury in the Acute Setting . Journal of Emergency Nursing . 2009 ; 35 ( 2 ): e5 - e40 . doi: 10 .1016/j.jen. 2008 . 12 .010 PMID: 19285163 26. Badjatia N , Carney N , Crocco T , Fallat M , Hennes H , Jagoda A , et al. Guidelines for Prehospital Management of Traumatic Brain Injury 2nd Edition . Prehospital Emergency Care. 2008 ; 12 ( sup1 ): S1 - S52 . 27. Kochanek P , Carney N , Adelson P , Ashwal S , Bell M , Bratton S , et al. Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and Adolescents-Second Edition . Pediatric Critical Care Medicine . 2012 ; 13 ( Suppl1 ): S1 - S2 . 28. Guidelines for the management of severe traumatic brain injury . J Neurotrauma . 2007 ; 24 ( Suppl1 ): s1 - 106 . 29. Kamerling S , Lutz N , Posner J , Vanore M. Mild Traumatic Brain Injury In Children: Practice Guidelines For Emergency Department and Hospitalized Patients . Pediatric Emergency Care . 2003 ; 19 ( 6 ): 431 - 440 . PMID: 14676497 30. Hebb M , Clarke D , Tallon J . Development of a provincial guideline for the acute assessment and management of adult and pediatric patients with head injuries . Can J Surg . 2007 ; 50 ( 3 ): 187 - 194 . PMID: 17568490 31. Cushman J , Agarwal N , Fabian T , Garcia V , Nagy K , Pasquale M , et al. Practice Management Guidelines for the Management of Mild Traumatic Brain Injury: The EAST Practice Management Guidelines Work Group . The Journal of Trauma: Injury , Infection, and Critical Care . 2001 ; 51 ( 5 ): 1016 - 1026 . 32. Barbosa R , Jawa R , Watters J , Knight J , Kerwin A , Winston E , et al. Evaluation and management of mild traumatic brain injury . Journal of Trauma and Acute Care Surgery . 2012 ; 73 : S307 - S314 . doi: 10 . 1097/TA.0b013e3182701885 PMID: 23114486 33. Maas A , Dearden M , Teasdale G , Braakman R , Cohadon F , Iannotti F , et al. EBIC-Guidelines for management of severe head injury in adults . Acta neurochir . 1997 ; 139 ( 4 ): 286 - 294 . PMID: 9202767 34. Vos P , Battistin L , Birbamer G , Gerstenbrand F , Potapov A , Prevec T et al. EFNS guideline on mild traumatic brain injury: report of an EFNS task force . Eur J Neurol . 2002 ; 9 ( 3 ): 207 - 219 . PMID: 11985628 35. Piek J . Guidelines for the pre-hospital care of patients with severe head injuries . Intensive Care Medicine . 1998 ; 24 ( 11 ): 1221 - 1225 . PMID: 9876987 36. Kochanek PM , Carney N , David Adelson P , Ashwal S , Bell MJ , Bratton S , et al. Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and AdolescentsSecond Edition . Pediatric Critical Care Medicine . 2012 ; 13 : S1 - S2 . doi: 10 .1097/PCC. 0b013e31823f435c PMID: 22217782 37. Head Injury Triage, assessment, investigation and early management of head injury in inants, children and adults [Internet] . 1st ed. London: National Institute for Health and Clinical Excellence: Guidance.; 2016 [cited 18 April 2016 ]. Available: http://www.ncbi.nlm.nih.gov/books/NBK53036/pdf/Bookshelf_ NBK53036.pdf. 38. NSW Ministry of Health. Adult Trauma Clinical Practice Guidelines Initial Management of Closed Head Injury in Adults 2nd Edition . Sydney: NSW Institute of Trauma and Injury Management; 2007 p. 1 - 129 . 39. Traumatic Brain Injury: Diagnosis, Acute Management and Rehabilitation [Internet]. 1st ed. Wellington: New Zealand Guidelines Group; 2006 [cited 18 April 2016 ]. Available: http://www.acc.co.nz/PRD_ EXT_CSMP/groups/external_communications/documents/guide/wim2_ 059414 .pdf. 40. Rusticali B , Villani R . Treatment of minor and severe traumatic brain injury. National reference guidelines . Minerva Anestesiol . 2008 ; 74 ( 10 ): 583 - 616 . PMID: 18854800 41. Undén J , Ingebrigtsen T , Romner B . Scandinavian guidelines for initial management of minimal, mild and moderate head injuries in adults: an evidence and consensus-based update . BMC Medicine . 2013 ; 11 ( 1 ): 50 . 42. Early management of patients with a head injury A national clinical guideline [Internet] . 1st ed. Edinburgh: Scottish Intercollegiate Guidelines Network; 2009 [cited 18 April 2016 ]. Available: http://www. sign.ac.uk/pdf/sign110.pdf. 43. Guidelines for minor head injured patients' management in adult age . The Study Group on Head Injury of the Italian Society for Neurosurgery. J Neutosurg Sci . 1996 ; 40 ( 1 ): 11 - 15 . 44. Procaccio F , Stocchetti N , Citerio G , Bernadino M , Beretta L , Della Corte F , et al. Guidelines for the treatment of adults with severe head trauma (part I) . Initial assessment; evaluation and pre-hospital treatment; current criteria for hospital admission; systemic and cerebral monitoring . J Neurosurg Sci . 2000 ; 44 ( 1 ): 1 - 10 . PMID: 10961490 45. Liao K , Chang C , Chang H , Chang K , Chen C , Chen T , et al. Clinical practice guidelines in severe traumatic brain injury in Taiwan . Surgical Neurology . 2009 ; 72 : S66 - S73 . doi: 10 .1016/j.surneu. 2009 . 07 . 004 PMID: 19818476 46. de Andrade AF , Marino R , Ciquini O , Gadelha Figueiredo EG , Machado AG . Guidelines for Neurosurgical Trauma in Brazil . World Journal of Surgery . 2001 ; 25 ( 9 ): 1186 - 1201 . PMID: 11571957 47. Berrigan L , Marshall S , McCullagh S , Velikonja D , Bayley M. Quality of clinical practice guidelines for persons who have sustained mild traumatic brain injury . Brain Injury . 2011 ; 25 ( 7-8 ): 742 - 751 . doi: 10 . 3109/02699052. 2011 .580317 PMID: 21604931 48. Grimmer K , Dizon J , Milanese S , King E , Beaton K , Thorpe O , et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool . BMC Med Res Methodol . 2014 ; 14 ( 1 ): 63 . 49. Qaseem A. The Development of Clinical Practice Guidelines and Guidance Statements of the American College of Physicians: Summary of Methods . Annals of Internal Medicine . 2010 ; 153 ( 3 ): 194 . doi: 10 .7326/ 0003 -4819-153-3- 201008030 -00010 PMID: 20679562 50. Peloso P , Carroll L , Cassidy J , Borg J , von Holst H , Holm L , et al. Critical evaluation of the existing guidelines on mild traumatic brain injury . Journal of Rehabilitation Medicine . 2004 ; 36 ( 0 ): 106 - 112 . 51. Lee J , Rittenhouse K , Bupp K , Gross B , Rogers A , Rogers F , et al. An analysis of Brain Trauma Foundation traumatic brain injury guideline compliance and patient outcome . Injury . 2015 ; 46 ( 5 ): 854 - 858 . doi: 10 .1016/j.injury. 2014 . 12 .023 PMID: 25661105 52. Fakhry S , Trask A , Waller M , Watts D. Management of Brain-Injured Patients by an Evidence-Based Medicine Protocol Improves Outcomes and Decreases Hospital Charges . The Journal of Trauma: Injury , Infection, and Critical Care . 2004 ; 56 ( 3 ): 492 - 500 . 53. Griesdale D , Örtenwall V , Norena M , Wong H , Sekhon M , Kolmodin L , et al. Adherence to guidelines for management of cerebral perfusion pressure and outcome in patients who have severe traumatic brain injury . Journal of Critical Care . 2015 ; 30 ( 1 ): 111 - 115 . doi: 10 .1016/j.jcrc. 2014 . 07 .026 PMID: 25179411 54. Vukić M , Negovetić L , Kovač D , Ghajar J , Glavić Z , Gopčević A . The Effect of Implementation of Guidelines for the Management of Severe Head Injury on Patient Treatment and Outcome . Acta Neurochirurgica . 1999 ; 141 ( 11 ): 1203 - 1208 . PMID: 10592121 55. Fervers B , Burgers J , Haugh M , Latreille J , Mika-Cabanne N , Paquete L , et al. Adaptation of clinical guidelines: literature review and proposition for a framework and procedure . International Journal for Quality in Health Care . 2006 ; 18 ( 3 ): 167 - 176 . PMID: 16766601 56. Harrison M , Graham I , van den Hoek J , Dogherty E , Carley M , Angus V . Guideline adaptation and implementation planning: a prospective observational study . Implementation Science . 2013 ; 8 ( 1 ): 49 . 57. Dries D. Defining the Need for Critical Care in Underserved Areas [Internet] . Society of Critical Care Medicine . 2016 [cited 18 April 2016 ]. Available: http://www.sccm.org/Communications/CriticalConnections/Archives/Pages/ Defining-the-Need-for-Critical-Care-in-Underserved-Areas.aspx . 58. Wong J , Côté P , Shearer H , Carroll L , Yu H , Varatharajan S , et al. Clinical practice guidelines for the management of conditions related to traffic collisions: a systematic review by the OPTIMa Collaboration . Disability and Rehabilitation . 2014 ; 37 ( 6 ): 471 - 489 . doi: 10 .3109/09638288. 2014 .932448 PMID: 24963833


This is a preview of a remote PDF: http://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0161554&type=printable

Anjni Patel, Mateus Mazorra Coelho Vieira, John Abraham, Nick Reid, Tu Tran, Kevin Tomecsek, João Ricardo N. Vissoci, Stephanie Eucker, Charles J. Gerardo, Catherine A. Staton. Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review, PLOS ONE, 2016, DOI: 10.1371/journal.pone.0161554