Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review
Quality of the Development of Traumatic Brain Injury Clinical Practice Guidelines: A Systematic Review
Anjni Patel 0 1
Mateus Mazorra Coelho Vieira 1
John Abraham 0 1
Nick Reid 0 1
Tu Tran 1
Kevin Tomecsek 0 1
João Ricardo N. Vissoci 1
Stephanie Eucker 0 1
Charles J. Gerardo 0 1
Catherine A. Staton 0 1
0 Division of Emergency Medicine, Department of Surgery, Duke University Medical Center , Durham , North Carolina, United States of America, 2 Centro Universitário Unicesumar, Maringá, Brazil, 3 Duke Global Health Institute, Duke University , Durham , North Carolina, United States of America, 4 Division of Global Neuroscience and Neurosurgery, Department of Neurosurgery, Duke University Medical Center , Durham , North Carolina, United States of America , 5 Faculdade Inga , Medicine Department , Maringá, 87005240 , Brazil
1 Editor: Johannes Boltze, Fraunhofer Research Institution of Marine Biotechnology , GERMANY
Traumatic brain injury (TBI) is a leading cause of death worldwide and is increasing exponentially particularly in low and middle income countries (LMIC). To inform the development of a standard Clinical Practice Guideline (CPG) for the acute management of TBI that can be implemented specifically for limited resource settings, we conducted a systematic review to identify and assess the quality of all currently available CPGs on acute TBI using the AGREE II instrument. In accordance with PRISMA guidelines, from April 2013 to December 2015 we searched MEDLINE, EMBASE, Google Scholar and the Duke University Medical Center Library Guidelines for peer-reviewed published Clinical Practice Guidelines on the acute management of TBI (less than 24 hours), for any level of traumatic brain injury in both high and low income settings. A comprehensive reference and citation analysis was performed. CPGs found were assessed using the AGREE II instrument by five independent reviewers and scores were aggregated and reported in percentage of total possible score. An initial 2742 articles were evaluated with an additional 98 articles from the citation and reference analysis, yielding 273 full texts examined. A total of 24 final CPGs were included, of which 23 were from high income countries (HIC) and 1 from LMIC. Based on the AGREE II instrument, the best score on overall assessment was 100.0 for the CPG from the National Institute for Health and Clinical Excellence (NIHCE, 2007), followed by the New Zealand Guidelines Group (NZ, 2006) and the National Clinical Guideline (SIGN, 2009) both with a score of 96.7. The CPG from a LMIC had lower scores than CPGs from higher income settings. Our study identified and evaluated 24 CPGs with the highest scores in clarity and presentation, scope and purpose, and rigor of development. Most of these CPGs were developed in HICs, with limited applicability or utility for resource limited settings. Stakeholder involvement, Applicability, and Editorial independence remain
Data Availability Statement: All relevant data are
within the paper and its Supporting Information files.
Funding: The authors received no research funding
for this work. Dr. Staton would like to acknowledge
salary support funding from the Fogarty International
Center (Staton, K01 TW010000-01A1).
Competing Interests: The authors have declared
that no competing interests exist.
weak and insufficiently described specifically with piloting, addressing potential costs and
implementation barriers, and auditing for quality improvement.
Traumatic brain injury (TBI) is one of the leading causes of death and disability in both
developing and developed countries, with the highest incidences among young people less than 30
years of age [
]. While the current global burden is unknown, previous conservative
estimates indicate an annual incidence of over 10 million people sustaining a TBI leading to
hospitalization or death, with road traffic injuries causing a preponderance of cases [
incidence is projected to continue to rise worldwide due to the continued increasing rates of
road traffic injuries, particularly in low and middle-income countries (LMIC) where the rates
are twice as high as in high-income countries [
]. Furthermore, the World Health
Organization suggests that upwards of 90% of road traffic injury deaths occur in LMIC. These trends
have been attributed to the rapid economic growth, urbanization, and motorization but limited
infrastructure improvements in LMIC [
Unfortunately, as the burden of TBI continues to increase globally, appropriate prevention
efforts have been limited, especially in LMIC, and healthcare quality remains poor, resulting in
disproportionately higher mortality rates [
]. One healthcare quality improvement measure
that has shown impact is the use of clinical practice guidelines (CPGs) [
]. In fact, in the last
several years a significant number of CPGs for acute TBI care has been developed worldwide
. However, CPGs vary in quality and comprehensiveness, leading to difficulties with
standardization of care, adaptation and implementation, particularly in resource limited settings
]. There is limited literature comparing and evaluating the strengths and weaknesses of all
available CPGs for the treatment of acute TBI.
The Brain Trauma Foundation first developed CPGs for the management of severe TBI in
the United States in 1995, with subsequent updated editions published in 2000 and 2007 [
]. These guidelines have gradually gained acceptance and increased use internationally, with
a number of countries adapting them to their individual needs. For example, until recently in
Saudi Arabia, patients with severe TBI were managed per individual provider knowledge and
experience. By implementing an ICU protocol derived from the Brain Trauma Foundation’s
guidelines, Saudi Arabian providers were able to significantly reduce hospital and ICU
mortality due to TBI [
Despite this, a major criticism of the Brain Trauma Foundation CPGs is that they may not
be appropriate for use in all locales due to differences in available resources. Subsequently, a
number of newer CPGs have been developed in many different countries and by various
practitioner groups based on data and capabilities specific to their respective practice environments
]. Applying these CPGs in LMICs with limited resources is challenging. Additionally,
these CPGs vary in quality and content, span across multiple disciplines, and are published in
disparate literature bases, making effective utilization challenging. Previous studies evaluating
quality of existing TBI CPGs, for instance, have focused on subsets of TBI severity such as mild
TBI only [
], or review only a limited number of CPGs .
We aimed to assess and summarize the quality of all currently available international acute
TBI CPGs by conducting a systematic review using the Appraisal of Guidelines for Research
and Evaluation (AGREE) II instrument [
]. We also compared the quality of CPGs created in
high resource countries with those from low and middle income. We expect that these results
2 / 17
will inform the development of a standard CPG for the acute management of TBI that can be
implemented specifically for limited resource settings.
Materials and Methods
Protocol and Registration
This systematic review is reported in accordance with the Preferred Reporting Items for
Systematic Review and Meta-Analyses (PRISMA) Statement [
], and is registered in the
PROSPERO database (International Prospective Register of Systematic Reviews) under the number
Articles mentioning clinical practice guidelines or recommendations for traumatic brain
injury, which met the following inclusion criteria were considered: acute management of TBI
(less than 24 hours), any level of traumatic brain injury, high and low income countries and
publication in English. No CPG population age restrictions were added as we chose to evaluate
both pediatric and adult CPGs. For multiple versions of CPGs, only the newest CPGs were
included in the analysis and the older versions were excluded.
We employed an extensive search strategy from April 2013 to December 2015 to identify
guidelines from the following electronic databases: MEDLINE, EMBASE and the Duke
University Medical Center Guidelines located in the Duke University Medical Center Library and
]. The Duke University Medical Center Guidelines is an electronic repository
curated by Duke University which gathers several national sources indexing CPGs, including
the following: National Guideline Clearinghouse, Center for Disease Control Guidelines,
Cochrane Library, AHRQ Evidence reports, American College of Emergency Medicine CPGs,
American Academy of Child & Adolescent Psychiatry, American Psychiatric Association
Guidelines, Canadian Medical Association, Health Services/Technology Assessment Text
Collection, Infectious Disease Society of America, Practice guidelines, Joint National Committee,
National Comprehensive Cancer Network Guidelines, National Institute for for Health and
Clinical Excellence guidelines, NIH Consensus statements archive, US Preventative Services
Task Force, Veterans Affairs Clinical Practice Guidelines. In addition, we manually searched
the references and performed a citation analysis of the included studies using Google Scholar
to include any potential CPG document that was not included in the initial steps.
The initial search comprised of the MeSH terms "Brain Injuries", "Guideline [Publication
Type]”, “Practice Guideline [Publication Type]”, “Practice Guidelines as Topic”, and their
respectives entry terms. Appendix 1 presents the search strategy used in the PubMed database.
We did not use limits for languages or date when searching the databases, but added to the
final list only documents in English.
Titles and abstract of the retrieved articles were independently evaluated by two reviewers (A.
P. and J.A.). Abstracts that did not provide enough information regarding the eligibility criteria
were retrieved for full-text evaluation. Reviewers (A.P. and J.A.) independently evaluated
fulltext articles and determined study eligibility. Disagreements were solved by consensus and if
3 / 17
disagreement persisted, a third reviewer’s opinion was sought (C.S). For CPGs with more than
one version, we reported only the most recent versions given their large overlap and anticipated
improvement in quality.
Quality of Clinical Practice Guidelines
Five appraisers independently assessed each eligible and selected guideline for quality in
accordance with the AGREE II instrument [
]. AGREE II is an instrument designed to assess
quality of clinical practice guidelines. It consists of 23 items divided into six domains: 1) scope
and purpose, 2) stakeholder involvement, 3) rigor of development, 4) clarity of presentation, 5)
applicability, and 6) editorial independence. Domain one, scope and purpose, evaluates
specificity of the overall objective, clinical questions and the patient population described in the
CPG. Domain two measures stakeholder involvement, inclusion of relevant professional
groups, patients’ views and preferences, and target users. Domain three evaluates the
guideline’s systematic methods, stating criteria for selecting evidence and explicit links between
evidence and recommendations, strengths and limitations of the evidence, consideration of risks
and benefits, external review prior to publication, and procedures for updating the guideline.
The fourth domain, clarity of presentation, evaluates for clarity, lack of ambiguity, determines
if different management options are presented, and assesses whether key recommendations are
easily identifiable. Domain five assesses if the CPGs address recommendations for clinical
application, barriers to application, cost or resource implications, and monitoring criteria.
Finally, domain six evaluates for editorial independence from funding bodies and addressing of
potential conflicts of interest.
Each item was scored on a seven-point scale (one = strongly disagree and seven = strongly
agree). Scores for each domain were calculated by using the sum of all items within a domain
and scaling the score as a percentage of the maximum possible score using the following
Maximum Possible Score
Minimum Possible Score Minimum Possible Score
The results from each guideline were summarized in a heatmap visualization with values for
each domain. All 23 items of the AGREE-II instrument were assessed with results reported in
percentage form for each of the six domains. A value of 100% indicated a domain in which all
items were scored with seven points (strongly agree). A value of 0% corresponded to a domain
in which all items were scored with one point (strongly disagree).
Each reviewer received a user’s manual of the AGREE II instrument, containing its
instructions. The six AGREE domains were reported independently for each included CPG.
Additionally, an independent global assessment was conducted from the six domains and reported
along with the appraiser's recommendation. Recommendations were measured in a
threeoption scale (“Yes”, “Yes with modifications”, and “No”), with a qualitative comment about the
]. For each domain and for the global assessment, we report average values with
respective standard deviations and range (minimum and maximum values).
Two reviewers (A.P. and J.A.) independently conducted the data extraction and disagreements
were resolved by a third reviewer (C.S.). Besides the AGREE assessment previously described,
the following general characteristics of the studies were collected: year of publication, location
where the guideline creation took place, the organization that created the guidelines, the main
focus of the guideline, the patient population of the guideline, and the severity of TBI the
4 / 17
guideline was addressing. Additionally, the countries of origin for each guideline were classified
into high-, middle-, and low-income based on World Bank definitions [
]. The senior
researcher (C.S.) moderated all discrepancies or doubts during the rating process.
Data analysis was performed descriptively and with graphical representation. Overall
assessments for each domain were calculated following the methods already described [
Consistency of evaluations of the AGREE II domain and for the overall assessment was calculated
with an intra-class correlation coefficient (ICC). Graphical solutions were carried out with R
software for statistical language .
The initial search strategy identified 2,742 titles and abstracts 80 of which were removed for
duplicates. From these, 2487 were excluded after reviewing abstracts. A reference and citation
analysis was performed on the remaining 175 articles yielding an additional 98 abstracts. Full
text analysis was then performed on a total of 273 articles of which only 24 [
inclusion criteria (Fig 1).
Overall, there were 24 guidelines that were included in this analysis (Table 1) representing 19
different organizations and spanning several countries on four continents. Of these 24 CPG's,
] were developed in high-income countries and only one [
] from a upper
middleincome country (Brazil). The CPGs evaluated covered the full scope of adult and pediatric
populations with four covering pediatric patients [
23, 24, 27, 29
], eight for adults patients [
33, 38, 40, 41, 43, 44
], and seven covering both populations [
26, 30, 34, 36, 37, 39, 42
]. Five did
not specify their population group [
31, 32, 35, 45, 46
]. Regarding the severity of TBI, one third
of the CPGs were developed for minor or mild TBI [
23, 24, 25, 29, 31, 32, 34, 33
], another third
covered severe TBI [
26, 27, 28, 33, 35, 36, 37, 44, 45
] and the rest were developed for all levels
of TBI severity [
30, 37, 38, 39, 40, 41, 42
]. The majority (19) of CPGs focused on the early
management of TBI [
23, 24, 25, 27, 29, 30, 31, 32, 34, 38, 39, 40, 41, 42, 43, 44, 46
], with two
focusing specifically on prehospital care [
], another two on both early management and ICU
], one covered prehospital, early management, and rehabilitation , and one
covered the entire breadth of management of severe TBI [
]. Of the 24 assessed CPGs, roughly
half (11) were developed by professional organizations [
23, 24, 25, 30, 31, 32, 35, 36, 37, 43,
], four were developed by non-profit organizations [
26, 27, 39, 42
], three by international
41, 33, 34
], another three by national institutes or government organizations [
], one from an academic organization , one did not specified the type of organization
], and the remainder of the CPGs were developed by mixture of different organizations: a
non-profit professional organization [
] and a professional organization with an academic
CPG Quality Assessment (AGREE)
All AGREE II assessments summaries (average per domain and standard deviation) are
described per domain and with the overall assessment. Domain specific results are summarized
in Fig 2. Overall assessment recommendations and comments are displayed in Table 2.
Interrater reliability for each domain is expressed in Table 3.
5 / 17
Fig 1. Study Flow Diagram.
Domain One—Scope and Purpose. The lowest score was 55.9 which was from Guidelines
for the Pre-hospital Care of Patients with Severe Head Injuries (ESICM, 1998). The highest
score was 86.9, from both the Adult Trauma Clinical Practice Guidelines, Initial Management
of Closed Head Injury in Adults (NSW MoH, 2011) and the ACEP Clinical Policy:
Neuroimaging and Decisionmaking in Adult Mild Traumatic Brain Injury in the Acute Setting (ACEP,
Domain Two—Stakeholder Involvement. The highest score of 86.9 was from the Early
Management of Patients with a Head Injury; A National Clinical Guideline. (SIGN, 2009) The
lowest score of 41.9 was from The Study Group on Head Injury of the Italian Society for
Neurosurgery: Guidelines for minor head injured patients' management in adult age (SINch, 1996).
Domain Three—Rigor of Development. The lowest score was 28.3 from The Study
Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for minor head
injured patients' management in adult age (SINch, 1996). The highest score of 93.3 was from
the New Zealand Guidelines Group: Traumatic Brain Injury: Diagnosis, Acute Management
and Rehabilitation (NZ, 2006).
Domain Four—Clarity of Presentation. The lowest score was 64.4 from the Guidelines
for Neurosurgical Trauma in Brazil (USP/BSN, 2001) and the highest value was 95.6 achieved
by two guidelines: Early Management of Patients with a Head Injury: A National Clinical
Guideline (SIGN, 2009) and the Guidelines for the Acute Medical Management of Severe
Traumatic Brain Injury in Infants, Children, and Adolescents (BTF, 2012).
Domain Five–Application. Two guidelines shared the lowest score (38.3): the Study
Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for Minor Head
Injured Patients’ Management in Adult Age (SINch, 1996) and Guidelines for the Pre-hospital
care of Patients with Severe Head Injuries (ESICM, 1998). The highest score of 95.0 belonged
to the following guideline: Early Management of Patients with a Head Injury. A National
Clinical Guideline (SIGN, 2009).
Domain Six—Editorial Independence. The lowest score of 40 points came from both the
Study Group on Head Injury of the Italian Society for Neurosurgery: Guidelines for minor
6 / 17
7 / 17
8 / 17
head injured patients' management in adult age (SINch, 1996) and the Guidelines for
Neurosurgical Trauma in Brazil (USP/BSN, 2001). The highest score was 93.3, from the New Zealand
Guidelines Group: Traumatic Brain Injury: Diagnosis, Acute Management and Rehabilitation
Overall Assessment. The 2007 CPG from the National Institute for Health and Clinical
Excellence (NIHCE, 2007) had the best overall assessment with maximum score from all
appraisers, followed by the New Zealand Guidelines Group (NZ, 2006) and the National
Clinical Guideline (SIGN, 2009) both with only one appraiser scoring six points each. The worst
ovserall assessment was for the CPG from The Study Group on Head Injury of the Italian
Society for Neurosurgery (SINch, 1996). Most CPGs would be recommend by the appraisers
(EFNS, 2001; AAP, 2001; NZ, 2006; BTF/AANS, 2007; NIHCE, 2007; ACEP, 2009; SIGN,
* Clinical Practice Guidelines have older versions, only most recent version of this Clincal Practice Guidelines was included
9 / 17
Fig 2. AGREE Scoring by domain for each Clinical Practice Guidelines. (CPG = Clinical Pratice
Guideline; D1 = Domain one, scope and porpouse; D2 = Domain 2, stakeholder involvement; D3 = Domain
three, rigor of development; D4 = Domain four, clarity of presentation; D5 = Domain five, applicability;
D6 = Domain six, editorial Independence, * = Indicates newest version of a CPG for which multiple versions
2009; NSW MoH, 2011). Three CPGs had 60% of appraisers recommending with
modifications (SINch/SIAARTI, 2000; EAST, 2002; CHOP, 2003). The guidelines from the Study
Group on Head Injury of the Italian Society for Neurosurgery (SINch, 1996) was the only CPG
that 100% of reviewers reported they would not recommend for use. Comments on the
strengths and weaknesses of each CPG are summarized in Table 2.
Appraisers Consistency. Overall reliability was very good, with three of the quality
domains (stakeholder involvement, rigor of development and editorial independence) and the
overall assessment with values above 0.70, and two (scope and purpose and applicability)
values around 0.65. The domain “clarity and presentation” had the lowest reliability (0.43)
This systematic review is the first to synthesize and collate all published clinical practice
guidelines for all types of traumatic brain injury into a single large-scale quality review. Our study
included 24 CPGs; across all TBI CPGs, the highest mean scores were achieved in clarity and
presentation, scope and purpose and rigour of development, while the main weaknesses across
CPGs were stakeholder involvement, applicability and editorial independence. The National
Institute for Health and Clinical Excellence (NIHCE, 2007) guidelines, the New Zealand
Guidelines Group (NZ, 2006), and the National Clinical Guideline (SIGN, 2009) were the three
CPGs with best results. The majority of CPGs evaluated in this study were developed by high
income countries (HICs), and are therefore minimally applicable in resource limited settings.
CPGs strengths and weaknesses
Overall, the strong scores in the clarity and presentation, scope and purpose, and rigor of
development domains have been reported in other systematic reviews evaluating TBI CPGs [14, 17,
10 / 17
A1 A2 A3 A4 A5
% of CPG
(CPGs = Clinical Pratice Guidelines; A1, A2, A3, A4, A5 = appraisers 1 to 5 respectively)
11 / 17
0% Well written and comprehensive text with good levels of evidence. Average methods and
stakeholder involvement. Merits in use despite population age. Presence of good
facilitators for applicability.
20% Well writtten and comprehensive text with good levels of evidence. Average methods and
0% Well written and comprehensive text with good levels of evidence. Average guideline's
updating system and stakeholder involvement.
20% Comprehensive text with average levels of evidence. Limited by demography (varying
resources in Italy). Good applicabilty. The text has good informations for prehospital care
but have average informations about the monitoring criteria.
0% Comprehensive text with good levels of evidence and population focus (pediatrics).
Average applicabilty and stakeholder involvement.
0% Well written and comprehensive text with good levels of evidence. Good methods but
average applicability. No conflicts of interest but resource-limited enviroment.
20% Bad levels of evidence with unclear methods. Average applicability and stakeholder
20% Comprehensive text with good levels of evidence and unclear methods. Average
guideline's uptodating system and monitoring criteria. The scope is limited though
acknowledged and stakeholder involvement.
20% Well written and comprehensive text. Unclear levels of evidence. Poor discussion on
implementation and recommendations. Bad methods, applicability and stakeholder
0% Comprehensive text with good levels of evidence but unclear selection criteria for the
evidence. No external review. Good methods and scope.
40% Bad levels of evidence, population remains unclear. Scope is limited by region. Bad
applicabilty and methods.
0% Comprehensive text with good levels of evidence. Presence of good facilitators for
applicability. Good scope and methods.
60% Comprehensive text with average levels of evidence. Bad methods and applicability. Low
0% Well written and comprehensive text with average evidence. Good methods. Scope is
limited the due age of the population.
0% Well written text. Good levels of evidence, scope and methods.
0% Well written and comprehensive text with good levels of evidence. Good applicability.
Average stakeholder involvement and editorial independence. Unclear guideline updating
0% Well written and comprehensive text with good levels of evidence. High lenght.
0% Unclear understanding text with unclear levels of evidence. Average guideline updating
system. Good scope. Bad stakeholder involvement.
0% Well written and comprehensive text with unclear levels of evidence (do not specify the
class of evidence). Average stakeholder involvement.
0% Well written and comprehensive text with good evidence based. Good applicability. No
competing interests by contributors.
0% 100% Bad levels of evidence and is outdated.
20% Well writen and comprehensive text
0% Comprehensive text with good levels of eviedence. The paper needs more informations on
the monitoring criteria. High lengh text.
60% Well written and comprehensive text. Unclear methods. Bad applicability.
47, 48]. This is likely attributed to the scientific rigor of developing a CPG, which typically
involves a highly methodical approach [
]. In general, the guidelines that were more recently
developed or updated, and those that had undergone numerous updates, most consistently
demonstrated the highest quality by AGREE II scores.
Our analysis indicates an overall improvement in the above domains in the most current
CPGs, consistent with other studies [
]. In a 2011 systematic review of CPGs for
managing mild TBI by Tavender et al, the NSW 2006 CPGs fared worse in all domains with the
exception of domain scope and purpose when compared to our values for the 2011 version of
the guidelines . Similarly, another systematic review evaluated an older 2003 version of
NIHCE determining an overall assessment score of 66.9 compared to a score of 100.0 on the
2007 version in our study [
]. It is noteworthy to mention that more recent CPGs have also
the advantage of newer and more rigorous evidence-based medicine in addition to the
availability of designing guidelines around the AGREE or AGREE II format. Nevertheless, a
frequently criticized area in our results, within the rigor of development domain, was the lack of
procedures for updating the guidelines for quality improvement (QI). Given the trend toward
improved CPG quality with newer revisions, development of a systematic QI procedure may
help to ensure quality of future CPGs without the resources required for a full new edition
every few years.
Older reviews have demonstrated limited Stakeholder involvement in CPG development, a
trend that persists in our current review of CPGs [
]. While there has been progressive
improvement in CPG development, the domains of stakeholder involvement, applicability, and
editorial independence remain weak, specifically when it comes to piloting interventions,
addressing potential costs and barriers to implementation, and auditing for quality
improvement. Recent literature suggests that successful implementation of CPGs reduces mortality and
], however applicability of guidelines to a given locale based on factors
such as availability and cost of resources, provider skills, and population needs and values, are
critically important for successful implementation of CPGs in a manner that will improve
patient care. Consideration of stakeholder involvement and applicability are imperative
considering these domains are intrinsically associated with CPG implementation and translation to
other settings such as LMICs.
It has been suggested that adaptation of existing CPGs to local situations may be a more
valid and cost-effective means of achieving high-quality CPGs worldwide [
recent attempts have revealed that this adaptation process remains complex and challenging,
requiring careful planning and implementation to avoid additional costly resource utilization
]. In particular, when the CPG recommendations require resources not present in a given
locale, alternatives to the suggested “optimal” recommendations are required. Given our
12 / 17
findings that most CPGs lacked information on addressing costs and barriers to
implementation, such as in resource-poor settings, this demonstrates an important void in existing CPGs.
Current attempts at adapting existing CPGs may require local practitioners and CPG
developers to return to the primary literature to develop location-specific alternatives that are more
practicable in their region or medical center. In addition, further research may be required to
develop these potential cost-reducing alternatives and demonstrate efficacy.
CPGs and relevance to LMICs
As identified in this systematic review most CPGs have been developed in HICs, which makes
them of questionable relevance to LMICs, especially for those populations with different
costbenefit parameters for medical care. Many hospitals in LMIC, particularly in more rural areas,
lack basic intensive or critical care capabilities, specialized staff, or even necessary diagnostic
imaging, factors upon which the majority of the highest scoring guidelines rely upon for
LMICs have potentially greater challenges and barriers to implementation than HICs based
on these factors, which need to be addressed to enable a CPG to be useful and beneficial. The
few CPGs with high applicability scores were developed by and tailored for HICs [
which makes them unlikely to be applicable for use in limited resourced settings and LMIC
countries in their current forms.
Individual CPGs quality assessment and previous research
Comparing our results with other systematic reviews found many similarities. The EAST
Evaluation and Management of the Mild Traumatic Brain Injury CPG (EAST, 2012) had similar
evaluation of the best and worst domains. Our assessment of the Guidelines for the Acute
Medical Management of Severe Traumatic Brain Injury in Infants, Children, and Adolescents
(BTF, 2012) was similar to that of Grimmer et al., only differing in the applicability domain,
which they assigned a fairly low score of 26.4, compared to our value of 66.7. However, the
overall assessment was similar [
]. We found similar evaluations for domains, overall
assessment and recommendations for the CPG from the National Institute for Health and Clinical
Excellence (NIHCE, 2007) [
14, 17, 47, 48, 58
] and the National Clinical Guideline (SIGN,
] in other studies.
We did find some differences in the domains of applicability and editorial independence in
the systematic review mild TBI CPGs by Tavender et al. For the ACEP Clinical Policy, we
assessed scores of 70 and 91.7 in the fields of applicability and editorial independence,
respectively, compared to their findings of 17 and 50 [
]. The same study, in addition to a 2011
evaluation by Berrigan et al. found respective scores of 36 and 50 in the applicability domain for
the New Zealand Guidelines Group, whereas our score was much higher at 82.5 [
the limited number of systematic reviews on this topic, we were not able to compare all CPGs
included in this study.
Limitations. The main limitations of our study are the subjectiveness of the AGREE II
tool and the potential bias of the reviewers performing the assessment. The AGREE II tool is a
23 question tool established to evaluate CPG quality. While it is a subjective tool, it is the
current gold standard; the AGREE II guidelines suggest using at least two and preferably four
appraisers with content specific knowledge [
]. We utilized five appraisers who all had
content specific knowledge; four were emergency medicine physicians with experience in research
and evidence-based methods and one was a masters level student with extensive research and
content topic experience. Given that the AGREE II assessment requires evaluation of the CPGs
based on the descriptions available in the published manuscripts, there is a small chance that
13 / 17
inaccurate assessments would be due to poor descriptions in the manuscript. However,
comparison of our AGREE II ratings to those of other researchers found similar scores [
Expanding the reviewers to include other relevant specialties such as neurosurgery, critical
care, or neurology would have provided additional input on specialty-specific
recommendations. Additionally, due to language limitations, we were only able to review CPGs in English.
The language limitation did not provide a significant barrier to most CPGs, given that many of
the articles written in different languages were also available in English, or failed to meet
inclusion criteria. However, given that many LMICs are non-English speaking, they may have
developed CPGs in other languages that we were unable to evaluate and subsequently missed.
Finally, we did not search some clinical databases, like the TRIP and GIN repository. However,
due to the several diverse repositories included through the Duke library search, it is unlikely
that we missed published CPGs.
Conclusions. Our review identifies two specific areas for improvement in clinical practice
guidelines addressing the acute management of TBI: (a) the domains of stakeholder
involvement, applicability, and editorial independence remain weak and insufficiently described
specifically when it comes to piloting interventions, addressing potential costs and
implementation barriers, and auditing for quality improvement; (b) CPGs created specifically for use in
low income settings are non-existent. Most of these CPGs were developed by high-income
countries with only one CPG from an upper middle income country, which was found to have
a poorer quality across all domains. This will limit the applicability and implementation
capacity of CPGs for limited resourced settings.
S1 Appendix. Search strategy for PubMed.
S2 Appendix. PRISMA 2009 Checklist.
Conceived and designed the experiments: JRNV SE CJG CAS.
Performed the experiments: AP MMCV JA NR TT KT.
Analyzed the data: AP MMCV JA NR TT KT JRNV SE CAS.
Contributed reagents/materials/analysis tools: MMCV JRNV.
Wrote the paper: AP MMCV JRNV SE CJG CAS.
14 / 17
15 / 17
16 / 17
1. Hyder A , Wunderlich C , Puvanachandra P , Gururaj G , Kobusingye O. The impact of traumatic brain injuries: a global perspective . NeuroRehabilitation . 2007 ; 22 ( 5 ): 341 - 353 . PMID: 18162698
2. Bruns J , Hauser W. The Epidemiology of Traumatic Brain Injury: A Review . Epilepsia . 2003 ; 44 : 2 - 10 .
3. Corrigan J , Selassie A , Orman J . The Epidemiology of Traumatic Brain Injury . Journal of Head Trauma Rehabilitation . 2010 ; 25 ( 2 ): 72 - 80 . doi: 10 .1097/HTR.0b013e3181ccc8b4 PMID: 20234226
4. Global status report on road safety 2013 Supporting a decade of action [Internet] . 1st ed. Geneva: Margaret Chan; 2013 [cited 18 April 2016 ]. Available: http://www.who.int/violence_injury_prevention/road_ safety_status/2013/en/.
5. Bulger E , Nathens A , Rivara F , Moore M , MacKenzie E , Jurkovich G . Management of severe head injury: Institutional variations in care and effect on outcome* . Critical Care Medicine . 2002 ; 30 ( 8 ): 1870 - 1876 . PMID: 12163808
6. Arabi Y , Haddad S , Tamim H , Al-Dawood A , Al-Qahtani S , Ferayan A , et al. Mortality reduction after implementing a clinical practice guidelines-based management protocol for severe traumatic brain injury . Journal of Critical Care . 2010 ; 25 ( 2 ): 190 - 195 . doi: 10 .1016/j.jcrc. 2009 . 05 .004 PMID: 19592201
7. Grol R , Grimshaw J . From best evidence to best practice: effective implementation of change in patients' care . The Lancet . 2003 ; 362 ( 9391 ): 1225 - 1230 .
8. Gupta D , Sharma D , Kannan N , Prapruettham S , Mock C , Wang J , et al. Guideline Adherence and Outcomes in Severe Adult Traumatic Brain Injury for the CHIRAG (Collaborative Head Injury and Guidelines) Study . World Neurosurgery. 2016 ; 89 : 169 - 179 . doi: 10 .1016/j.wneu. 2015 . 12 .097 PMID: 26806065
9. O 'Lynnger T , Shannon C , Le T , Greeno A , Chung D , Lamb F , et al. Standardizing ICU management of pediatric traumatic brain injury is associated with improved outcomes at discharge . Journal of Neurosurgery: Pediatrics . 2016 ; 17 ( 1 ): 19 - 26 . doi: 10 .3171/ 2015 .5. PEDS1544 PMID : 26451717
10. Alexander T , Fuller G , HArgovan P , Clarke D , Muckart D , Thomson S. An audit of the quality of care of traumatic brain injury at a busy regional hospital in South Africa . S Afr J Surg . 2009 ; 47 ( 4 ): 1202 , 124 - 6 .
11. Bullock R , Chesnut R , Clifton G , Marion D , Narayan R , Newell D , et al. Guidelines for the management of severe head injury . Eur J Emerg Med . 1996 ; 3 ( 2 ): 109 - 27 . PMID: 9028756
12. The Brain Trauma Foundation . The American Association of Neurological Surgeons. The Joint Section on Neurotrauma and Critical Care. Initial management . J Neurotrauma . 2000 ; 17 ( 6-7 ): 463 - 469 . PMID: 10937888
13. Carson S , Davis-O'Reilly C , Drexel P , Fu R , Norris S , Pappas M , et al. Guidelines for the Management of Severe Traumatic Brain Injury: Introduction . Journal of Neurotrauma , 2007 . 24 : p. S1 - S106 . J Neurotraum. 2016 ; 24 ( sup1 ): 1 - 116 .
14. Tavender E , Bosch M , Green S , O'Connor D , Pitt V , Phillips K , et al. Quality and Consistency of Guidelines for the Management of Mild Traumatic Brain Injury in the Emergency Department . Academic Emergency Medicine. 2011 ; 18 ( 8 ): 880 - 889 . doi: 10 .1111/j.1553- 2712 . 2011 . 01134 . x PMID : 21843224
15. Alarcon J , Rubiano A , Chirinos M , Valderrama A , Gich I , Bonfill X et al. Clinical practice guidelines for the care of patients with severe traumatic brain injury . Journal of Trauma and Acute Care Surgery . 2013 ; 75 ( 2 ): 311 - 319 . doi: 10 .1097/TA.0b013e3182924bf8 PMID: 23887565
16. Rusnak M , Mauritz W , Lecky F , Kaniasnky M , Brazinova A . Evaluation of traumatic brain injury guidelines using AGREE instrument . Bratisl Lek Listy . 2008 ; 109 ( 8 ): 374 - 380 . PMID: 18837249
17. Brouwers M , Kho M , Browman G , Burgers J , Cluzeau F , Feder G , et al. AGREE II: advancing guideline development, reporting and evaluation in health care . Canadian Medical Association Journal . 2010 ; 182 ( 18 ): E839 - E842 . doi: 10 .1503/cmaj.090449 PMID: 20603348
18. Moher D , Liberati A , Tetzlaff J , Altman D . Preferred Reporting Items for Systematic Reviews and MetaAnalyses: The PRISMA Statement . PLoS Med . 2009 ; 6 ( 7 ):e1000097. doi: 10.1371/journal.pmed. 1000097 PMID: 19621072
19. Search-Clinical Tools-LibGuides at Duke University Medical Center [Internet]. Guides.mclibrary. duke.edu . 2016 [cited 18 April 2016 ]. Available: http://guides.mclibrary.duke.edu/clinicaltools.
20. Vlayen J , Aertgeerts B , Hannes K , Sermeus W , Ramaekers D. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit . International Journal for Quality in Health Care . 2005 ; 17 ( 3 ): 235 - 242 . PMID: 15743883
21. World Bank Country and Lending Groups-World Bank Data Help Desk [Internet]. Datahelpdesk.worldbank.org . 2016 [cited 1 July 2016 ]. Available: https://datahelpdesk.worldbank.org/knowledgebase/ articles/906519.
22. The R Project for Statistical Computing , Version 3.2 .1. Viena , Austria: R Foundation for Statistical Computing; 2015 .
23. Schutzman S , Barnes P , Duhaime A , Greenes D , Homer C , Jaffe D , et al. Evaluation and Management of Children Younger Than Two Years Old With Apparently Minor Head Trauma: Proposed Guidelines . PEDIATRICS . 2001 ; 107 ( 5 ): 983 - 993 . PMID: 11331675
24. The management of minor closed head injury in children . Committee on Quality Improvement , American Academy of Pediatrics. Commission on Clinical Policies and Research , American Academy of Family Physicians. Pediatrics . 1999 ; 104 ( 6 ): 1407 - 1415 . PMID: 10585999
25. Jagoda A , Bazarian J , Bruns J , Cantrill S , Gean A , Howard P , et al. Clinical Policy: Neuroimaging and Decisionmaking in Adult Mild Traumatic Brain Injury in the Acute Setting . Journal of Emergency Nursing . 2009 ; 35 ( 2 ): e5 - e40 . doi: 10 .1016/j.jen. 2008 . 12 .010 PMID: 19285163
26. Badjatia N , Carney N , Crocco T , Fallat M , Hennes H , Jagoda A , et al. Guidelines for Prehospital Management of Traumatic Brain Injury 2nd Edition . Prehospital Emergency Care. 2008 ; 12 ( sup1 ): S1 - S52 .
27. Kochanek P , Carney N , Adelson P , Ashwal S , Bell M , Bratton S , et al. Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and Adolescents-Second Edition . Pediatric Critical Care Medicine . 2012 ; 13 ( Suppl1 ): S1 - S2 .
28. Guidelines for the management of severe traumatic brain injury . J Neurotrauma . 2007 ; 24 ( Suppl1 ): s1 - 106 .
29. Kamerling S , Lutz N , Posner J , Vanore M. Mild Traumatic Brain Injury In Children: Practice Guidelines For Emergency Department and Hospitalized Patients . Pediatric Emergency Care . 2003 ; 19 ( 6 ): 431 - 440 . PMID: 14676497
30. Hebb M , Clarke D , Tallon J . Development of a provincial guideline for the acute assessment and management of adult and pediatric patients with head injuries . Can J Surg . 2007 ; 50 ( 3 ): 187 - 194 . PMID: 17568490
31. Cushman J , Agarwal N , Fabian T , Garcia V , Nagy K , Pasquale M , et al. Practice Management Guidelines for the Management of Mild Traumatic Brain Injury: The EAST Practice Management Guidelines Work Group . The Journal of Trauma: Injury , Infection, and Critical Care . 2001 ; 51 ( 5 ): 1016 - 1026 .
32. Barbosa R , Jawa R , Watters J , Knight J , Kerwin A , Winston E , et al. Evaluation and management of mild traumatic brain injury . Journal of Trauma and Acute Care Surgery . 2012 ; 73 : S307 - S314 . doi: 10 . 1097/TA.0b013e3182701885 PMID: 23114486
33. Maas A , Dearden M , Teasdale G , Braakman R , Cohadon F , Iannotti F , et al. EBIC-Guidelines for management of severe head injury in adults . Acta neurochir . 1997 ; 139 ( 4 ): 286 - 294 . PMID: 9202767
34. Vos P , Battistin L , Birbamer G , Gerstenbrand F , Potapov A , Prevec T et al. EFNS guideline on mild traumatic brain injury: report of an EFNS task force . Eur J Neurol . 2002 ; 9 ( 3 ): 207 - 219 . PMID: 11985628
35. Piek J . Guidelines for the pre-hospital care of patients with severe head injuries . Intensive Care Medicine . 1998 ; 24 ( 11 ): 1221 - 1225 . PMID: 9876987
36. Kochanek PM , Carney N , David Adelson P , Ashwal S , Bell MJ , Bratton S , et al. Guidelines for the Acute Medical Management of Severe Traumatic Brain Injury in Infants, Children, and AdolescentsSecond Edition . Pediatric Critical Care Medicine . 2012 ; 13 : S1 - S2 . doi: 10 .1097/PCC. 0b013e31823f435c PMID: 22217782
37. Head Injury Triage, assessment, investigation and early management of head injury in inants, children and adults [Internet] . 1st ed. London: National Institute for Health and Clinical Excellence: Guidance.; 2016 [cited 18 April 2016 ]. Available: http://www.ncbi.nlm.nih.gov/books/NBK53036/pdf/Bookshelf_ NBK53036.pdf.
38. NSW Ministry of Health. Adult Trauma Clinical Practice Guidelines Initial Management of Closed Head Injury in Adults 2nd Edition . Sydney: NSW Institute of Trauma and Injury Management; 2007 p. 1 - 129 .
39. Traumatic Brain Injury: Diagnosis, Acute Management and Rehabilitation [Internet]. 1st ed. Wellington: New Zealand Guidelines Group; 2006 [cited 18 April 2016 ]. Available: http://www.acc.co.nz/PRD_ EXT_CSMP/groups/external_communications/documents/guide/wim2_ 059414 .pdf.
40. Rusticali B , Villani R . Treatment of minor and severe traumatic brain injury. National reference guidelines . Minerva Anestesiol . 2008 ; 74 ( 10 ): 583 - 616 . PMID: 18854800
41. Undén J , Ingebrigtsen T , Romner B . Scandinavian guidelines for initial management of minimal, mild and moderate head injuries in adults: an evidence and consensus-based update . BMC Medicine . 2013 ; 11 ( 1 ): 50 .
42. Early management of patients with a head injury A national clinical guideline [Internet] . 1st ed. Edinburgh: Scottish Intercollegiate Guidelines Network; 2009 [cited 18 April 2016 ]. Available: http://www. sign.ac.uk/pdf/sign110.pdf.
43. Guidelines for minor head injured patients' management in adult age . The Study Group on Head Injury of the Italian Society for Neurosurgery. J Neutosurg Sci . 1996 ; 40 ( 1 ): 11 - 15 .
44. Procaccio F , Stocchetti N , Citerio G , Bernadino M , Beretta L , Della Corte F , et al. Guidelines for the treatment of adults with severe head trauma (part I) . Initial assessment; evaluation and pre-hospital treatment; current criteria for hospital admission; systemic and cerebral monitoring . J Neurosurg Sci . 2000 ; 44 ( 1 ): 1 - 10 . PMID: 10961490
45. Liao K , Chang C , Chang H , Chang K , Chen C , Chen T , et al. Clinical practice guidelines in severe traumatic brain injury in Taiwan . Surgical Neurology . 2009 ; 72 : S66 - S73 . doi: 10 .1016/j.surneu. 2009 . 07 . 004 PMID: 19818476
46. de Andrade AF , Marino R , Ciquini O , Gadelha Figueiredo EG , Machado AG . Guidelines for Neurosurgical Trauma in Brazil . World Journal of Surgery . 2001 ; 25 ( 9 ): 1186 - 1201 . PMID: 11571957
47. Berrigan L , Marshall S , McCullagh S , Velikonja D , Bayley M. Quality of clinical practice guidelines for persons who have sustained mild traumatic brain injury . Brain Injury . 2011 ; 25 ( 7-8 ): 742 - 751 . doi: 10 . 3109/02699052. 2011 .580317 PMID: 21604931
48. Grimmer K , Dizon J , Milanese S , King E , Beaton K , Thorpe O , et al. Efficient clinical evaluation of guideline quality: development and testing of a new tool . BMC Med Res Methodol . 2014 ; 14 ( 1 ): 63 .
49. Qaseem A. The Development of Clinical Practice Guidelines and Guidance Statements of the American College of Physicians: Summary of Methods . Annals of Internal Medicine . 2010 ; 153 ( 3 ): 194 . doi: 10 .7326/ 0003 -4819-153-3- 201008030 -00010 PMID: 20679562
50. Peloso P , Carroll L , Cassidy J , Borg J , von Holst H , Holm L , et al. Critical evaluation of the existing guidelines on mild traumatic brain injury . Journal of Rehabilitation Medicine . 2004 ; 36 ( 0 ): 106 - 112 .
51. Lee J , Rittenhouse K , Bupp K , Gross B , Rogers A , Rogers F , et al. An analysis of Brain Trauma Foundation traumatic brain injury guideline compliance and patient outcome . Injury . 2015 ; 46 ( 5 ): 854 - 858 . doi: 10 .1016/j.injury. 2014 . 12 .023 PMID: 25661105
52. Fakhry S , Trask A , Waller M , Watts D. Management of Brain-Injured Patients by an Evidence-Based Medicine Protocol Improves Outcomes and Decreases Hospital Charges . The Journal of Trauma: Injury , Infection, and Critical Care . 2004 ; 56 ( 3 ): 492 - 500 .
53. Griesdale D , Örtenwall V , Norena M , Wong H , Sekhon M , Kolmodin L , et al. Adherence to guidelines for management of cerebral perfusion pressure and outcome in patients who have severe traumatic brain injury . Journal of Critical Care . 2015 ; 30 ( 1 ): 111 - 115 . doi: 10 .1016/j.jcrc. 2014 . 07 .026 PMID: 25179411
54. Vukić M , Negovetić L , Kovač D , Ghajar J , Glavić Z , Gopčević A . The Effect of Implementation of Guidelines for the Management of Severe Head Injury on Patient Treatment and Outcome . Acta Neurochirurgica . 1999 ; 141 ( 11 ): 1203 - 1208 . PMID: 10592121
55. Fervers B , Burgers J , Haugh M , Latreille J , Mika-Cabanne N , Paquete L , et al. Adaptation of clinical guidelines: literature review and proposition for a framework and procedure . International Journal for Quality in Health Care . 2006 ; 18 ( 3 ): 167 - 176 . PMID: 16766601
56. Harrison M , Graham I , van den Hoek J , Dogherty E , Carley M , Angus V . Guideline adaptation and implementation planning: a prospective observational study . Implementation Science . 2013 ; 8 ( 1 ): 49 .
57. Dries D. Defining the Need for Critical Care in Underserved Areas [Internet] . Society of Critical Care Medicine . 2016 [cited 18 April 2016 ]. Available: http://www.sccm.org/Communications/CriticalConnections/Archives/Pages/ Defining-the-Need-for-Critical-Care-in-Underserved-Areas.aspx .
58. Wong J , Côté P , Shearer H , Carroll L , Yu H , Varatharajan S , et al. Clinical practice guidelines for the management of conditions related to traffic collisions: a systematic review by the OPTIMa Collaboration . Disability and Rehabilitation . 2014 ; 37 ( 6 ): 471 - 489 . doi: 10 .3109/09638288. 2014 .932448 PMID: 24963833