A descriptive analysis of non-Cochrane child-relevant systematic reviews published in 2014

Gates et al. BMC Medical Research Methodology (2018) 18:99 https://doi.org/10.1186/s12874-018-0562-2 RESEARCH ARTICLE Open Access A descriptive analysis of non-Cochrane child-relevant systematic reviews published in 2014 Michelle Gates1, Sarah A Elliott1,2, Cydney Johnson1, Denise Thomson1,2, Katrina Williams2,3,6, Ricardo M Fernandes2,4,5,7 and Lisa Hartling1,2* Abstract Background: Consumers, clinicians, policymakers and researchers require high quality evidence to guide decisionmaking in child health. Though Cochrane systematic reviews (SRs) are a well-established source of evidence, little is known about the characteristics of non-Cochrane child-relevant SRs. To complement published descriptions of Cochrane SRs, we aimed to characterize the epidemiologic, methodological, and reporting qualities of non-Cochrane child-relevant SRs published in 2014. Methods: English-language child-relevant SRs of quantitative primary research published outside the Cochrane Library in 2014 were eligible for this descriptive analysis. A research librarian searched MEDLINE, CINAHL, Web of Science, and PubMed in August 2015. A single reviewer screened articles for inclusion; a second verified the excluded studies. Reviewers extracted: general characteristics of the review; included study characteristics; methodological approaches. We performed univariate analyses and presented the findings narratively. Results: We identified 1598 child-relevant SRs containing a median (IQR) 19 (11, 33) studies. These originated primarily from high-income countries (n = 1247, 78.0%) and spanned 47 of the 53 Cochrane Review Groups. Most synthesized therapeutic (n = 753, 47.1%) or epidemiologic (n = 701, 43.8%) evidence. Though 39.3% (n = 628) of SRs included evidence related to children only, few were published in pediatric-specific journals (n = 283, 17.7%). Reporting quality seemed poor based on the items we assessed; few reviews mentioned an a-priori protocol (n = 246, 15.4%) or registration (n = 111, 6.9%), and only 23.4% (n = 374) specified a primary outcome. Many SRs relied solely on evidence from non-RCTs (n = 796, 49.8%). Less than two-thirds (n = 953, 59.6%) appraised the quality of included studies and assessments of the certainty of the body of evidence were rare (n = 102, 6.4%). Conclusions: Child-relevant Cochrane SRs are a known source of high quality evidence in pediatrics. There exists, however, an abundance of evidence from non-Cochrane SRs that may be complementary. Our findings show that high-quality non-Cochrane SRs may not be practical nor easy for knowledge users to find. Improvements are needed to ensure that evidence syntheses published outside of the Cochrane Library adhere to the high standard of conduct and reporting characteristic of Cochrane SRs. Keywords: Systematic review, Pediatrics, Child, Methods * Correspondence: 1 Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, AB, Canada 2 Cochrane Child Health, Department of Pediatrics, University of Alberta, Edmonton, AB, Canada Full list of author information is available at the end of the article © The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Gates et al. BMC Medical Research Methodology (2018) 18:99 Background Consumers, clinicians, policymakers and researchers require up-to-date evidence to guide decision-making, but keeping up with the rapid accumulation of primary research is a challenge. [1] Systematic reviews (SRs) provide a transparent synthesis of all available evidence on a given research question. [2] They are used to inform healthcare decisions, [3] evidence-based practice guidelines, [4] and can facilitate the identification of knowledge gaps, thereby reducing research waste by guiding where new research is most needed. [5, 6] However, findings of SRs that are outdated, methodologically flawed, or incompletely reported are of limited use. [7] Although clear standards for conduct and reporting are publicly available to authors and widely endorsed, [8] many reviews fall short [9] of best-practice standards. [10, 11] In a comparison of biomedical SRs published in 2004 [12] and 2014, [9] Page et al. (2016) warned that the rise in popularity of evidence syntheses has been accompanied by a proliferation of substandard SRs. [9] Performance bonuses for publication [13] and the rise of predatory publishers [14, 15] are two suggested contributors. These poorly-conducted reviews run the risk of propagating biased, misleading, or harmful conclusions. [16] Periodically characterizing all available SRs can increase awareness of the areas in need of improvement. Given the shortage of child-specific evidence for healthcare decision making, [17–19] the need for high quality evidence syntheses in pediatrics is clear. In 2009 [20] and 2013, [21] authors from our research group undertook descriptive analyses of all child-relevant SRs in the Cochrane Library. Reporting quality of the SRs was highly variable and many were out-of-date. [21] Few focused solely on children or conducted child-specific subgroup analyses. [21] No similar efforts have been made to characterize child-relevant non-Cochrane SRs. Cochrane reviews represent a small minority [9] of the mounting number of available reviews, [16] and are known to differ from non-Cochrane reviews in terms of scope and reporting quality. [9, 12] Thus, the knowledge gap in terms of the quality and quantity of non-Cochrane evidence available in the field of pediatrics is substantial. We aimed to identify all non-Cochrane child-relevant SRs published over 1 year (2014) and describe the characteristics of these SRs, their included studies, and methodological approaches. Methods Design We conducted a cross-sectional descriptive study of all SRs on any child-relevant topic published outside of the Page 2 of 10 Cochrane Library over 1 year. Ethics approval was not required because there were no human participants. Eligibility criteria English language child-relevant SRs of quantitative primary research published outside of the Cochrane Library in 2014 were eligible. We used Cochrane [10] standards to define SRs for inclusion, with some flexibility (e.g., we did not require a quality appraisal) to account for the fact that many SRs published outside of the Cochrane library might not meet their stringent criteria nor adhere to published reporting standards such as the PRISMA state (...truncated)