Effect of Risk of Bias on the Effect Size of Meta-Analytic Estimates in Randomized Controlled Trials in Periodontology and Implant Dentistry

RESEARCH ARTICLE Effect of Risk of Bias on the Effect Size of Meta-Analytic Estimates in Randomized Controlled Trials in Periodontology and Implant Dentistry Clovis Mariano Faggion, Jr1*, Yun-Chun Wu2, Moritz Scheidgen1, Yu-Kang Tu2 1 Department of Periodontology, Faculty of Dentistry, University of Münster, Münster, Germany, 2 Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan * Abstract Background OPEN ACCESS Citation: Faggion CM, Jr, Wu Y-C, Scheidgen M, Tu Y-K (2015) Effect of Risk of Bias on the Effect Size of Meta-Analytic Estimates in Randomized Controlled Trials in Periodontology and Implant Dentistry. PLoS ONE 10(9): e0139030. doi:10.1371/journal. pone.0139030 Editor: Sam Eldabe, The James Cook University Hospital, UNITED KINGDOM Risk of bias (ROB) may threaten the internal validity of a clinical trial by distorting the magnitude of treatment effect estimates, although some conflicting information on this assumption exists. Objective The objective of this study was evaluate the effect of ROB on the magnitude of treatment effect estimates in randomized controlled trials (RCTs) in periodontology and implant dentistry. Received: January 28, 2015 Accepted: July 27, 2015 Methods Published: September 30, 2015 A search for Cochrane systematic reviews (SRs), including meta-analyses of RCTs published in periodontology and implant dentistry fields, was performed in the Cochrane Library in September 2014. Random-effect meta-analyses were performed by grouping RCTs with different levels of ROBs in three domains (sequence generation, allocation concealment, and blinding of outcome assessment). To increase power and precision, only SRs with meta-analyses including at least 10 RCTs were included. Meta-regression was performed to investigate the association between ROB characteristics and the magnitudes of intervention effects in the meta-analyses. Copyright: © 2015 Faggion et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All relevant data are within the paper and its Supporting Information files. Funding: Y. C. Wu and Y. K. Tu were partly supported by a grant from the Ministry of Science and Technology in Taiwan (grant number: MOST 103 2314 - B - 002 - 032 - MY3). Competing Interests: The authors have declared that no competing interests exist. Results Of the 24 initially screened SRs, 21 SRs were excluded because they did not include at least 10 RCTs in the meta-analyses. Three SRs (two from periodontology field) generated information for conducting 27 meta-analyses. Meta-regression did not reveal significant PLOS ONE | DOI:10.1371/journal.pone.0139030 September 30, 2015 1/9 Bias and Treatment Estimates differences in the relationship of the ROB level with the size of treatment effect estimates, although a trend for inflated estimates was observed in domains with unclear ROBs. Conclusion In this sample of RCTs, high and (mainly) unclear risks of selection and detection biases did not seem to influence the size of treatment effect estimates, although several confounders might have influenced the strength of the association. Introduction Risk of bias (ROB) is an important issue to consider when appraising studies. Generally, the higher the ROB of a study, the less confidence there will be that the results are valid. For example, a meta-analysis formed by randomized clinical trials (RCTs) with low ROB will probably generate stronger evidence to support clinical decision-making than a meta-analysis formed by RCTs with high or unclear ROB. Levels of ROB may interfere with the treatment effect estimates by inflating or reducing the real values. For example, studies with high ROB have been found to generate exaggerated treatment effect estimates [1]. Thus, there is a general consensus that authors of all studies should report all of the measures necessary to produce a study with a low ROB [1, 2]. The Cochrane Collaboration has developed a methodology to evaluate the ROBs of RCTs in different domains [3]. Among other domains, using an adequate randomization process and masking the people involved in the study are important steps to minimize selection and performance/detection biases, respectively. A lack of allocation concealment, an important component of the randomization process, has been demonstrated to inflate treatment estimates in some medical fields [4]. However, attempts to evaluate the effects of different levels of bias on the magnitude of treatment effect estimates have not been performed in periodontology and implant dentistry. Therefore, the objective of this study was to evaluate, in three domains, the influence of ROB on the effect size of meta-analytic estimates in systematic reviews (SRs) of RCTs published in periodontology and implant dentistry. Materials and Methods Eligibility criteria To be included in this study, an article should be a SR of RCTs in the fields of periodontology and implant dentistry and published in the Cochrane Database of Systematic Reviews. Excluded from the analysis were SRs published in paper-based journals, SRs including RCTs with noninterventional purposes (e.g., no therapy-related outcome), and SRs including studies with non-RCT designs. Homogeneity of comparisons should allow a meta-analysis to be conducted of the RCTs included in the SR. Therefore, SRs without a meta-analysis were excluded. Literature search From 15 to 18 September 2014, a literature search was performed in the Cochrane Database of Systematic Reviews for Cochrane SRs on interventions in the fields of periodontology and implant dentistry. Searches were performed directly in the Cochrane Library homepage, by using the “browse by topics” option (Fig 1). Searches were performed independently and in PLOS ONE | DOI:10.1371/journal.pone.0139030 September 30, 2015 2/9 Bias and Treatment Estimates Fig 1. Search for systematic reviews in the Cochrane Library of systematic reviews. doi:10.1371/journal.pone.0139030.g001 duplicate by two authors (CMF and MS). Any disagreements in SR selection were resolved by discussion until consensus was achieved. PLOS ONE | DOI:10.1371/journal.pone.0139030 September 30, 2015 3/9 Bias and Treatment Estimates Rationale for assessment and data extraction RCTs published in the included SRs were further selected for assessment. SRs from the Cochrane Collaboration were chosen because they use similar approaches for assessing ROB in the primary studies. Furthermore, Cochrane SRs are considered to be of high methodological quality [5, 6], and they may provide the best available evidence possible. Information about ROB in three domains—namely, sequence generation, allocation concealment, and blinding of outcome assessment—was retriev (...truncated)