Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention

Lazarus et al. BMC Medical Research Methodology (2015) 15:85 DOI 10.1186/s12874-015-0079-x RESEARCH ARTICLE Open Access Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention Clément Lazarus1,2,3, Romana Haneef1,2, Philippe Ravaud1,2,3,4,5 and Isabelle Boutron1,2,3,4* Abstract Background: Spin represents specific reporting strategies, either intentional or unintentional, to convince the reader that the beneficial effect of the experimental intervention in terms of efficacy and safety is greater than that shown by the results. The objectives of this study were to 1) develop a classification of spin specific to non-randomized studies assessing an intervention and 2) estimate the prevalence of spin in abstracts of reports of such studies. Methods: In a first step, we developed a specific classification of spin for non-randomized studies by a literature review and pilot study. In a second step, 2 researchers trained in the field of methodology evaluated the prevalence of spin in the abstract of all non-randomized studies assessing an intervention published in the BioMed Central Medical Series journals between January 1, 2011 and December 31, 2013. All disagreements were resolved by consensus. We also determined whether the level of spin in abstract conclusions was high (spin reported without uncertainty or recommendations for further trials), moderate (spin reported with some uncertainty or recommendations for further trials) or low (spin reported with uncertainty and recommendations for further trials). Results: Among the 128 assessed articles assessed, 107 (84 %) had at least one example of spin in their abstract. The most prevalent strategy of spin was the use of causal language, identified in 68 (53 %) abstracts. Other frequent strategies were linguistic spin, inadequate implications for clinical practice, and lack of focus on harm, identified in 33 (26 %), 25 (20 %), and 34 (27 %) abstracts respectively. Abstract conclusions of 61 (48 %) articles featured a high level of spin. Conclusion: Abstract of reports of non-randomized studies assessing an intervention frequently includes spin. Efforts to reduce the prevalence of spin in abstract for such studies are needed. Introduction Spin, or distortion of study findings, can be used by authors to enhance their study findings more than the results justify [1, 2]. Recent studies demonstrated a high prevalence of spin in study reports. Spin was found in more than half of the abstract conclusions of randomized controlled trials with statistically non-significant results for the primary outcome [3]; moreover, one third of reports of diagnostic accuracy studies contained a form of over-interpretation [4]. The spin used consisted * Correspondence: 1 METHODS Team, Centre of Research in Epidemiology and Statistics Sorbonne Paris Cité (CRESS), UMR 1153, INSERM, Paris, France 2 Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine de Paris, Paris, France Full list of author information is available at the end of the article mainly of a focus on statistically significant results (within-group comparison, secondary outcomes, subgroup analyses, modified population of analyses); or interpreting statistically non significant results for the primary outcomes as showing treatment equivalence or comparable effectiveness. A recent study in the field of cancer found that the prevalence of spin has increased over time [5]. In the same field, a randomized controlled trial demonstrated that spin in abstracts could modify a reader’s interpretation of study results [6]. Non-randomized studies are commonly used in medical research to evaluate interventions. They are particularly useful to draw conclusions about the safety or efficacy of interventions in real-world settings, to assess rare or long-term adverse events or when randomization is not possible (e.g., surgical procedures). However, these © 2015 Lazarus et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Lazarus et al. BMC Medical Research Methodology (2015) 15:85 designs have important limitations. Non randomized designs are also susceptible to many type of spin which could be same or different from those previously described [7]. Particularly, contrary to randomized clinical trials, they may not allow for establishing causal inferences but rather, only an association [8–10]. Our study aimed to 1) develop a classification of spin for non-randomized studies assessing therapeutic interventions and 2) estimate the prevalence of spin in the abstracts of non-randomized studies evaluating a therapeutic intervention published in BioMed Central Medical Series journals. Methods Development of a classification of spin Spin was defined as the use of specific reporting strategies, either intentional or unintentional, to convince the reader that the beneficial effect of the experimental treatment in terms of efficacy and safety is higher than is actually shown by the data. To develop the classification of spin, we performed a literature review of studies of spin for other study designs [3–5, 9, 11] as well as studies of distorted presentation and interpretation of findings from non-randomized studies [12–19]. From these data, we developed a preliminary classification, discussed among the authors and tested by two 2 researchers (CL and RH) with a sample of 15 articles. The classification was discussed until consensus was achieved among the authors. Prevalence of spin in abstracts We selected a sample of reports of non-randomized studies evaluating an intervention published in 25 journals of the BioMed Central Medical Series that regularly publish clinical studies. We selected these journals because they are open-access and publish reports of non-randomized studies evaluating therapeutic interventions from a large range of medical specialties. Search strategy We searched MEDLINE via PubMed (search date January 21, 2014) for all articles published in the 25 BioMed Central Medical Series journals between January 1, 2011 and December 31, 2013. The list of selected journals and complete search strategy are respectively in Additional files 1 and 2. Study identification One researcher (CL) screened all titles, abstracts and, if necessary, the full-text articles of the citations retrieved and selected all reports of non-randomized studies assessing a therapeutic intervention defined as a pharmacologic (...truncated)