Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit

Boutron et al. BMC Medicine (2019) 17:105 https://doi.org/10.1186/s12916-019-1330-9 RESEARCH ARTICLE Open Access Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/ caregivers’ interpretation of treatment benefit Isabelle Boutron1,2,3*, Romana Haneef1,2,3, Amélie Yavchitz1,3, Gabriel Baron3, John Novack4, Ivan Oransky5, Gary Schwitzer6 and Philippe Ravaud1,2,3,7 Abstract Background: News stories represent an important source of information. We aimed to evaluate the impact of “spin” (i.e., misrepresentation of study results) in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit. Methods: We conducted three two-arm, parallel-group, Internet-based randomized trials (RCTs) comparing the interpretation of news stories reported with or without spin. Each RCT considered news stories reporting a different type of study: (1) pre-clinical study, (2) phase I/II non-RCT, and (3) phase III/IV RCT. For each type of study, we identified news stories reported with spin that had earned mention in the press. Two versions of the news stories were used: the version with spin and a version rewritten without spin. Participants were patients/caregivers involved in Inspire, a large online community of more than one million patients/caregivers. The primary outcome was participants’ interpretation assessed by one specific question “What do you think is the probability that ‘treatment X’ would be beneficial to patients?” (scale, 0 [very unlikely] to 10 [very likely]). Results: For each RCT, 300 participants were randomly assigned to assess a news story with spin (n = 150) or without spin (n = 150), and 900 participants assessed a news story. Participants were more likely to consider that the treatment would be beneficial to patients when the news story was reported with spin. The mean (SD) score for the primary outcome for abstracts reported with and without spin for pre-clinical studies was 7.5 (2.2) versus 5.8 (2.8) (mean difference [95% CI] 1.7 [1.0–2.3], p < 0.001); for phase I/II non-randomized trials, 7.6 (2.2) versus 5.8 (2.7) (mean difference 1.8 [1.0–2.5], p < 0.001); and for phase III/IV RCTs, 7.2 (2.3) versus 4.9 (2.8) (mean difference 2.3 [1.4–3.2], p < 0.001). Conclusions: Spin in health news stories reporting studies of pharmacologic treatments affects patients’/ caregivers’ interpretation. (Continued on next page) * Correspondence: 1 INSERM, UMR 1153, Epidemiology and Biostatistics Research Center (CRESS), Methods Team, Paris, France 2 Faculté de Médecine, Paris Descartes University, Paris, France Full list of author information is available at the end of the article © The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Boutron et al. BMC Medicine (2019) 17:105 Page 2 of 10 (Continued from previous page) Trial registration: ClinicalTrials.gov, NCT03094078, NCT03094104, NCT03095586 Keywords: Randomized trial, Spin, Distorted interpretation, Detrimental research practices Background News stories represent an important source of information for patients [1–3]. In the USA, according to surveys of the Pew Research Center, 93% of the population reads at least some news online [4] and 67% declares following health news somewhat or very closely [5]. Furthermore, about two thirds of the time, reading health news prompts follow-up actions such as searching for more information or talking about it with others [6]. However, some evidence suggests that many news stories do not accurately represent research results and could mislead readers with “spin,” defined as “the presentation of information in a particular way, a slant, especially a favorable one” [7–12]. A systematic assessment of news stories highlighted in Google News showed that 88% of news stories were distorted (i.e., reported with spin) [13]. Several different types of spin could be used to distort the study results. The most frequent are misleading reporting such as not reporting adverse events, misleading interpretation such as claiming a causal effect despite the non-randomized study design, overgeneralization of the results such as extrapolating a beneficial effect from an animal study to humans, and highlighting a single-patient experience for the success of a new treatment instead of focusing on the group results [13]. Spin in news stories is often related to the presence of spin in the published article and its press release [9, 14]. A quantitative content analysis of 534 press releases and related research articles and news stories showed that the main source of spin was the press release [15]. A comparison of newspaper stories and peer-reviewed research papers in genetics showed that newspaper articles accurately conveyed the results, but there was an overemphasis on benefit and an underemphasis on risk in both the scientific article and related news story [16, 17]. The impact of spin has been rarely explored. A previous study showed that spin reported in abstracts of randomized controlled trials (RCTs) with a statistically non-significant primary outcome can affect trialists’ interpretations; the experimental treatments were rated more beneficial when the abstracts were reported with spin versus no spin [18]. To our knowledge, the impact of spin in health news stories on patients’/caregivers’ interpretation of the study results has not been evaluated in an experimental study. The aim of this study was to evaluate the impact of spin in health news stories reporting various types of studies evaluating pharmacologic treatments on patients’/caregivers’ interpretation of the benefits of treatment. Methods Study design We planned three Internet-based RCTs (ib-RCTs) comparing the interpretation of news stories reported with or without spin. We defined “spin” as a misrepresentation of study results, regardless of motive (intentionally or unintentionally), that overemphasizes the efficacy or overstates safety of the treatment as compared with that shown by the results [19]. Each RCT considered news stories reporting a different type of study evaluating pharmacologic treatments: (1) pre-clinical study, (2) phase I/II non-randomized trial, and (3) phase III/IV RCT. The protocol used for each RCT is detailed elsewhere [20]. Participants Participants were patients or caregivers involved in Inspire, a l (...truncated)