Presenting comparative study PRO results to clinicians and researchers: beyond the eye of the beholder

Quality of Life Research, Nov 2017

Purpose Patient-reported outcome (PRO) results from clinical trials can inform clinical care, but PRO interpretation is challenging. We evaluated the interpretation accuracy and perceived clarity of various strategies for displaying clinical trial PRO findings. Methods We conducted an e-survey of oncology clinicians and PRO researchers (supplemented by one-on-one clinician interviews) that randomized respondents to view one of the three line-graph formats (average scores over time for two treatments on four domains): (1) higher scores consistently indicating “better” patient status; (2) higher scores indicating “more” of what was being measured (better for function, worse for symptoms); or (3) normed scores. Two formats displayed proportions changed (pie/bar charts). Multivariate modeling was used to analyze interpretation accuracy and clarity ratings. Results Two hundred and thirty-three clinicians and 248 researchers responded; ten clinicians were interviewed. Line graphs with “better” directionality were more likely to be interpreted accurately than “normed” line graphs (OR 1.55; 95% CI 1.01–2.38; p = 0.04). No significant differences were found between “better” and “more” formats. “Better” formatted graphs were also more likely to be rated “very clear” versus “normed” formatted graphs (OR 1.91; 95% CI 1.44–2.54; p < 0.001). For proportions changed, respondents were less likely to make an interpretation error with pie versus bar charts (OR 0.35; 95% CI 0.2–0.6; p < 0.001); clarity ratings did not differ between formats. Qualitative findings informed the interpretation of the survey findings. Conclusions Graphic formats for presenting PRO data differ in how accurately they are interpreted and how clear they are perceived to be. These findings will inform the development of best practices for optimally reporting PRO findings.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://link.springer.com/content/pdf/10.1007%2Fs11136-017-1710-6.pdf

Presenting comparative study PRO results to clinicians and researchers: beyond the eye of the beholder

Presenting comparative study PRO results to clinicians and researchers: beyond the eye of the beholder Michael Brundage 0 1 2 4 5 6 Amanda Blackford 0 1 2 4 5 6 Elliott Tolbert 0 1 2 4 5 6 Katherine Smith 0 1 2 4 5 6 Elissa Bantug 0 1 2 4 5 6 Claire Snyder 0 1 2 4 5 6 locations) 0 1 2 4 5 6 0 Katherine Smith 1 Elliott Tolbert 2 Amanda Blackford 3 Michael Brundage 4 Department of Oncology, Queens Cancer Research Institute, Queen's University Kingston , Kingston, ON , Canada 5 Claire Snyder 6 Elissa Bantug Purpose Patient-reported outcome (PRO) results from clinical trials can inform clinical care, but PRO interpretation is challenging. We evaluated the interpretation accuracy and perceived clarity of various strategies for displaying clinical trial PRO findings. Methods We conducted an e-survey of oncology clinicians and PRO researchers (supplemented by one-on-one clinician interviews) that randomized respondents to view one of the three line-graph formats (average scores over time for two treatments on four domains): (1) higher scores consistently indicating “better” patient status; (2) higher scores indicating “more” of what was being measured (better for function, Vol.:(011233456789) - Presented in part at the 2016 International Society for Quality of Life Research Annual Meeting. worse for symptoms); or (3) normed scores. Two formats displayed proportions changed (pie/bar charts). Multivariate modeling was used to analyze interpretation accuracy and clarity ratings. Results Two hundred and thirty-three clinicians and 248 researchers responded; ten clinicians were interviewed. Line graphs with “better” directionality were more likely to be interpreted accurately than “normed” line graphs (OR 1.55; 95% CI 1.01–2.38; p = 0.04). No significant differences were found between “better” and “more” formats. “Better” formatted graphs were also more likely to be rated “very clear” versus “normed” formatted graphs (OR 1.91; 95% CI 1.44–2.54; p < 0.001). For proportions changed, respondents were less likely to make an interpretation error with pie versus bar charts (OR 0.35; 95% CI 0.2–0.6; p < 0.001); clarity ratings did not differ between formats. Qualitative findings informed the interpretation of the survey findings. Conclusions Graphic formats for presenting PRO data differ in how accurately they are interpreted and how clear they 2 3 4 Johns Hopkins School of Medicine, 550 N. Broadway, Room 1111, Baltimore, MD 21205, USA Johns Hopkins School of Medicine, Baltimore, MD, USA Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway, Room 725, Baltimore, MD 21205, USA Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway, Room 726, Baltimore, MD 21205, USA Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway, Room 649, Baltimore, MD 21205, USA Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins, 1650 Orleans Street, Baltimore, MD 21287, USA Cancer Clinic of Southeastern Ontario, 25 King Street West, Kingston, ON K7L 5P9, Canada are perceived to be. These findings will inform the development of best practices for optimally reporting PRO findings. Introduction An emphasis on patient-centered care has increased the demand for patient-reported outcomes (PROs), data collected directly from patients about health conditions and impacts of treatments [ 1, 2 ]. PROs can inform patient care in a variety of ways [ 3, 4 ], including data representing ‘the voice of the patient’ in randomized clinical trials to inform decision-making by patients and clinicians based on trial results [ 4–7 ]. Oncologists have endorsed the use of PROs for this purpose, and there is evidence that PRO results can influence treatment choices [ 8–11 ]. The optimal integration of PRO results from clinical trials, and other comparative research into clinical care requires that clinicians understand and interpret PROs accurately and be able to communicate PRO findings to their patients where appropriate. This understanding can be challenging because of the variety of PRO questionnaires, variation in their scoring (e.g., higher scores indicating better or worse outcomes), their scaling (e.g., scores ranging from 0 to 100 as worst-to-best, or scores normed to a defined population), and how statistical and clinical significance of the findings are addressed [ 9, 12–15 ]. This study was part of a larger research program designed to examine approaches for presenting PRO data to promote their understanding and use. Previously, we evaluated existing approaches to presenting study PRO results [ 15 ]. We then partnered with stakeholder workgroups of clinicians and patients to develop improved graphical presentation approaches [ 14 ]. In a separate research stream, we focused on the communication of PROs to patients (e.g., in educational materials or decision aids) [ 16 ]. In this study, we focused on the reporting of PROs from clinical trials to clinicians and PRO researchers, building on our earlier (...truncated)


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs11136-017-1710-6.pdf

Michael Brundage, Amanda Blackford, Elliott Tolbert, Katherine Smith, Elissa Bantug, Claire Snyder, PRO Data Presentation Stakeholder Advisory Board (various names and locations). Presenting comparative study PRO results to clinicians and researchers: beyond the eye of the beholder, Quality of Life Research, 2017, pp. 1-16, DOI: 10.1007/s11136-017-1710-6