A randomised controlled trial of email versus mailed invitation letter in a national longitudinal survey of physicians

PLOS ONE RESEARCH ARTICLE A randomised controlled trial of email versus mailed invitation letter in a national longitudinal survey of physicians Benjamin Harrap1, Tamara Taylor2, Grant Russell3, Anthony Scott ID4* a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Harrap B, Taylor T, Russell G, Scott A (2023) A randomised controlled trial of email versus mailed invitation letter in a national longitudinal survey of physicians. PLoS ONE 18(8): e0289628. https://doi.org/10.1371/journal. pone.0289628 Editor: Fares Alahdab, Houston Methodist Academic Institute, UNITED STATES Received: August 25, 2022 Accepted: July 21, 2023 Published: August 22, 2023 Peer Review History: PLOS recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. The editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0289628 Copyright: © 2023 Harrap et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Data can requested using the following non-author email: at the Melbourne Institute: Applied Economic and Social Research at the 1 Centre for Health Policy, Melbourne School of Population and Global Health, The University of Melbourne, Carlton, Victoria, Australia, 2 Government and Social Research Division, Big Village, Melbourne, Australia, 3 Department of General Practice, Monash University, Melbourne, Victoria, Australia, 4 Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia * Abstract Despite their low cost, the use of email invitations to distribute surveys to medical practitioners have been associated with lower response rates. This research compares the difference in response rates from using email approach plus online completion rather than a mailed invitation letter plus a choice of online or paper completion. A parallel randomised controlled trial was conducted during the 11th annual wave of the nationally representative Medicine in Australia: Balancing Employment and Life (MABEL) longitudinal survey of doctors. The control group was invited using a mailed paper letter (including a paper survey plus instructions to complete online) and three mailed paper reminders. The intervention group was approached in the same way apart from the second reminder when they were approached by email only. The primary outcome is the response rate and the statistical analysis was blinded. 18,247 doctors were randomly allocated to the control (9,125) or intervention group (9,127), with 9,108 and 9,107 included in the analysis. Using intention to treat analysis, the response rate in the intervention group was 35.92% compared to 37.59% in the control group, a difference of -1.66 percentage points (95% CI: -3.06 to -0.26). The difference was larger for General Practitioners (-2.76 percentage points, 95% CI: -4.65 to -0.87) compared to other specialists (-0.47 percentage points, 95% CI: -2.53 to 1.60). For those who supplied an email address, the average treatment effect on the treated was higher at -2.63 percentage points (95% CI: -4.50 to -0.75) for all physicians, -3.17 percentage points (95% CI: -5.83 to -0.53) for General Practitioners, and -2.1 percentage points (95% CI: -4.75 to 0.56) for other specialists. For qualified physicians, using email to invite participants to complete a survey leads to lower response rates compared to a mailed letter. Lower response rates need to be traded off with the lower costs of using email rather than mailed letters. Background Web surveys have consistently lower response rates than all other survey modes [1]. Surveys of medical practitioners remain a key source of information about clinical practice, health service delivery, and clinical attitudes and experience. A key issue with survey data is that they can PLOS ONE | https://doi.org/10.1371/journal.pone.0289628 August 22, 2023 1 / 13 PLOS ONE University of Melbourne. In addition, data can be requested from the Australian Data Archive https:// dataverse.ada.edu.au/dataverse/mabel. This provides access to main MABEL survey data but not the data used in this trial. Funding: This research used data from the MABEL longitudinal survey of doctors. Funding for MABEL was provided by the National Health and Medical Research Council (2007 to 2016: 454799 and 1019605); the Australian Department of Health and Ageing (2008); Health Workforce Australia (2013); The University of Melbourne, Medibank Better Health Foundation, the NSW Department of Health, and the Victorian Department of Health and Human Services (2017); and the Australian Government Department of Health, the Australian Digital Health Agency, and the Victorian Department of Health and Human Services (2018). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. AS was the Chief Investigator for all above grants but did not receive salary or payments. BH and TT were employed using the above research funding. GR did not receive salary or payments. Competing interests: The authors have declared that no competing interests exist. Email versus mail in a survey of physicians have low external validity because the sample may be less representative due to response bias caused by recruitment methods and non-random selection of physicians completing the survey. Although a low response rate does not necessarily mean low external validity [2], the focus on response rates remains a key feature of the survey methods literature for physicians [3–5]. Systematic reviews and meta-analyses have examined different methods of increasing response rates in surveys of medical practitioner populations [3,6–8], such as changing features of survey design and delivering incentives. Email contact and online survey completion is popular as costs are lower but research has shown that response rates also tend to be lower, with a mailed approach more effective and recommended [3]. For example, in a meta-analysis of 48 studies of health professionals, three studies found that mailed surveys were associated with higher response rates than online/web modes, with no difference in response rates between online modes and mixed modes [3]. Pit, Vo (6) conducted a systematic review of methods used to increase response rates for GPs, and found postal surveys were more effective than phone or email surveys (as a singular method of distribution), and a sequential mixedmode of reminders was more effective than using online only or online and paper surveys concurrently. Beebe, Jacobson [9] found that a sequential mixed mode (web followed by mail) survey of health professionals had a hi (...truncated)