Evaluation of the clinical effectiveness of a work-based mentoring programme to develop clinical reasoning on patient outcome: A stepped wedge cluster randomised controlled trial

RESEARCH ARTICLE Evaluation of the clinical effectiveness of a work-based mentoring programme to develop clinical reasoning on patient outcome: A stepped wedge cluster randomised controlled trial Aled Williams ID1☯*, Alison Rushton ID2☯, James J. Lewis3☯, Ceri Phillips4☯ a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 1 University Hospital of Wales Physiotherapy Department, Cardiff and Vale University Health Board, Cardiff, Wales, United Kingdom, 2 School of Sport, Exercise and Rehabilitation Sciences, University of Birmingham, Birmingham, England, United Kingdom, 3 Y Lab (Public Services Innovation Lab for Wales), School of Social Sciences, Cardiff University, Cardiff, Wales, United Kingdom, 4 College of Human and Health Sciences, Swansea University, Swansea, Wales, United Kingdom ☯ These authors contributed equally to this work. * Abstract OPEN ACCESS Citation: Williams A, Rushton A, Lewis JJ, Phillips C (2019) Evaluation of the clinical effectiveness of a work-based mentoring programme to develop clinical reasoning on patient outcome: A stepped wedge cluster randomised controlled trial. PLoS ONE 14(7): e0220110. https://doi.org/10.1371/ journal.pone.0220110 Editor: Christophe Leroyer, Universite de Bretagne Occidentale, FRANCE Background Despite persistent calls to measure the effectiveness of educational interventions on patient outcomes, few studies have been conducted. Within musculoskeletal physiotherapy, the effects of clinical mentoring on postgraduate physiotherapists have been explored, but its impact on patient outcomes is unknown. The objective of this trial was to assess the effectiveness of a work-based mentoring programme to facilitate physiotherapist clinical reasoning on patient outcomes. Received: January 31, 2019 Accepted: July 9, 2019 Published: July 31, 2019 Copyright: © 2019 Williams et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: All data files are available from the figshare database (https:// figshare.com/s/cbd137213b638b69a2d0). Funding: The authors received no specific funding for this work. Competing interests: The authors have declared that no competing interests exist. Methods In a stepped-wedge cluster RCT in the musculoskeletal physiotherapy outpatient departments of a large NHS organisation, 16 physiotherapists were randomised by cluster to receive the intervention—150 hours of mentored clinical practice—at one of 3 time periods; control was usual training. 441 patients submitted outcome measures: Patient-Specific Functional Scale (PSFS) (primary outcome measure), EQ-5D-5L, patient activation and patient satisfaction (secondary outcome measures). A further secondary outcome measure of physiotherapist performance was collected by an independent assessor observing the physiotherapists practice. Results 80.0% of intervention patients achieved clinically significant PSFS scores compared with 63.8% of control patients. Binary logistic regression analysis modelling for time, cluster and patient characteristics showed strong statistical evidence for this difference (p = 0.023; odds PLOS ONE | https://doi.org/10.1371/journal.pone.0220110 July 31, 2019 1 / 21 The impact of work-based mentoring on patient outcome ratio 4.24, 95%CI 1.22, 14.79). Physiotherapist performance scores improved from a mean of 47.8% (SD 3.60) pre-intervention to a mean of 56.0% (SD 4.24) (p<0.001). There was no statistical evidence for differences between groups on other secondary outcomes. Conclusion This is the first study that we aware of that provides patient outcomes measurement of an established educational intervention in physiotherapy, providing evidence that this type of intervention positively impacts patient outcomes and physiotherapist performance. This provides a basis for further research in education across other healthcare disciplines and outcome measures. Introduction Minimal research has investigated the effect of healthcare education and training on quality of care and patient outcome. The paucity of evidence has led the UK Department of Health to call for research investigating healthcare education and patient outcomes [1], specifically measuring staff and patient experience, clinical effectiveness and safety [2]. This call to measure the association between healthcare intervention and patient outcome is not new in the medical education literature, where it has long been argued that the education and development of clinicians should be evaluated on the basis of whether they achieve better health outcomes [3–8]. Different levels of evidence have been outlined, advocated and widely used for healthcare education, as illustrated by Kirkpatrick’s 4-level method for training evaluation [9] and Moore’s 7-level [10] outcomes model of continuing medical education. Table 1 below illustrates the longer established Kirkpatrick’s model along with examples of typical research measurements: It has been generally acknowledged that “Level 4 evidence” of patient outcomes on Kirkpatrick’s scale is difficult to obtain [13–17] commonly leading to lower level “surrogate outcomes” such as competency and clinician performance being used to evaluate educational interventions [3, 7]. Where level 4 outcomes have been obtained in medicine [18, 19] and nursing [20–22] the most frequently used outcomes to evaluate training and education are mortality rates, length of time in theatre, length of stay in hospital, complication rates and patient satisfaction. While this represents progress within the published research by exploring experience and safety (adverse events and patient mortality rates], measuring the clinical effectiveness of clinician education on patient outcomes continues to lack published evidence. Table 1. Kirkpatrick’s 4 levels of training evaluation [11]. Levels and Descriptors [11] Examples [12] Level 1: Reaction A measure of the satisfaction of the participants who attended the program Survey Level 2: Learning The extent to which participants increased their knowledge, learned or improved present skills, or changed their attitudes Test/examination Level 3: Behaviour The extent to which participants applied what they learned when they returned to their jobs Ratings by supervisor, peers, patients; direct observation Level 4: Results The improvement of morale, the increase in sales or Validated patient outcome scales production, the reduction in turnover, the increase in customer satisfaction, the return-on-investment, and any other benefits that came from attending the program. https://doi.org/10.1371/journal.pone.0220110.t001 PLOS ONE | https://doi.org/10.1371/journal.pone.0220110 July 31, 2019 2 / 21 The impact of work-based mentoring on patient outcome Multiple authors across specialties in healt (...truncated)