Acceptance checklist for clinical effectiveness pilot trials: a systematic approach

BMC Medical Research Methodology Acceptance checklist for clinical effectiveness pilot trials: a systematic approach Georgina Charlesworth 0 1 Karen Burnell 2 Juanita Hoe 4 Martin Orrell 0 4 Ian Russell 3 0 Research and Development Department, North East London NHS Foundation Trust, Goodmayes Hospital , Barley Lane, Ilford, Essex IG3 8XJ, England 1 Research Department of Clinical , Educational and Health Psychology , University College London , 1-19 Torrington Place, London WC1E 7HB , UK 2 School of Health Sciences and Social Work , James Watson (West), 2 King Richard 1st Road, Portsmouth PO1 2FR, England 3 West Wales Organization for Rigorous Trials in Health, Swansea University College of Medicine , Singleton Park, Swansea SA2 8PP, Wales 4 UCL Mental Health Sciences Unit, University College London , Charles Bell House, 67-73 Riding House Street, London W1W 7EJ , UK Conducting a pilot trial is important in preparing for, and justifying investment in, the ensuing larger trial. Pilot trials using the same design and methods as the subsequent main trial are ethically and financially advantageous especially when pilot and main trial data can be pooled. For explanatory trials in which internal validity is paramount, there is little room for variation of methods between the pilot and main trial. For pragmatic trials, where generalisability or external validity is key, greater flexibility is written into trial protocols to allow for 'real life' variation in procedures. We describe the development of a checklist for use in decision-making on whether pilot data can be carried forward to the main trial dataset without compromising trial integrity. We illustrate the use of the checklist using a pragmatic trial of psychosocial interventions for family carers of people with dementia as a case study. Pilot trial; Internal pilot; Randomized controlled trial; Feasibility; CONSORT; ACCEPT; SHIELD; Carer supporter programme - Introduction Pilot studies focus on the science, process, management and resources of planned studies [1]. Their purpose is to increase the likelihood of success of interventions in subsequent larger studies by ensuring they are appropriate and effective in practice [2-4]. Pilot studies help to justify the investment of money and time in those subsequent studies [5]. Hence they are often a requirement of funding bodies [6]. The aims of this article are to: (1) consider the nature and extent of permissible changes to procedures between the pilot and main trial without breaching methodological integrity (2) propose a checklist of areas where amendments to trial procedures may arise (3) illustrate the process of deciding whether to accept data from pilots into main trials. Background Pilot trials have been defined as miniature versions of full trials conducted to ensure the detailed design, methods and procedures are all robust and fit for purpose [7]. The term pilot trial should not be confused with the term pilot study. The latter has been used synonymously with the term feasibility study in which trial processes, resources, management and scientific factors are scrutinised in order to facilitate the planning of large-scale investigations [8]. Recent reviews have highlighted a variety of situations where the term pilot trial has been used, or, some would argue, misused. For example: as a substitute for hypothesistesting studies rather than being preparation for a full trial; to make inadequately powered studies sound more attractive to publishers; and, to report on outcomes rather than design, methods and procedures [2,9,10]. Having established the definition of pilot trials as full trials in miniature we next need to define the common subtypes of pilot trial, namely the internal and external pilot. Lancaster and colleagues define external pilot trials as stand alone pieces of work, planned and conducted independently of the main trial [6]. In contrast internal pilot trials are set up with the intention of being incorporated into the main trial. Lancaster et al. described the purpose of pilot trials as being limited to collection of data for sample size re-calculation such that pilots do not allow for the pre-testing of the feasibility of other factors relating to the trial. However, we suggest that a degree of feasibility testing should be allowable in the internal pilots of pragmatic trials, and that scientific integrity can be maintained as long as the degree of variation to procedure is within the limits of variation likely to be seen in the full pragmatic trial. Treweek and Zwarenstein have argued that if we want evidence from trials to be used in clinical practice and policy, trialists should make every effort to make their trials widely applicable, which means trials should be pragmatic in attitude [11]. It would seem pragmatic to extend this attitude to internal pilots. There are considerable advantages to carrying data forward from pilot trials, including: reduced cost, reduced burden on study populations, increased numbers of participants in the full trial, and maintenance of momentum from pilot to main trial [12]. However, the scientific integrity of a trial requires that the aims and methods of the pilot study, if not its geographical extent, match those of the full trial. If feed forward of data is to take place from pilot to full trial, those charged with overseeing the adequacy of trial conduct, for example Trial Steering Committees (TSCs) and Data Monitoring and Ethics Committees (DMECs) need a structured approach to identifying and documenting any protocol developments that take place after the start of the planned internal pilot. Within the literature on trial methodology, there are existing tools for reporting pilot trials and for defining success in pilot studies. For example, Thabane et al. [8] highlight the importance of assessing the success of a pilot study based on predefined criteria, typically relating to recruiting and retaining participants. Their approach is valuable when assessing feasibility and acceptability of study methods, but is not focussed on decision-making relating to the appropriateness or otherwise of carrying data forward to the main trial dataset. They suggest four potential outcomes for pilot studies: stop; continue but modify protocol (feasible with modifications); continue without modifications but monitor closely (feasible with close monitoring); or continue without modification (feasible as is). Where a decision has been made to stop then there should be no carry forward of data, whereas decisions to continue without modification would allow for data carry-forward. Within the category of continue but modify protocol (feasible with modifications) it may or may not be appropriate to carry data forward to the full trial. It is within this category that further consideration is required regarding the nature of modifications and their potential impact on the integrity of the final dataset. In considering app (...truncated)