How to increase the potential policy impact of environmental science research

Bilotta et al. Environmental Sciences Europe How to increase the potential policy impact of environmental science research Gary S Bilotta 0 1 Alice M Milner 0 2 3 Ian L Boyd 0 3 4 0 Optimise the directness of the study Policy-makers can make more use of evidence from studies that examine similar processes/populations/environments/ ecosystems to that of policy interest, including consider- ation of the appropriateness of the temporal and spatial scales of observations. For example, one of the criticisms of the scientific evidence of the impacts of neonicotinoid in- secticides on insect pollinators centres on the use of labora- tory conditions to simulate exposures in the wild. In their article 'A restatement of the natural science evidence base concerning neonicotinoid insecticides and insect pollina- tors' , Godfray et al. [2] state that 'the strengths of laboratory studies are that they allow carefully controlled experiments to be performed on individual insects subjected to 1 School of Environment and Technology, University of Brighton , Brighton BN2 4GJ , UK 2 Department of Geography, Royal Holloway, University of London , Egham, Surrey TW20 0EX , UK 3 Department for Environment, Food and Rural Affairs , London SW1P 3JR , UK 4 College Gate, University of St Andrews , St Andrews, Fife KY16 9AJ , UK This article highlights eight common issues that limit the policy impact of environmental science research. The article also discusses what environmental scientists can do to resolve these issues, including (1) optimising the directness of their study so that it examines similar processes/populations/environments/ecosystems to that of policy interest; (2) using the most powerful study design possible, to increase confidence in the identified causal mechanisms; (3) selecting a sufficient sample size, to reduce the chance of false positives/negatives and increase policy-makers' confidence in extrapolation of the findings; (4) minimizing the risk of bias through randomization of study units to treatment and control groups (reducing the risk of selection bias), blinding of study units and investigators (reducing the risk of performance and detection bias), following-up study units from enrolment to study completion (reducing the risk of attrition bias) and prospectively registering the study on a publically-available platform (reducing the risk of reporting and publication bias); (5) proving that statistical analyses meet test assumptions by reporting the results of statistical assumption checks, ideally publishing full datasets online in an open-access format; (6) publishing the research whether statistically significant or not, policy-makers are just as interested in the negative or insignificant results as they are in the positive results; (7) making the study easy to find and use, the title and abstract of an article are of high importance in determining whether articles are examined in detail or not and used to inform policy; (8) contributing towards systematic reviews on environmental topics, to provide policy-makers with comprehensive, reproducible and updateable syntheses of all the evidence on a given topic. Policy; Environmental science; Study design; Bias; Statistical power; Randomization; Statistical validity; Systematic review - Background Evidence from environmental science is used to inform public policies but those policies sometimes deviate in significant ways from what the evidence may seemingly support. This can be a source of frustration for some environmental scientists. Frequently, deviation of policy from scientific evidence occurs because policy implementation is multidimensional and includes electoral, ethical, cultural, practical, legal and economic considerations [1]. Occasionally, deviation of policy from apparent scientific evidence occurs because of problems surrounding the quality or reporting of the scientific evidence itself. The following eight sections of this article identify common issues associated with evidence from environmental science well-defined exposure. The weaknesses are that they are conducted under very artificial conditions (which may affect tolerance to external stress), any avoidance response by the insect is limited and hence the exposure dose and form is determined solely by the experimenter, and responses at the colony or population level are both difficult to study and to extrapolate to the field [2]. The directness of a study is something that depends on the purpose for which the study is to be used, and this may only become fully apparent after a study is published. However, optimising directness of a study to policy questions can often be considered from the onset of study design (in balance with the degree of experimental control). Scientists can aid the decision by policy-makers, who are considering how similar a study is to the situation of policy interest, by reporting as much background information as possible on the study units and the conditions of the study. Use the most powerful study design possible Study design underlies how much confidence policymakers will have in the findings. Non-randomised observational studies that lack control groups and simply report correlations between variables will typically attract less confidence than a randomised controlled study on the same topic. This is because policy-makers recognise that correlation between observations does not signify causation. Tyler Vigen has created a website called Spurious Correlations, which demonstrates this point in a number of amusing ways. Vigen trawls data sets and matches parameters until he comes up with a correlation. In the example shown in Figure 1, Vigen presents the correlation (0.99) between US spending on science, space, and technology and Suicides by hanging, strangulation and suffocation. The more data are trawled for patterns, the more likely it is that the patterns found will simply reflect chance associations. This might be innocuous as long as we are comparing clearly unrelated variables, such as those shown in Figure 1. But if environmental scientists find a chance correlation between two variables that just happen to have a plausible functional cause and effect relationship, then there is a higher risk of misinterpretation. Where there is a choice, this is why policy-makers will often place more confidence in evidence from scientific studies that have both control groups and treatment groups, and where the individual study units have been allocated to these different groups based on some random allocation process that is not possible to predict. Without a control group (i.e. study units that are dealt with in exactly the same way as study units in the experimental group except for the treatment applied), it is difficult to determine whether a given treatment really had an effect or whether, for example, there was a natural change over time in the outcome of interest that may be unconnected with the treatment. Witho (...truncated)