Setting global research priorities for child protection in humanitarian action: Results from an adapted CHNRI exercise

PLOS ONE, Aug 2018

Background Armed conflict, natural disaster, and forced displacement affect millions of children each year. Such humanitarian crises increase the risk of family separation, erode existing support networks, and often result in economic loss, increasing children’s vulnerability to violence, exploitation, neglect, and abuse. Research is needed to understand these risks and vulnerabilities and guide donor investment towards the most effective interventions for improving the well-being of children in humanitarian contexts. Methods The Assessment, Measurement & Evidence (AME) Working Group of the Alliance for Child Protection in Humanitarian Action (ACPHA) identified experts to participate in a research priority setting exercise adapted from the Child Health and Nutrition Research Initiative (CHNRI). Experts individually identified key areas for research investment which were subsequently ranked by participants using a Likert scale. Research Priority Scores (RPS) and Average Expert Agreement (AEA) were calculated for each identified research topic, the top fifteen of which are presented within this paper. Results Intervention research, which aims to rigorously evaluate the effectiveness of standard child protection activities in humanitarian settings, ranked highly. Child labor was a key area of sector research with two of the top ten priorities examining the practice. Respondents also prioritized research efforts to understand how best to bridge humanitarian and development efforts for child protection as well as identifying most effective way to build the capacity of local systems in order to sustain child protection gains after a crisis. Conclusions Rigorous, scientific research that assesses the scope of child protection risks, examines the effectiveness of interventions to improve child well-being, and translates evidence to practice is critical. Findings from this research priority setting exercise offer guidance for a global research agenda on child protection in humanitarian settings, encouraging cooperation among donors, implementers, and academics to pursue a coordinated approach to evidence generation.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0202570&type=printable

Setting global research priorities for child protection in humanitarian action: Results from an adapted CHNRI exercise

August Setting global research priorities for child protection in humanitarian action: Results from an adapted CHNRI exercise Laura Gauer Bermudez 0 1 Katharine Williamson 1 Lindsay Stark 1 0 Columbia University School of Social Work , New York , New York, United States of America, 2 Save the Children , London , United Kingdom , 3 Department of Population and Family Health, Columbia University Mailman School of Public Health , New York , New York, United States of America, 4 George Warren Brown School of Social Work, Washington University in Saint Louis , St. Louis, Missouri , United States of America 1 Editor: Emma Sacks, Johns Hopkins School of Public Health , UNITED STATES - Data Availability Statement: All relevant data are within the paper and its supporting information files. Funding: The study was made possible with funds from Save the Children provided to the Alliance for Child Protection in Humanitarian Action, https:// www.savethechildren.net. Co-author, Katharine Williamson, is a co-leader of the Alliance's AME task force and is also an employee of Save the Children, serving as their Senior Humanitarian Child Protection Advisor. Background Armed conflict, natural disaster, and forced displacement affect millions of children each year. Such humanitarian crises increase the risk of family separation, erode existing support networks, and often result in economic loss, increasing children's vulnerability to violence, exploitation, neglect, and abuse. Research is needed to understand these risks and vulnerabilities and guide donor investment towards the most effective interventions for improving the well-being of children in humanitarian contexts. Methods The Assessment, Measurement & Evidence (AME) Working Group of the Alliance for Child Protection in Humanitarian Action (ACPHA) identified experts to participate in a research priority setting exercise adapted from the Child Health and Nutrition Research Initiative (CHNRI). Experts individually identified key areas for research investment which were subsequently ranked by participants using a Likert scale. Research Priority Scores (RPS) and Average Expert Agreement (AEA) were calculated for each identified research topic, the top fifteen of which are presented within this paper. Results Intervention research, which aims to rigorously evaluate the effectiveness of standard child protection activities in humanitarian settings, ranked highly. Child labor was a key area of sector research with two of the top ten priorities examining the practice. Respondents also prioritized research efforts to understand how best to bridge humanitarian and development efforts for child protection as well as identifying most effective way to build the capacity of local systems in order to sustain child protection gains after a crisis. Competing interests: The authors have declared that no competing interests exist. Abbreviations: ACPHA, Alliance for Child Protection in Humanitarian Action; AEA, Average Expert Agreement; AME, Assessment, Measurement & Evaluation Working Group; CHNRI, Child Health and Nutrition Research Initiative; CP AoR, Child Protection Area of Responsibility; RPS, Research Priority Scores; UNHCR, United Nations High Commissioner for Refugees; NGO, Non-Governmental Organizations; UN, United Nations; UNICEF, United Nation Children's Fund. Conclusions Rigorous, scientific research that assesses the scope of child protection risks, examines the effectiveness of interventions to improve child well-being, and translates evidence to practice is critical. Findings from this research priority setting exercise offer guidance for a global research agenda on child protection in humanitarian settings, encouraging cooperation among donors, implementers, and academics to pursue a coordinated approach to evidence generation. Introduction The number of people affected by humanitarian crises is on the rise, perpetuated by armed conflict and natural disasters [ 1 ]. In 2017, there were over 65 million forcibly displaced people, over half of whom were under the age of 18 [ 2 ]. In addition, over one billion children live in countries affected by armed conflict [ 3 ]. Environmental factors, including climate change, are likely to increase the number of conflicts and intensify the severity of natural disasters [4±5]. Armed conflicts and large-scale disasters increase the potential for family separation and the erosion of existing support systems, putting children at risk of abuse, exploitation, violence, and neglect. The widespread economic shocks that often accompany humanitarian crises create further vulnerabilities for children when households employ negative coping strategies to manage economic stress. In Lebanon, where over one million Syrian refugees have been registered with the United Nations High Commissioner for Refugees (UNHCR), child marriage and child labor have been reported as families struggle financially [6±7]. Children in circumstances of economic and physical insecurity are also at risk of child trafficking, sexual exploitation, and recruitment by armed forces and extremist groups. Within these contexts, child protection experts in non-governmental organizations (NGO), multilateral institutions such as the UN Children's Fund and the United Nations High Commissioner for Refugees, work to prevent and respond to incidents of abuse, neglect, exploitation, and violence against children. These efforts can take the form of broader systemsstrengthening interventions that seek to build the capacity of national actors to implement effective social support systems that care for children and families, both in formal and informal spheres. As a complement to systems strengthening, child protection initiatives may also take the form of direct implementation, such as the establishment of ªChild Friendly Spaces (CFS)º that allow children safe zones to play, parenting trainings that emphasize alternatives to physical punishment, or family tracing and reunification for unaccompanied or separated children. Yet, the assumptions that drive such child protection efforts in humanitarian practice have not yet been fully based on scientific evidence. Protection risks are often estimated and prioritized based on anecdotal accounts [ 8 ], definitions of child protection concepts are often not standardized [ 9 ], and there is scant evidence on the effectiveness of many of the sector's universally agreed upon standard interventions [10±12]. To begin addressing these gaps in empirical research within the sector of child protection in humanitarian contexts, a research priority setting exercise, adapted from the Child Health and Nutrition Research Initiative (CHNRI), was undertaken to identify and rank research priorities. This manuscript presents the process and results of this participatory ranking methodology designed to guide future research investment. 2 / 15 Methods The Child Health and Nutrition Research Initiative (CHNRI) was designed as a tool to help guide policy and investment in global health research, specifically children's health. CHNRI has since been used to establish research priorities across a broad array of global health disciplines [13±20]. The method is comprised of four stages (i) determining the boundaries of investigation and creating evaluation criteria; (ii) obtaining and systematically listing input from key stakeholders on critical priorities/tasks (referred to as ªresearch questionsº) to address gaps in sectoral evidence or knowledge; (iii) enlisting stakeholders to rank the research questions based on a pre-defined set of evaluation criteria; (iv) calculation of research priority scores and agreement between experts (Fig 1]. A more detailed explanation of the CHNRI method has been published elsewhere [21±23]. The present study was commissioned by the Assessment, Measurement and Evidence Working Group of the Alliance for Child Protecton in Humanitarian Action (ACPHA) and was informed by prior consensus-building efforts in the sector [24±25]. In collaboration with a Lead Researcher, the CHNRI method was adapted to prioritize research topics in the sector of child protection in humanitarian settings. For the purposes of this exercise, a `humanitarian setting' was defined as ªacute or chronic situations of conflict, war or civil disturbance, natural disaster, food insecurity or other crises that affect large civilian populations and result in significant excess mortalityº [ 26 ]. The goal of `child protection' efforts are ªto protect children from abuse, neglect, exploitation, and violenceº [ 27 ]. And `children' were defined as ªindividuals under the age of 18º [ 28 ]. Experts working on issues of child protection in humanitarian settings were then invited to take part in semi-structured interviews to discuss the gaps in knowledge and evidence that Fig 1. Research priorities setting exercise flow diagram. 3 / 15 existed within the sector and to generate research priorities to address these gaps. Forty-seven experts participated in this first round of evidence generation with representatives from NonGovernmental Organizations (NGOs), United Nations (UN) agencies, donor agencies, and research institutions. Experts were initially identified through three coordination bodies±the Alliance for Child Protection in Humanitarian Action (ACPHA), the Child Protection Area of Responsibility (CP AoR), and UNHCR with the network extended through snowball sampling. Respondents were strategically diversified to include inputs from those involved in various child protection job functions including implementation, coordination, policy development, and academia from a range of geographic locations (Table 1]. Recruitment continued on a rolling basis and ended once data saturation, defined as the point at which no new data were being generated, was achieved. The final sample was consistent with previous research that identified 45±55 as the number of experts at which collective opinion stabilizes [ 29 ]. Aligned with prior CHNRI studies in humanitarian contexts [ 14 ], interviews were held via Skype with experts notified in advance that they would be requested to provide their opinions on the most important areas for investment to improve the state of evidence in the field of child protection in humanitarian settings in the next 3±5 years. Participants were encouraged to follow up by email in the event they were able to generate further ideas after the interview had concluded. Through an iterative process, the Lead Researcher then collated 24 hours of interview notes to identify 90 unique research priorities, condensing interrelated research ideas and simplifying concepts for use in the ranking exercise. The priorities were then thematically organized into the following pre-determined themesÐEpidemiological Research; Policy and Systems Research; and Intervention Research (Table 2]. The research team provided review and consensus on the themes and categorization after which the areas for research were listed within the online survey. The survey was pilot tested by individuals who were not involved in the n (%) 19 (40.4) 28 (59.6) 26 (55.3) 6 (12.8) 6 (12.8) 7 (14.9) 1 (2.1) 4 (8.5) 1 (2.1) 15 (31.9) 45 (95.7) 31 (66.0) 20 (42.6) 11 (23.4) Abbreviation Number of Items EPI PSR INT 22 13 development of research questions but who had general knowledge of humanitarian concepts and survey design. Further, to ensure that question order did not bias results, we implemented a page randomization that shuffled page order within the survey for each new respondent. Experts who participated in the interview process were invited to take part in the online ranking portion of the prioritizatio exercise. Two additional experts who were either not previously available or who reached out to participate after the period for interviews had passed, were also invited to take part in survey. Each of the 90 research priorities were ranked on four criteria: (i) Relevance±research will support learning that contributes to the prevention and response to abuse, neglect, exploitation, or violence in humanitarian settings; (ii) Feasibility±research is feasible to conduct in an ethical way; (iii) Originality±research will generate new findings or methods; and (iv) Applicability±research will be readily applied to programs and policies. Relative weights were not assigned to scoring criteria. For each research question, participants were offered six possible responses: strongly agree (5 points); agree (4 points); undecided (3 points); disagree (2 points); strong disagree (1 point); and insufficiently informed (considered non-applicable/no response). The scoring matrix was a deviation from past CHNRI studies which typically offered four possible responses±yes (1 point), no (0 points), undecided (0.5 points), and insufficiently informed/no response. In the development of the present research design, the study team elected to use a full Likert scale to allow for greater granularity when analyzing scores. Aligned with the CHNRI methodology [13±20], every research question was provided a priority score under each of the four judging criterion, calculated by taking the point totals and dividing them by the maximum number of points available, after excluding from the denominator those who did not answer the question or reported they were insufficiently informed, a percentage was calculated [ 14 ]. For each question, the overall Research Priority Score (RPS) was then calculated by taking the mean of the total priority scores for each judging criterion, as calculated above. Research questions were then ranked from highest to lowest on overall priority scores and the top fifteen presented in Table 3. Standard deviations for RPS are also included to show the variation between total priority scores for each judging criterion (Table 3, S1 Annex). 5 / 15 Research Type Relevance Feasibility Originality Applicability Average Expert Agreement AEA INT INT PSR INT PSR EPI PSR INT INT INT PSR EPI INT PSR INT In addition, the Average Expert Agreement (AEA) was calculated for each research question. In order to obtain AEA values, we consolidated ªstrongly agreeº and ªagreeº as well as ªstrongly disagreeº and ªdisagreeº. For each judging criterion, the number of modal responses was then divided by the total number of scorers for that question, again excluding those who did not answer the question or who reported they were insufficiently informed on the research question being assessed. Following this calculation, the ratios were then summed and divided by the number of judging criteria. Both RPS and AEA were calculated for the entire group of respondents as well as for subgroups, in order to analyze differences in priorities for those located in field settings as compared to those based in non-operational settings. Data were analyzed using Microsoft Excel. 6 / 15 Ethics statement Formal ethics review is usually not requested for undertaking CHNRI exercises [13±20] as the exercise does not involve personal or otherwise sensitive data. Participants were solicited via established professional networks whose purpose is to facilitate and enable information-sharing. Prior to participation in initial Skype interviews, all participants were informed on the nature of the research and the anonymity of their feedback. Results Of the 49 respondents invited to take part in the online ranking, 41 experts participated, eliciting a response rate of 83.7 percent. Research questions from all three of the research domains (epidemiological research; policy and systems research; and intervention research) featured in the top 15 research priorities. Intervention research was the most predominant domain voted upon by experts with 8 of the top 15 priorities identified falling within this realm. Policy and systems research followed with 5 priorities and epidemiologic research with only 2 featured priorities ranking in the top 15 (Table 3). The range of overall RPS was 63.28 to 86.33, with the highest ranked priority being the rigorous evaluation of the effectiveness of cash-based social safety nets to improve child wellbeing. Within the top 15 priorities, RPS ranged from 80.70 to 86.33. Intervention research which aims to rigorously evaluate the effectiveness of standard child protection activities provided in humanitarian settings ranked highly. Two questions concerning child labor, specifically estimating the prevalence and understanding the effectiveness of interventions to reduce the practice, ranked in the top ten priorities. Respondents also prioritized research efforts to understand how best to mobilize local systems, including the local social service workforce and para-social work models, in order to sustain child protection gains after international actors have departed a crisis. AEA scores ranged from 41.55 to 85.63, representing the percentage of respondents who provided the same score on a research priority (averaged across four judging criteria). For the top 15 research investment options, AEA ranged from 69.04 (to build the capacity of child protection sector staff in empirical research design and data analysis planning) to 85.75 (to evaluate the effectiveness of interventions to reduce child labor) (Table 3). We found higher levels of respondent agreement among research questions with higher RPS rankings, demonstrating that a certain level of consensus was attained in order for research topics to be prioritized in the higher ranks (Fig 2). Standard deviations (SD) were also analyzed in order to assess variation between the judging criterion. Among the top 15 reserch priorities, SDs ranged between 2.5 and 5.4 with the exception of the evaluation of psychosocial programming with an SD of 8.2 due to the comparatively lower score provided on Originality. This is likely due to the recent work on this particular topic that has been widely circulated [ 30 ] and therefore was deemed less original in the ranking process. When comparing all RPS scores among respondents who resided within an operational setting versus those who did not, there was a correlation co-efficient of 0.32, indicating a weak but positive association. The top ten research priorities differed between the two groups (Table 4). With the exception of rigorously evaluating family strengthening programs, which ranked highly for both groups of respondents, there were no other priorities that jointly ranked among the top ten. For field-based respondents, the most important initiative was to identify best practices for bridging humanitarian and development initiatives for child protection system strengthening. Field-based respondents tended towards the identification of best practices while also prioritizing capacity building for child protection sector staff in empirical 7 / 15 Fig 2. Top ten research priorities comparing RPS & AEA. research design and data analysis planning. In contrast, respondents who were not based in operational settings showed greater enthusiasm for the rigorous evaluation of interventions, with an examination of the effects of cash-based social safety nets on child well-being outcomes ranking highest. Discussion The limitations to rigorous research on child protection in humanitarian crises are notable, with harsh operational conditions, short project cycles, and inadequate funding all considered hindrances to scientific inquiry on child protection within these contexts [31±32]. However, recent efforts have begun to demonstrate that robust social science methodologies within the sector are both needed and possible [33±35]. This prioritization exercise, which is among the first known systematic inquiries on research investments for child protection in humanitarian contexts using the CHNRI methodology, offers initial insight on the research interests and evidence needs of sector experts. Intervention research comprised three of the top four research priorities, aligning with many previous CHNRI studies that have similarly found intervention research to be of importance to stakeholders [ 36 ]. As previously noted, there is a dearth of rigorous evaluation to determine the effectiveness of common child protection interventions in humanitarian settings. The lack of quantitative data to document intervention effectiveness inhibits the ability of humanitarian actors to design evidence-based programs, a hindrance increasingly problematic for funding appeals and policy advocacy. This prioritization suggests that understanding 8 / 15 Research Type Research Priority Score (RPS) PSR INT EPI PSR INT INT INT INT INT EPI INT INT INT PSR EPI INT PSR PSR INT INT intervention effectiveness is of particular interest to the sector, ranging from examinations of family-strengthening to capacity-building interventions to activities aimed at reducing child labor. Because the sample more heavily represents individuals in technical advisory and other operational capacities, the interest in intervention research most visibly highlights the needs of practitioners to have their programming rigorously tested and evaluated with respect to child well-being outcomes. As the top priority among both intervention research topics and the entire ranking exercise, understanding the effects of cash-based social safety nets on child well-being outcomes has emerged as highly importance for the sector. Cash transfers have gained prominence as multiple studies have found them effective in improving the welfare of children, including through improved health and nutrition outcomes as well as increased educational attainment [37±39]. The assumption driving the proliferation of cash-based social safety net interventions in humanitarian contexts is that they are an effective way of mitigating crisis-induced economic shocks, thereby preventing the use of coping strategies that may have negative effects on children such as school drop-out, child labor, and family separation. Yet, these assumptions have not been fully tested within disaster, conflict-affected, or displacement contexts, environments where children face unique risks and vulnerabilities. Further, the majority of existing evidence on the effects of cash transfers do not examine child protection outcomes such as reductions in violence, abuse, and exploitation, information of great interest to sector experts. 9 / 15 In addition to understanding the effectiveness of singular child protection interventions on child-well-being outcomes, experts indicated a need to also evaluate multi-sectoral interventions, considering this one of the highest priorities for research. A relatively broad mandate, this methodological research priority underscores the need for study designs that allow for the rigorous evaluation of multiple components within increasingly complex program designs, including analyses on how various components interact with one another. Such research endeavors are inherently more complicated, yet recent guidance from the global health sector has shown this to be a priority that spans disciplines within development and humanitarian assistance [40±42]. Similarly, as multi-sectoral and interdisciplinary interventions are prioritized by funders, experts within this study have identified a need to quantitatively demonstrate the added value of child protection interventions when mainstreamed within other sectors, such as health, nutrition, or education. Prior research on the effects of nutrition supplementation and play/stimulation on stunted children in Jamaica provides an example of how social scientists have captured the additive effects of non-sector related interventions [ 43 ]. If protection interventions are found to be effective in improving non-protection related outcomes for children, this type of evidence would support an argument that child protection considerations and/or program components are necessary to achieve desired results in other areas of humanitarian relief. Child labor in humanitarian settings was also a common theme with both intervention effectiveness and prevalence data among the top 10 priorities for research investment. Similar to cash transfers, child labor has been examined across multiple development settings [44±46], however, data from humanitarian contexts is extremely sparse and generally limited to anecdotal information. As urban environments have become a more common setting for humanitarian crises, there is an increased risk that children will be used for begging, street vending, and other forms of exploitation [47±48]. There is a need to understand the prevalence, dynamics, and effective interventions to reduce this protection risk for children who have been displaced as well as children from affected host communities. In order for child protection programming to be more responsive to current humanitarian contexts, experts felt that there was value in 1) better understanding the protection risks of children with disabilities (particularly non-observable disabilities) and 2) translating any existing evidence on implementing humanitarian programs in urban settings into more tangible guidance for CP practitioners. Disability inclusion has gained traction as a critical component within humanitarian assistance, however, experts noted this work to primarily address physical disabilities where programmatic accommodations are often tangible and straightforward, such as the fitting and distribution of assistive devices. In contrast, many experts noted feeling ill-equipped to properly serve children with cognitive and intellectual disabilities, agreeing that an examination of the protection risks for children with disabilities, particularly non-observable disabilities, should be prioritized. Similarly, experts felt more guidance on child protection programming in urban humanitarian crises would be beneficial. Indeed, as rapid urbanization has resulted in more densely population cities and towns, the potential impacts of a humanitarian crisis increase, particularly in areas with weak infrastructure and insufficient governance [ 49 ]. The Syrian refugee crisis has seen over 5 million people flee to neighboring countries, seeking refuge predominately in the cities and towns of Lebanon and Jordan with another 6 million internally displaced within Syria, again primarily in urban and peri-urban settings [ 50 ]. This trend differs from past decades of humanitarian assistance that was largely provided within camp-based settings, requiring a new framework for understanding how best to support children in crisis. Other actors within humanitarian response have begun to give this issue greater attention in the past several years [51±52] enabling the identified priority of secondary literature review and as 10 / 15 relevant, the translation and integration of evidence into child protection strategies and program design. Localization and sustainability were also key themes. Within the top 15 research priorities, experts conveyed a need to identify best practices for both engaging the local social service workforce in emergency settings and establishing sustainable para-social work models such that structures will exist past the duration of humanitarian intervention. At the same time, respondents would like to understand best practices for bridging humanitarian and development initiatives for child protection systems strengthening. Taken together, these items demonstrate a desire to understand how best to engage local social service structures (formal and informal) and connect the work done during a crisis to a longer-term development agenda. When scrutinizing the findings further, three trends emerged. First, among the top 15 research priorities, participants routinely scored research questions much higher for relevance than originality. It is speculated that this score variation may be a result of recent efforts by the sector to discuss and advocate for a more robust evidence base in humanitarian contexts [53± 55]. The relatively frequent discussion about these evidence needs may have made a number of research questions appear unoriginal to participants yet still highly relevant because the research had yet to be carried out. This finding highlights the readiness of child protection experts to move forward an actionable research agenda for humanitarian settings. Next, there were notable differences in the priorities of field and non-field based staff with only one research topic ranking within the top ten for both sub-groups (rigorously evaluate the effectiveness of family strengthening interventions to improve child well-being). As compared to non-field based respondents, those residing within an operational setting were less likely to identify rigorous evaluation within their top priorities. Instead, these respondents tended towards the identification of best practices, a logical reaction given that such research would presumably result in straight-forward guidance to program design. At the same time, field-based staff highly ranked capacity building in empirical research design and data analysis planning for the child protection sector, demonstrating a desire to build the skills required to further evidence generation. Lastly, our study explored research topics within the professional sector of ªchild protection in humanitarian settingsº, which had a rather expansive purview. As such, some of the research priorities identified by experts were similarly broad in scope. It is our hope that as the sector progresses in the collection and translation of rigorous evidence that future priority setting exercises on child protection in humanitarian settings will be able to focus on particular needs within narrower sub-specialties. Limitations The CHNRI method is based on purposive sampling where individuals are invited to participate based on their expertise in a given field. This method relies on a non-representative sample to aggregate knowledge and experiences. The findings are therefore limited to the perceptions of a discrete group of individuals and it is possible that additional areas for research investment may have emerged if a larger sample was recruited though, as earlier noted, prior quantitative work has demonstrated collective opinion to stabilize with as few as 45±55 participants [ 29 ], however, this finding was based on binary ªyesº or ªnoº responses as opposed to the Likert scale implemented in this project. Further, given the low-cost and replicability of the procedure, it is attractive to a variety of sectors as a means of fostering transparency and enhancing systematization in the creation of a research agenda. In our study. non-field based staff were more likely to respond to requests for interviews and as such, had greater representation within the study (Table 1). This created a certain level 11 / 15 of bias towards the insights and experiences of child protection experts currently based in non-operational settings. When secondarily analyzing results based on whether respondents resided in operational or non-operational settings, we did find variation in the prioritization of research items (Table 4). These findings indicate that even when saturation appears to have been reached, the rank ordering of priorities can be influenced by the characteristics of the sample. Deviating from standard CHNRI procedure, we requested that participants rank research priorities against pre-determined criteria using a Likert scale as opposed to binary ªyesº or ªnoº responses. This decision was informed by the lack of existing evidence within the sector of child protection in humanitarian action and the anticipation that a large majority of research items would be affirmatively ranked by respondents, making it difficult to discern which were of highest priority. While Likert scales have been used extensively in other crowdsourcing methods [56±58], more research is needed to examine the benefits and drawbacks of using a Likert scale within an adapted CHNRI framework. Lastly, our study did not include ªimpactº as a ranking criterion. Such a criterion would have participants rank research based on the likelihood it would result in a reduction of protection risks or improved responses to child protection violations. While our research criterion of ªrelevanceº included similar language, it did not explicitly request input on the ability of a research question, once answered, to impact the lives of children. Further research priority setting exercises on child protection may wish to include ªimpactº as a ranking criterion separate from ªrelevanceº in order to further ascertain the merit of a research idea. Conclusion Rigorous, scientific research that assesses the scope of child protection risks, examines the effectiveness of existing child protection interventions, and translates evidence to practice is critical to move the sector forward and respond to donor calls for programming that is evidence-based. This CHNRI adaptation solicited inputs from a range of sector experts with variation across geographic location and job function. It is our hope that findings can guide a global research agenda, facilitating cooperation among donors, implementers, and academics to pursue a coordinated approach to evidence generation. Supporting information S1 Annex. Full ranking. (DOCX) Acknowledgments Author Contributions Conceptualization: Katharine Williamson, Lindsay Stark. Data curation: Laura Gauer Bermudez. Formal analysis: Laura Gauer Bermudez. Methodology: Laura Gauer Bermudez. Supervision: Katharine Williamson, Lindsay Stark. The authors express gratitude to the participants who took time from their busy schedules to generate and rank research priorities for this exercise. 12 / 15 Writing ± original draft: Laura Gauer Bermudez. Writing ± review & editing: Katharine Williamson, Lindsay Stark. Wessells M. What are we learning about protecting children in the community? An inter-agency review of evidence on community based child protection mechanisms. London: Save the Children. 13 / 15 14 / 15 1. Burkle FM Jr, Martone G , Greenough PG . The changing face of humanitarian crises . Brown J. World Aff . 2013 ; 20 : 19 . 2. UNHCR. Figures at a glance: Statistical yearbooks . 2016 [Accessed 28 January 2018 ]. Available from: http://www.unhcr.org/en-us/ figures-at-a-glance .html 3. Jordans MJ , Pigott H , Tol WA . Interventions for children affected by armed conflict: a systematic review of mental health and psychosocial support in low-and middle-income countries . Current Psychiatry Reports . 2016 Jan 1 ; 18 ( 1 ):9. https://doi.org/10.1007/s11920-015-0648-z PMID: 26769198 4. Devlin C , Hendrix CS . Trends and triggers redux: Climate change, rainfall, and interstate conflict . Political Geography . 2014 Nov 30 ; 43 : 27 ± 39 . 5. Intergovernmental Panel on Climate Change. Climate Change 2014 ± Impacts, Adaptation and Vulnerability: Regional Aspects . Cambridge University Press; 2014 Dec 29. 6. Bartels SA , Michael S , Roupetz S , Garbern S , Kilzar L , Bergquist H , et al. Making sense of child, early and forced marriage among Syrian refugee girls: a mixed methods study in Lebanon . BMJ Global Health. 2018 Jan 1 ; 3 ( 1 ):e000509. https://doi.org/10.1136/bmjgh-2017 -000509 PMID: 29515914 7. DeJong J , Sbeity F , Schlecht J , Harfouche M , Yamout R , Fouad FM , et al. Young lives disrupted: gender and well-being among adolescent Syrian refugees in Lebanon . Conflict and Health . 2017 Nov; 11 ( 1 ): 23 . 8. Theis J , UNICEF WCARO. Notes on Child Protection Systems in Fragile States . 2012 [Accessed 15 May 2018 ]. Available from: https://www.unicef.org/wcaro/english/Notes_on_child_protection_systems_ in_fragile_states_-_ May _ 2012 .pdf 9. Stark L , Landis D. Violence against children in humanitarian settings: A literature review of populationbased approaches . Social Science & Medicine. 2016 Mar 31 ; 152 : 125 ± 37 . 10. Ager A , Metzler J , Vojta M , Savage K. Child friendly spaces: a systematic review of the current evidence base on outcomes and impact . Intervention. 2013 Jul 1 ; 11 ( 2 ): 133 ± 47 . 11. Noble E , Ward L , French S , Falb K. State of the Evidence: A Systematic Review of Approaches to Reduce Gender-Based Violence and Support the Empowerment of Adolescent Girls in Humanitarian Settings . Trauma, Violence, & Abuse . 2017 : 1524838017699601 . 13. Angood C , Khara T , Dolan C , Berkley JA . Research priorities on the relationship between wasting and stunting . PloS One . 2016 May 9 ; 11 ( 5 ):e0153221. https://doi.org/10.1371/journal.pone. 0153221 PMID: 27159235 14. Prudhon C , Maclaine A , Hall A , Benelli P , Harrigan P , Frize J . Research priorities for improving infant and young child feeding in humanitarian emergencies . BMC Nutrition . 2016 Dec; 2 ( 1 ): 27 . 15. Yoshida S , Martines J , Lawn JE , Wall S , Souza JP , et al. Cousens S. Setting research priorities to improve global newborn health and prevent stillbirths by 2025 . Journal of Global Health. 2016 Jun; 6 ( 1 ). 16. Bahl R , Martines J , Ali N , Bhan MK , Carlo W , Chan KY , et al. Research priorities to reduce global mortality from newborn infections by 2015 . The Pediatric Infectious Disease Journal. 2009 Jan 1 ; 28 ( 1 ): S43± 8 . 17. Kennedy HP , Yoshida S , Costello A , Declercq E , Dias MA , Duff E , et al. Asking different questions: research priorities to improve the quality of care for every woman, every child . The Lancet Global Health. 2016 Nov 1 ; 4 ( 11 ) :e777±9 . https://doi.org/10.1016/ S2214 -109X( 16 ) 30183 - 8 PMID: 27663682 18. Rudan I , El Arifeen S , Bhutta ZA , Black RE , Brooks A , Chan KY , et al. Setting research priorities to reduce global mortality from childhood pneumonia by 2015 . PLoS Medicine . 2011 Sep 27 ; 8 ( 9 ): e1001099. https://doi.org/10.1371/journal.pmed. 1001099 PMID: 21980266 19. Wazny K , Sadruddin S , Zipursky A , Hamer DH , Jacobs T , Kallander K , et al. Setting global research priorities for integrated community case management (iCCM): Results from a CHNRI (Child Health and Nutrition Research Initiative) exercise . Journal of Global Health. 2014 Dec; 4 ( 2 ). 20. Tomlinson M , Rudan I , Saxena S , Swartz L , Tsai AC , Patel V . Setting priorities for global mental health research . Bulletin of the World Health Organization. 2009 Jun; 87 ( 6 ): 438 ± 46 . https://doi.org/10.2471/ BLT.08.054353 PMID: 19565122 21. Rudan I. Setting health research priorities using the CHNRI method: IV. Key conceptual advances . Journal of Global Health. 2016 Jun; 6 ( 1 ). 22. Rudan I , Gibson JL , Ameratunga S , Arifeen SE , Bhutta ZA , Black M , et al. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method . Croatian Medical Journal. 2008 Dec 15 ; 49 ( 6 ): 720 ± 33 . https://doi.org/10.3325/cmj. 2008 . 49 .720 PMID: 19090596 23. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21st century . Journal of Global Health. 2016 Jun; 6 ( 1 ). 24. Ager A , Stark L , Akesson B , Boothby N. Defining best practice in care and protection of children in crisis-affected settings: A Delphi study . Child Development . 2010 Jul 1 ; 81 ( 4 ): 1271 ± 86 . https://doi.org/10. 1111/j.1467- 8624 . 2010 . 01467 . x PMID : 20636695 25. Bennouna C , Mansourian H , Stark L . Ethical considerations for children's participation in data collection activities during humanitarian emergencies: A Delphi review . Conflict and Health . 2017 Dec; 11 ( 1 ): 5 . 26. Toole MJ , Waldman RJ . The public health aspects of complex emergencies and refugee situations . Annual Review of Public Health . 1997 May; 18 ( 1 ): 283 ± 312 . 27. UNICEF. What is Child Protection? Child Protection Information Sheet . 2016 [Accessed 28 January 2018 ]. Available from: https://www.unicef.org/publications/files/Child_Protection_Information_Sheets. pdf. 28. UNICEF., United Nations Centre for Human Rights. Convention on the Rights of the Child: Information Kit. United Nations Centre for Human Rights; 1991 . 29. Yoshida S , Rudan I , Cousens S. Setting health research priorities using the CHNRI method: VI. Quantitative properties of human collective opinion . Journal of Global Health. 2016 Jun; 6 ( 1 ). 30. Vision World. Evaluation of Child Friendly Spaces: An Interagency Series of Impact Evaluations in Humanitarian Emergencies . Available from: Toole MJ, Waldman RJ . The public health aspects of complex emergencies and refugee situations . Annual Review of Public Health . 1997 May; 18 ( 1 ): 283 ± 312 . 31. Bennett C. ( 2015 ). The development agency of the future . Fit for protracted crises . 32. Roberts L , Hofmann CA . Assessing the impact of humanitarian assistance in the health sector . Emerging Themes in Epidemiology. 2004 Dec; 1 ( 1 ):3. https://doi.org/10.1186/ 1742 -7622-1-3 PMID: 15679909 33. Falb KL , Tanner S , Ward L , Erksine D , Noble E , Assazenew A , et al. Creating opportunities through mentorship, parental involvement, and safe spaces (COMPASS) program: multi-country study protocol to protect girls from violence in humanitarian settings . BMC Public Health. 2016 Dec 1 ; 16 ( 1 ): 231 . 34. Rubenstein BL , Spencer C , Mansourian H , Noble E , Munganga GB , Stark L . Community-based surveillance to monitor trends in unaccompanied and separated children in eastern DRC . Child Abuse & Neglect. 2015 Dec 1 ; 50 : 76 ± 84 . 35. Stark L , Rubenstein BL , Mansourian H , Spencer C , Noble E , Chisolm-Straker M . Estimating child separation in emergencies: results from North Kivu . Child Abuse & Neglect. 2016 Feb 1 ; 52 : 38 ± 46 . 36. Rudan I , Yoshida S , Chan KY , Sridhar D , Wazny K , Nair H , et al. Setting health research priorities using the CHNRI method: VII. A review of the first 50 applications of the CHNRI method . Journal of Global Health. 2017 Jun; 7 ( 1 ). 37. Behrman JR , Parker SW , Todd PE . Schooling impacts of conditional cash transfers on young children: Evidence from Mexico . Economic Development and Cultural Change . 2009 Apr; 57 ( 3 ): 439 ± 77 . PMID: 20209076 38. Rivera JA , Sotres-Alvarez D , Habicht JP , Shamah T , Villalpando S. Impact of the Mexican program for education, health, and nutrition (Progresa) on rates of growth and anemia in infants and young children: a randomized effectiveness study . Jama. 2004 Jun 2 ; 291 ( 21 ): 2563 ± 70 . https://doi.org/10.1001/jama. 291.21.2563 PMID: 15173147 39. Robertson L , Mushati P , Eaton JW , Dumba L , Mavise G , Makoni J , et al. Effects of unconditional and conditional cash transfers on child health and development in Zimbabwe: a cluster-randomised trial . The Lancet. 2013 Apr 13 ; 381 ( 9874 ): 1283 ± 92 . 40. Craig P , Dieppe P , Macintyre S , Michie S , Nazareth I , Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance . International Journal of Nursing Studies. 2013 May 1 ; 50 ( 5 ): 587 ± 92 . https://doi.org/10.1016/j.ijnurstu. 2012 . 09 .010 PMID: 23159157 41. Datta J , Petticrew M. Challenges to evaluating complex interventions: a content analysis of published papers . BMC Public Health . 2013 Dec; 13 ( 1 ): 568 . 42. Rehfuess EA , Bartram J . Beyond direct impact: Evidence synthesis towards a better understanding of effectiveness of environmental health interventions . International Journal of Hygiene and Environmental Health. 2014 Mar 1 ; 217 ( 2 ±3): 155 ±9. https://doi.org/10.1016/j.ijheh. 2013 . 07 .011 PMID: 23973507 43. Grantham-McGregor SM , Powell CA , Walker SP , Himes JH . Nutritional supplementation, psychosocial stimulation, and mental development of stunted children: the Jamaican Study . The Lancet. 1991 Jul 6 ; 338 ( 8758 ): 1 ± 5 . 44. Bandara A , Dehejia R , Lavie-Rouse S. The impact of income and non-income shocks on child labor: evidence from a panel survey of tanzania . World Development. 2015 Mar 1 ; 67 : 218 ± 37 . 45. de Carvalho Filho IE. Household income as a determinant of child labor and school enrollment in Brazil: Evidence from a social security reform . Economic Development and Cultural Change . 2012 Jan 1 ; 60 ( 2 ): 399 ± 435 . 46. De Hoop J , Rosati FC . Cash transfers and child labor . The World Bank Research Observer. 2014 Apr 1 ; 29 ( 2 ): 202 ± 34 . 47. Al-Qdah T , Lacroix M. Iraqi refugees in Jordan: Lessons for practice with refugees globally . International Social Work . 2011 Jul; 54 ( 4 ): 521 ± 34 . 48. Fakih A , Ibrahim M. The impact of Syrian refugees on the labor market in neighboring countries: empirical evidence from Jordan . Defence and Peace Economics . 2016 Jan 2 ; 27 ( 1 ): 64 ± 86 . 49. Zetter R , Deikun G. Meeting humanitarian challenges in urban areas . Forced Migration Review . 2010 Feb 1 ( 34 ): 5 . 50. UNHCR. Syria emergency. 2017 [Accessed 28 January 2018 ]. Available from: http://www.unhcr.org/ en-us/syria-emergency.html 51. Parker E , Maynard V . Humanitarian response to urban crises . 2015 [Accessed 15 May 2018 ]. Available from: Parker E, Maynard V . Humanitarian response to urban crises. 52. Sanderson D , Kayden JS , Leis J , editors. Urban disaster resilience: New dimensions from international practice in the built environment . Routledge; 2016 Apr 14 . 53. Ager A , Metzler J , Vojta M , Savage K. Child friendly spaces: a systematic review of the current evidence base on outcomes and impact . Intervention. 2013 Jul 1 ; 11 ( 2 ): 133 ± 47 . 54. Noble E , Ward L , French S , Falb K. State of the evidence: A systematic review of approaches to reduce gender-based violence and support the empowerment of adolescent girls in humanitarian settings . Trauma, Violence, & Abuse . 2017 Jan 1 : 1524838017699601 . 55. Rubenstein BL , Stark L . The impact of humanitarian emergencies on the prevalence of violence against children: an evidence-based ecological framework . Psychology, Health & Medicine. 2017 Mar 6 ; 22 ( sup1 ): 58 ± 66 . 56. Heiko A . Consensus measurement in Delphi studies: review and implications for future quality assurance . Technological Forecasting and Social Change . 2012 Oct 1 ; 79 ( 8 ): 1525 ± 36 . 58. Jorm AF . Using the Delphi expert consensus method in mental health research . Australian & New Zealand Journal of Psychiatry . 2015 Oct; 49 ( 10 ): 887 ± 97 .


This is a preview of a remote PDF: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0202570&type=printable

Laura Gauer Bermudez, Katharine Williamson, Lindsay Stark. Setting global research priorities for child protection in humanitarian action: Results from an adapted CHNRI exercise, PLOS ONE, 2018, DOI: 10.1371/journal.pone.0202570