Stakeholder-driven, consensus development methods to design an ethical framework and guidelines for engaged research
Stakeholder-driven, consensus development methods to design an ethical framework and guidelines for engaged research
Giselle Corbie-Smith 0 1
Mysha Wynn 1
Alan Richmond 1
Stuart Rennie 1
Melissa Green 1
Stephanie M. Hoover 1
Sable Watson-Hopper 1
Kyle Simone Nisbeth 1
0 Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, North Carolina, United States of America, 2 Department of Social Medicine, University of North Carolina School of Medicine, Chapel Hill, North Carolina, United States of America, 3 Project Momentum, Inc., Rocky Mount, North Carolina, United States of America, 4 Community-Campus Partnerships for Health , Raleigh, North Carolina , United States of America
1 Editor: Bridget Young, University of Liverpool , UNITED KINGDOM
Increasingly, researchers seek to engage communities, patients, and stakeholders as partners in the process and products of health research. However, there is no existing stakeholder-driven ethical framework for such engaged scholarship. We employed an iterative, stakeholder-engaged method to develop a data-driven framework for the ethical review and conduct of engaged scholarship. We used consensus development conference methods and a modified Delphi survey to engage 240 community members, ethicists, and academic researchers. This multi-staged process produced a framework with 4 domains: vision of equitable and just research, relationship dynamics, community-informed risk/benefits assessment, and accountability. Within the framework, 4 cross-cutting considerations and 15 statements explicate the stakeholders' priorities for the ethical review and conduct of engaged scholarship. Though the findings are promising, the study is limited in that it focuses on stakeholder perspectives, but does not actually evaluate or apply the findings in the field. The stakeholder-engaged framework provides a platform for further articulation of ethical practices and policy for engaged scholarship.
Funding: This work was supported by the
Greenwall Foundation (PI Giselle Corbie-Smith).
The funder had no role in the study design, data
collection and analysis, decision to publish, or
preparation of the manuscript.
Competing interests: The authors have declared
that no competing interests exist.
In both developed and ascending nations, there is growing interest in engaging communities,
patients, and stakeholders as partners in the process and products of engaged scholarship [
Engaged scholarship is an umbrella term that encompasses a diversity of existing
nomenclature, including patient-centered outcomes research, community-based research, and
community-based participatory research [
]. In biomedical and public health research specifically,
engaged scholarship has been traced back to Kurt Lewin's demonstration projects in the 1930s
]. Present day, engaged scholarship emerges as a novel approach to translational and
dissemination research [
Engaged scholarship in public health and biomedical research is grounded in an ethos of
social justice and a common goal: to address the health concerns of populations living in
underserved, economically constrained, or minority communities [
]. We use the word
community broadly, to capture a non-homogenous group of people with diverse characteristics
who are linked by social ties, share common perspectives, illness, or health conditions, and
who engage in joint action to advance health [
]. To achieve health equity, engaged scholarship
assumes that knowledge is co-owned and co-created within communities. Engaged
scholarship flattens conventional power and knowledge hierarchies between researcher and
participant by sharing power and decision-making. In contrast to being relegated to the role of
ªresearch subjectsº or being rendered invisible in the academic's research agenda,
nonacademic partners offer essential expertise to solve complex problems in public health and health
With a shared commitment to social justice and action, engaged scholarship encourages
ethical research practices [
]. In terms of the ethical principal of beneficence, research
questions in engaged scholarship often are responsive to social, economic, and political contexts
that are associated with diminished power and autonomy of certain populations . In
research that uses engaged approaches, there is an expectation among academic and
nonacademic collaborators that research will be designed to address inequities experienced by
potential participants [
]. When determining potential participants, scholars refrain from practices
such as targeting groups unfairly, excluding certain groups from participating, or by assuming
that certain populations will be less likely to access research benefits [
The unique features of engaged scholarship give rise to an alternative set of tensions,
conflicts, and dilemmas. Ethical issues in engaged scholarship include, but are not limited to,
historical considerations of past scientific misconduct, socioeconomic inequities influencing
incentive structures, and balance of competing interests of group or individual harms or
benefits in communities that endorse a more collective ethos [
]. Turning to the existing literature,
we show the limitations in the current evidence for the ethical review and conduct of engaged
The Belmont Report [
] and the set of principles derived from it, has given researchers a tidy
way of examining and resolving ethical conflicts as they relate to human subjects research.
Analyses that engage the ethical principles of respect, beneficence, and justice, as well as the
frameworks that derive from them, were developed for traditional research contexts where
investigator-initiated research is reviewed by institutional review boards (IRBs) at academic
institutions, conducted in controlled settings, and where communities play a more passive role
]. A principle-based analysis of research ethics concerns prioritizes investigator expertise
and scientific validity as primary criteria in the design and review of research [
investigator expertise and scientific validity are important in all research, additional
considerations and expertise are needed to make ethical decisions in engaged scholarship [
Further, current practice and recent revisions to the Common Rule are intended to guide
the review of human subjects research. However, the Common Rule as written and in practice
may not cover the full range of ethical considerations that arise in engaged research
approaches nor adequately address the needs and interests of community investigators [11±13,
6]. To ensure a review of the ethical concerns central to engaged research, some authors have
suggested changes or additional layers to IRB review [
11, 14, 15
]. Many processes have been
proposed: stronger community representation on IRBs; community-level considerations in
policies, applications, and processes; increased understanding of engaged scholarship by the
2 / 12
IRB; greater transparency of IRB deliberations and comments and an established
communitylevel review, i.e. a blended review. Some recommendations have been implemented, for
example by the Bronx Community IRB [
In addition to the necessary revisions to federal regulations and IRB processes, the field is
so new that no comprehensive, data-informed, and stakeholder-driven examination of these
issues has been undertaken; thus no clear recommendations exist. Attempts to address ethics
have been primarily conceptual in nature. For example, Chen et al.'s framework is a specific
application of an existing ethical framework for clinical research [
]. HeÂbert and colleagues'
ethical assertions are an extension of the principles of community-based participatory research
]. Another gap in this literature is due to an overrepresentation, in that much of what has
been published comes from the perspective of investigators, IRB members, or ethicists. A key
exception is Ross et al.'s framework, which was developed with academic and community
partners, as well as human subjects protection personnel [
]. Their framework highlighted issues
with human subjects specifically, which is only one of many ethical considerations in engaged
As engaged scholarship promotes the engagement of patients, families, and community
members as co-constructors of knowledge, community and academic partners require
research ethics guidance that is appropriately matched to the complex ethical issues that arise
in community-academic partnerships. The engagement of nonacademic partners and
stakeholders is key to explicating research ethics for engaged scholarship. Our paper seeks to fill this
gap in the literature with an author team of academic-community partners who engaged
stakeholders in order to articulate an ethics framework.
Our purpose was (1) to engage a diverse group of community, ethics, and academic experts
with a range of experience in engaged scholarship, which would (2) yield a
stakeholderengaged, data-driven framework for the ethical review and conduct of engaged scholarship. In
this paper, we report on the iterative, multi-staged, stakeholder-engaged methods that we used
in order to draw on the experience and expertise of community members/advocates, academic,
bioethicists, and research ethics stakeholders. We provide the stakeholder-engaged framework
and ethical recommendations to be applied to the conduct and review of engaged scholarship.
We used well-described consensus development methods, which are frequently applied to
create clinical guidelines for a particular disease, clinical scenario, or discipline [
definition, consensus development methods are standardized approaches intended to organize the
available evidence and opinions of experts and systematically convert them into guidelines
that aim to improve practice and influence policy . These methods often rely on both the
opinions of experts and a literature review. The literature review is conducted to ensure all
participants have a similar level of understanding and to highlight available evidence (or lack
thereof) upon which the guidelines can be built. In our project, the experts were defined as
community, ethics, and academic professionals with a range of experience in health-related
engaged scholarship. The crux of expert engagement was via two consensus development
procedures: consensus development conference methods and modified Delphi technique [19±21].
Consensus development conference methods require in-person interaction to facilitate
dialogue, debate, and discussion of priority issues [
]. The modified Delphi ranking method
employs rating and ranking of the priority of ethical statements using online surveys.
Consistent with consensus development procedures, we used sequential, iterative stages
(see Table 1). In Stage 1, we prepared for stakeholder engagement by gathering expert opinion
3 / 12
1. Team member retrieved 804 1. Team member reviewed and re- 12 recommendations on
unique articles from PubMed and reviewed articles to develop list of the ethical responsibilities
Scopus. Search terms: (community- themes represented in any of the 44 for academic and
based participatory research, articles. community partners
participatory research, community 2. Entire team developed initial
based research, consumer driven and draft of statements, reviewed
research, OR community engaged statements, and revised statements.
research) AND (ethics OR morals).
Required English language.
2. Team member completed initial
screening using Covidence [
Exclusion criteria: Animal or
biological studies, safety of field
research, individual or
emotionbased research. 652 articles were
removed after title and abstract
screening. 108 articles were removed
after full text review, leaving 44
4 case studies, 2 provided
by community experts and
2 provided by academic
1. Team recruited from 500 1. Author team requested 5±6 1. Team selected subset of 4 case
health-related engaged PowerPoint slides on the academic- studies to represent 2 community
scholarship experts, which community partnership, ethical experts and 2 academic experts.
included authors identified in challenges, relevance to engaged Presented cases were limited 4 to
the narrative literature review, research, problem identification, and limit time on case review and
registered attendees at a lessons learned increase time for consensus
national conference on 2. Of 8 received, 5 were from development procedures at the
engaged scholarship, academic experts and 3 were from workshop.
Community-Campus community experts. Case studies
Partnerships for Health addressed: community regulatory
members, community review process, trust,
engagement and ethics cores miscommunication, informed
of the Clinical and consent, incentives, and duty to
Translational Science Award report suspected child abuse.
2. Of 500 individuals contact,
22 expressed interest and then
8 ultimately submitted case
Stage 2: Consensus Development with Stakeholders
PLOS ONE | https://doi.org/10.1371/journal.pone.0199451
4 / 12
recommendations and 4
Stage 3: Broader Stakeholder Engagement and Final Analysis
and evidence by (1) conducting a narrative literature review on engaged scholarship ethics and
(2) soliciting case studies from experts [
]. In Stage 2, we engaged stakeholders via consensus
development workshop and key informant interviews. The consensus development workshop
was the primary data source, and informant interviews were conducted to achieve theoretical
], ensuring that the themes from the workshop were complete and representative
of three distinct stakeholder perspectives: IRB, academic institutions, and community
organizations. We synthesized the data from the workshop and interviews yielding 15 statements
and 4 domains. In Stage 3, we sought a broader range of stakeholders by disseminating n
5 / 12
modified Delphi survey online, which allowed us to finalize the framework. The survey was
modified in the sense that we only completed a single data collection timepoint, whereas
traditional Delphi methods call for multiple rounds of data collection to result in convergence.
Because the results found an overwhelming endorsement of the 15 statements, convergence
was achieved, and subsequent Delphi survey timepoints were not needed. All procedures were
consistent with University of North Carolina IRB approval, which determined exempt status
and waived informed consent procedures.
The rigor of the study was ensured by multiple aspects of the design. First, multiple data
sources were triangulated across the stages of data collection. In addition to data collection, the
author team, which included both community and university affiliated researchers, sought
consensus when analyzing data and interpreting findings. Data collection and analysis were
both iterative processes, ensuring the trustworthiness of the final analysis.
Results and discussion
Engaged scholarship ethics framework
The stakeholder-engaged, iterative analysis yielded an ethics framework for the review and
conduct of engaged scholarship. The framework is anchored by four domains: vision of
equitable and just research, relationship dynamics, community-informed risks/benefits assessment,
and accountability. The four domains are further articulated by four cross-cutting
considerations: capacity building, translation for improved health, individual vs. group concerns:
collective ethos, and power/hierarchy (see Fig 1). Lastly, the 15 responsibility statements are
nested within the four domains (see Table 2).
Vision of equitable and just research. Vision of equitable and just research is the central
domain, in that stakeholders heavily emphasized its overarching importance in the ethical
review and conduct of research. Our findings raised the question of what is considered just in
the context of engaged scholarship. From our findings, there is no single shared definition of
justice. Instead, to ensure a shared set of values and expectations, community and academic
partners should define justice and its implications for the research design, conduct, and
communication. The vision of equitable and just research statements (#1±4) provide guidance in
determining what justice means for a particular partnership and project. For example, in a
given project, partners design, conduct, and communication of research need to be responsive
to specific historic and/or ongoing points of injustice experienced by communities.
We found that stakeholders frame questions of justice at the community level. The Belmont
] framed the topic by asking, ªWho ought to receive the benefits of research and bear
its burdens?º Based on our findings, the stakeholder-driven question becomes, ªHow, when
designing, conducting, and communicating health research, do we promote justice for
communities?º This is especially important when considering that some communities may be
considered vulnerable in the research process and potentially have the most to gain from advances
in research for social, economic, and/or historical reasons. Therefore, engaged scholarship
ethics should emphasize research that focuses both on what communities care about and also
finds linkages between community and research priorities.
With an emphasis on community throughout engaged scholarship partnerships, partners
need to make a clear and explicit effort to ensure that communities are left better off than
when the research began. In our findings, improving the community is not limited to the
health outcomes of a given study to be reported in a publication. Rather, improvement to the
community is a broad call to achieve equity. Community improvement takes on many forms,
including equitable division of resources between academic and nonacademic entities to
complete the proposed research, as well as capacity building within communities and stakeholder
6 / 12
Fig 1. Engaged scholarship ethics framework. Note. Vision of just and equitable research is the overarching domain with three
additional domains: relationship dynamics, accountability, and community-informed risks/benefits assessment. The outer circle
represents the four cross-cutting considerations: capacity building, translation for improved health, power/hierarchy, and individual vs.
group concerns: collective ethos.
organizations to carry out and manage aspects of the research and participate effectively as
Relationship dynamics. The goal of this domain is to address power hierarchies inherent
in the social structures that lead to health inequalities. A key power hierarchy manifests in the
historic relationships between academic institutions and communities that can shape the
research process. Relationship dynamics statements (#5±7) provide recommendations to meet
this goal. Our findings underline the fundamental importance of building and sustaining
relationships across time, including before project initiation. Building and sustaining relationships
in community-engaged research requires that stakeholders are a part of all phases of the
research continuumÐidentifying research questions, planning, implementing, and disseminating
7 / 12
Community member: ªI see a community as . . . having the 1. Researchers and communities strive for active partnerships that
resources, and the people, and the energy, and the imagination, and honor shared power and resources, co-learning and mutual respect.
the curiosity, and the strength to solve their problem . . . They're 2. Community engaged research is responsive to the structural
looking for allies to help them through whatever process they have conditions responsible for poor health and deprivation, and
to go through in order to help level the playing field, and research is contributes to the improvement of fundamental participant and
one of those [processes]. . . . [E]ngaged research makes it so much community welfare.
better for everybody, for the community to have the need and the
dream to better themselves and for researchers and institutions that 3. Community engagement should be guided by a broad conception
have so much in terms of knowledge and access to understand of justice.
what's going on in the community and to work as allies with
4. Community and academic researchers in partnership, determine
whether and how proposed research is important, relevant, and
Community member: ªI think when you're going to engage in a 5. Those parties involved in community engaged research (CEnR)
community, you have to look at what their core values are. It's not should engage the community of interest in the planning,
the same; every community is different. When you say community, implementation and dissemination of research.
what are you talking about? . . .Are you talking about Whites? 6. Researchers and communities should have transparent
Asians? Vietnamese? Thai? Hispanic? . . .. You're not going to learn communication with one another to foster trustworthiness.
that by Googling. You need to get in there. You need to get there
and engage, actually meet people and connect with people who are
probably a part of that community . . . Start talking. Just start
talking to them! That's prior to you developing your structure
because you need to have that relationship and that understanding
of who they are and what their core values are.º
7. Research should be initiated after first gaining familiarity with the
setting in which the research will be conducted.
IRB member: ªIf we're talking about sharing information about or
disclosing the risk and benefits through the informed consent
process, that's one thing, . . .but then how can we expand that? I
think by virtue of including the community earlier on in the 9. Identification of potential participants should be informed by
research process, getting their input to that process, so that the community and academic researcher expertise to ensure fair
research questions, research methodology, the whole approach selection and scientific validity.
incorporates community input. I would think that could only
enhance it, could only improve trust because you're no longer so 10. The process of obtaining consent should be informed by
much on the outside parachuting in and taking something from the community and academic researcher expertise to take into account
cultural, historical, and social context.
8. In engaged research, attention must be paid not only to risks,
benefits, and autonomy of individual research participants, but risks,
benefits, and autonomy as they relate to communities.
community and leaving; you're actually involving up front and
getting input and I think that can only help enhance the trust that
you would hope is there already . . .
11. Communities should provide input as to what constitutes
acceptable risks and benefits.
Academic: ªResearchers have not traditionally gotten results back 12. Researchers and communities are accountable for their presence
to the community of which they were obtained they. . .they went off and impact.
and published in professional journals and that was their target for 13. Findings and data should be accessible to every stakeholder in
the results of research and now more and more there's both an order to increase dissemination of results and support sustainability.
expectation, and certainly in community engaged research . . . If the
community gives you this, they get something back for it. And so 14. Community and academic researchers should aim for either the
how to communicate results to people what does that mean, what sustainability, responsible closure, or transition of projects.
does that look like, how long does that go on for.º
15. Community and academic researchers should commit to building
and maintaining relationships over time.
findings. The intended result is an increased level of transparency built on openness and
honesty, where relationships are characterized by mutual trust, respect, benefit, and shared power
Many successful and long-standing research collaborations have explicitly built partnership
capacity by developing shared language, systems, structures, and capacities that identify
historical and contemporary oppressions that influence partnerships and the process of research
]. This shared understanding in turn allows research teams to explicitly address the goal of
action for health equity. Partnership capacity building efforts often include understanding
how racism, elitism, and other forms of discrimination are experienced in the partnership and
may be a microcosm of larger societal forms of oppression.
Community-informed risks and benefits. The conduct of research in a historically
disadvantaged community may involve risks and benefits that are difficult for research funders,
8 / 12
investigators, or ethics committees to anticipate. The community-informed risks and benefits
statements (#8±11) further clarify that community engagement is necessary across many areas
of ethical review, including fair selection of potential participants, informed consent
procedures, autonomy of individuals, and risks/benefits to individuals and community. Throughout
stakeholder engagement, we found that there was a dual emphasis on the community itself
and on individuals in the community. To draw on their unique perspective and knowledge,
stakeholders engaged in the research must be part of the assessment of risks and benefits.
Our emphasis on community engagement in risks/benefits assessment is consistent with
recent trends. In recognition of the limited scope of academic IRBs, there is growing interest
in community review boards that consider topics that represent perspectives of stakeholders
regarding risks and benefits of the proposed research [
]. In contrast to university's focus
on the production of new knowledge, groups historically underserved by medicine and research
(e.g. patients, stakeholders, community members) often take into account cultural, historical,
and social contexts of the community, how research and medicine have impacted the
community in previous studies, and broader expectations of benefits. Because these factors are not
typically considered in academic ethics review, communities have developed innovative models for
assessing community risk/benefit, including Community Review Processes [
Community Review Processes are community-based and lead processes for research ethics review.
Their role is to consider the extent to which proposed research benefits or harms the
community's norms and values and to serve as gatekeepers to minimize harm. The community-led
review is especially important in light of the power dynamics at play with university IRBs.
Holding IRB processes at a university reinforces the hierarchy of academic institutions having greater
decision-making power compared to community organizations, despite the latter being more
directly impacted by the implemented research. Regardless of the IRB settingÐcommunity or
university, research partners should have a shared understanding of the research ethics,
regulatory processes, and timelines associated with institutional approvals and reporting.
Accountability. All partners in the research process have the responsibility to ensure that
the process and products of engaged scholarship adhere to its underlying principles. This
includes the expectation that research will lead to actions to improve health, co-learning and
capacity building, community rights to self-determination, mutual respect, and shared power
]. Though several authors have made important contributions to providing a framework
to assess research related risks for communities and individuals [
4, 6, 26
], our findings extend
the current calls to accountability. Notably, our findings suggest that there is mutual
accountability for the potential societal benefits of research as part of the risk/benefit calculus.
Outlined in the accountability statements (#12±15), these benefits include building
researchrelated capacity, ensuring the availability and use of research findings and products by all
partners to improve community health, and sustaining the research partnership, the intervention,
Inherent in the conduct of engaged scholarship, we also realize research relationships may
benefit all stakeholders by creating new social ties, insights, and experiences. To ensure
effective working relationshipsÐboth now and in the futureÐall parties in engaged scholarship
should play an active role in developing, maintaining, and, when necessary, respectfully ending
research collaborations. The ending of research collaborations requires particular care, as our
findings suggest that all partners have a commitment to building and maintaining
relationships over time. Beyond the partnership, all research partners have responsibilities to
individuals and communities that are part of the research process and, as noted above, to each other. In
particular, fidelity to the principles of engaged approaches ensures a focus on action and
translation to improve health as both academic and community researchers become the face of the
9 / 12
research project in their communities and as research findings are disseminated in
communities and peer reviewed literature.
Cross-cutting considerations. As indicated, the following cross-cutting considerations
pervade all domains: (1) capacity building, (2) translation for improved health, (3) individual
vs. group concerns: collective ethos, and (4) power/hierarchy. Regarding capacity building,
stakeholders suggested that ethical conduct of engaged scholarship should take into
consideration the potential for capacity building. The capacities are not necessarily linked to a project's
specific research activities. Rather, capacity building is meant to go beyond any single study
and aim for community change.
Regarding translation for improved health, ethical conduct should take into consideration
the potential of the project to improve the community's health. In other words, engaged
scholarship should not aim to yield data for data's sake or to only support an academic institution's
objectives. Instead, research should be conceived, designed, and conducted to increase the
likelihood that the projects yield sustainable programs with the ultimate goal of improved
Regarding collective ethos, the ethical conduct of engaged scholarship considers the
potential to benefit the individual research participants and the community. Among stakeholders,
what research does or does not do for a community was a common concern. Paired with this
concern, stakeholders observed that an exclusive focus on the welfare of individual participants
has often been at the expense of addressing group-level concerns. Attention to the group
brings into focus the historical dimension of community, including past policies/actions that
are part of collective memory and connected to current health issues.
The final cross-cutting consideration is power/hiearchy. Stakeholders raised concerns that,
though issues of power are pervasive, so is the tendency to neglect or gloss over issues of
power. Across all domains, power dynamics shape engaged scholarship. For example,
academic researchers in community engaged research are typically in positions of social,
economic, or political power relative to the communities with whom they partner. As mentioned
above, IRBs being housed at universities, as opposed to held within community settings,
reinforces the relative power of academic institutions. Collaborative decision-making power is
jeopardized when community partners are not routinely involved in review and regulation of
research ethics. Stakeholders recommended that power dynamics need to be observed, named,
and discussed, in order to help minimize negative impacts.
Using a robust, multi-staged approach, this framework emphasized stakeholder engagement
to yield domains, cross-cutting considerations, and domain-specific recommendations.
Our study could have been enhanced by using other rigorous techniques, such as conducting
a systematic review as opposed to narrative review, soliciting additional case studies to
ensure broad representation, additional recruitment efforts to ensure higher response rate
for the Delphi survey, and utilizing stratified sampling for the Delphi survey. Nonetheless,
the iterative, multi-staged process corrected for biases and other limitations by not using
other techniques. We urge caution in unreflective application of the findings presented here.
Additional evidence would further clarify and support recommendations for the ethical
review and conduct of engaged scholarship. We suggest that the 15 statements could be
adapted into a checklist with a ranking system. Future research could pilot test the checklists
use in IRB processes or ongoing community-academic partnered research. In tandem, we
recommend all parties engage in ongoing scrutiny to refine their own ethical practices and
10 / 12
We present a framework for the ethical review and conduct of engaged scholarship. By
engaging stakeholders across various settings (academic researchers, community researchers, and
ethicists,), this framework extends the current discourse on the ethics of engaged scholarship.
Our goal is to offer a stakeholder-engaged, data-driven framework upon which policies and
practices can be built. This framework and recommendation could be made a part of
continuing education and schedule of procedures for IRB members, particularly those committees
that are charged with the review of engaged research. We offer these recommendations for
community advisory boards and patient representatives on research studies to frame ethical
concerns in discussions with other research stakeholders. In addition, we see opportunities to
integrate these recommendations in existing IRB standard operating procedures. Finally, we
see this framework and recommendations included in institutional offerings of the responsible
conduct of research to ensure that both researchers and research staff based at academic
institutions prioritize the ethical priorities of engaged scholarship. We hope the framework will be
useful to key players who may influence their institutional and organizational policies,
researchers co-leading the next generation of public health and biomedical research, and
community partners invested in co-creating a platform for equitable research.
Conceptualization: Giselle Corbie-Smith, Mysha Wynn, Stuart Rennie, Stephanie M. Hoover.
Data curation: Melissa Green, Kyle Simone Nisbeth.
Formal analysis: Giselle Corbie-Smith, Alan Richmond.
Funding acquisition: Giselle Corbie-Smith, Melissa Green.
Investigation: Giselle Corbie-Smith, Melissa Green, Sable Watson-Hopper, Kyle Simone
Methodology: Giselle Corbie-Smith, Melissa Green.
Project administration: Giselle Corbie-Smith, Melissa Green.
Resources: Giselle Corbie-Smith, Melissa Green.
Software: Giselle Corbie-Smith, Melissa Green.
Supervision: Giselle Corbie-Smith, Melissa Green.
Validation: Giselle Corbie-Smith, Mysha Wynn, Melissa Green, Stephanie M. Hoover.
Visualization: Giselle Corbie-Smith, Stephanie M. Hoover.
Writing ± original draft: Giselle Corbie-Smith, Stuart Rennie, Melissa Green, Stephanie M.
Hoover, Kyle Simone Nisbeth.
Writing ± review & editing: Mysha Wynn, Alan Richmond, Stuart Rennie, Stephanie M.
Hoover, Sable Watson-Hopper.
11 / 12
1. McAllister CL , Green BL , Terry MA , Herman V , Mulvey L . Parents, practitioners, and researchers: community-based participatory research with early head start . Am J Public Health Res . 2003 ; 93 ( 10 ): 1672 ± 9 .
2. Israel BA , Schulz AJ , Parker EA , Becker AB , Community-Campus Partnerships for Health. Communitybased participatory research: policy recommendations for promoting a partnership approach in health research . Educ Health . 2001 ; 14 ( 2 ): 182 ± 97 .
3. Minkler M , Wallerstein N , editors. Community-based participatory research for health: from process to outcomes . 2nd ed. San Francisco, CA: Jossey-Bass; 2010 .
4. Anderson EE , Solomon S , Heitman E , DuBois JM , Fisher CB , Kost RG , et al. Research ethics education for community-engaged research: a review and research agenda . J Empir Res Hum Res Ethics 2012 ; 7 ( 2 ):3± 19 . https://doi.org/10.1525/jer. 2012 . 7 . 2 .3 PMID: 22565579
5. MacQueen KM , McLellan E , Metzger DS , Kegeles S , Strauss RP , Scotti R , et al. What is community? an evidence-based definition for participatory public health . Am J Public Health Res . 2001 ; 91 ( 12 ): 1929 ± 38 .
6. Ross LF , Loup A , Nelson RM , Botkin JR , Kost R , Smith GR , et al. Human subjects protections in community-engaged research: a research ethics framework . J Empir Res Hum Res Ethics . 2010 ; 5 ( 1 ):5± 18 . https://doi.org/10.1525/jer. 2010 . 5 . 1 .5 PMID: 20235860
7. Wallerstein N , Duran B . Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity . Am J Public Health Res . 2010 ; 100 ( S1 ) :S40±6 .
8. Israel BA , Schulz AJ , Parker EA , Becker AB . Review of community-based research: assessing partnership approaches to improve public health . Annu Rev Public Health . 1998 ; 19 ( 1 ): 173 ± 202 .
9. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Ryan KJP . The Belmont Report: ethical principles and guidelines for the protection of human subjects of research . US Government Printing Office; 1978 .
10. Emanuel EJ , Wendler D , Grady C . What makes clinical research ethical? JAMA 2000 ; 283 ( 20 ): 2701 ± 11 . PMID: 10819955
11. Chen DT , Jones L , Gelberg L . Ethics of clinical research within a community-academic partnered participatory framework . Ethn Dis . 2006 ; 16 ( 1 Suppl 1 ):S118± 35 .
12. Hudson KL , Collins FS . Bringing the common rule into the 21st century . N Engl J Med . 2015 ; 373 ( 24 ): 2293 ±6. https://doi.org/10.1056/NEJMp1512205 PMID: 26509903
13. Smith SA , Blumenthal DS . Community health workers support community-based participatory research ethics: lessons learned along the research-to-practice-to-community continuum . J Health Care Poor Underserved . 2012 ; 23 ( 4 Suppl) : 77 .
14. HeÂbert JR , Satariano WA , Friedman DB , Armstead CA , Greiner A , Felder TM , et al. Fulfilling ethical responsibility: moving beyond the minimal standards of protecting human subjects from research harm . Prog Community Health Partnersh . 2015 ; 9(S): 41 ± 50 .
15. Shore N , Wong KA , Seifer SD , Grignon J , Gamble VN . Introduction to special issue: advancing the ethics of community-based participatory research . J Empir Res Hum Res Ethics . 2008 ; 3(2):1±4 . https:// doi.org/10.1525/jer. 2008 . 3 . 2 .1 PMID: 19385741
16. Del Campo FM , Casado J , Spencer P , Strelnick H. The development of the Bronx Community Research Review Board: a pilot feasibility project for a model of community consultation . Prog Community Health Partnersh . 2013 ; 7 ( 3 ): 341 ± 52 . https://doi.org/10.1353/cpr. 2013 .0037 PMID: 24056516
17. Barnette JJ , Danielson LC , Algozzine RF . Delphi methodology: an empirical investigation . Contemporary Educational Research Quarterly . 1978 ; 3 ( 1 ): 67 ± 73 .
18. Jairath N , Weinstein J . The Delphi methodology (Part one): A useful administrative approach . Can J Nurs Adm . 1994 ; 7 ( 3 ): 29 ± 42 . PMID: 7880844
19. James D , Warren-Forward H . Research methods for formal consensus development . Nurse Res . 2015 ; 22 ( 3 ): 35 ± 40 . https://doi.org/10.7748/nr. 22.3.35.e1297 PMID: 25587865
20. Black N , Murphy M , Lamping D , McKee M , Sanderson C , Askham J , et al. Consensus development methods: a review of best practice in creating clinical guidelines . J Health Serv Res Policy . 1999 ; 4 ( 4 ): 236 ± 48 . https://doi.org/10.1177/135581969900400410 PMID: 10623041
21. Hsu C-C, Sandford BA . The Delphi technique: making sense of consensus . Practical Assessment , Research & Evaluation. 2007 ; 12 ( 10 ): 1 ± 8 .
22. Charmaz K. Constructing grounded theory: a practical guide through qualitative analysis . Thousand Oaks , CA, Sage; 2006 .
23. Veritas Health Innovation, Covidence systematic review software . Melbourne, Australia , 2017 . Available from: http://www.covidence.org
24. Quinn SC . Ethics in public health research: protecting human subjects: the role of community advisory boards . Am J Public Health Res . 2004 ; 94 ( 6 ): 918 ± 922 .
25. Shore N , Park A , Castro P , Wat E , Sablan-Santos L , Isaacs ML , Freeman E , Cooks JM , Drew E , Seifer SD . Redefining research ethics review: case studies of five community-led models . Seattle, WA: Community-Campus Partnerships for Health; 2014 . Available from: https://ccph.memberclicks.net/assets/ Documents/PapersReports/crp-chapter1. pdf .
26. Kass NE . An ethics framework for public health . Am J Public Health Res . 2001 ; 91 ( 11 ): 1776 ± 1782 .