A proposal of a new evaluation framework towards implementation of genetic tests
RESEARCH ARTICLE
A proposal of a new evaluation framework
towards implementation of genetic tests
Erica Pitini ID1*, Elvira D’Andrea1,2, Corrado De Vito1, Annalisa Rosso1, Brigid Unim ID1,
Carolina Marzuillo1, Antonio Federici3, Emilio Di Maria4, Paolo Villari1
1 Department of Public Health and Infectious Diseases, Sapienza University of Rome, Rome, Italy,
2 Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and
Women’s Hospital and Harvard Medical School, Boston, Massachusetts, United Sates of America, 3 Ministry
of Health, Rome, Italy, 4 Department of Health Sciences, University of Genova, Division of Medical Genetics,
Galliera Hospital, Genova, Italy
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
*
Abstract
Background
OPEN ACCESS
Citation: Pitini E, D’Andrea E, De Vito C, Rosso A,
Unim B, Marzuillo C, et al. (2019) A proposal of a
new evaluation framework towards implementation
of genetic tests. PLoS ONE 14(8): e0219755.
https://doi.org/10.1371/journal.pone.0219755
Editor: Yvonne Bombard, University of Toronto,
CANADA
Received: November 13, 2018
The existing frameworks for the evaluation of genetic and genomic applications clearly
address the technical and clinical value of a test, but are less concerned with the way
genetic services are delivered and organized. We therefore aimed to develop a comprehensive new framework that includes an assessment of service delivery.
Methods
A new framework was built on the evaluation dimensions identified through a systematic
review of the existing frameworks and a Delphi survey of Italian experts in public health
genomics.
Accepted: July 2, 2019
Published: August 5, 2019
Results
Copyright: © 2019 Pitini et al. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Our framework has four sections. The first two sections, respectively, guide the evidence
collection process for the genetic test (analytic validity; clinical validity; clinical utility; personal utility) and its delivery models (organizational aspects; economic evaluation; ethical,
legal and social implications; patient perspective). The third section guides the formulation
of the research priorities to be addressed in future research. Finally, the fourth section suggests three criteria to summarize the collected evidence (net benefit, cost-effectiveness,
feasibility).
Data Availability Statement: All relevant data are
within the manuscript and its Supporting
Information files.
Funding: EP, ED, CDV, AR, BU, CM, PV report
grants from Italian Ministry of Health, project
“Definizione e promozione di programmi per il
sostegno all’attuazione del Piano d’Intesa del 13/3/
13 recante Linee di indirizzo su La Genomica in
Sanità Pubblica”: http://www.ccm-network.it/
progetto.jsp?id=node/1914&idP=740; European
Union Horizon 2020 research and innovation
programme MSCA-RISE-2014 (Marie Skłodowska-
Conclusion
We have successfully developed an evaluation framework for the evaluation of genetic tests
that includes an assessment of service delivery. It also introduces some neglected evaluation dimensions such as personal utility and patient perspective.
PLOS ONE | https://doi.org/10.1371/journal.pone.0219755 August 5, 2019
1 / 15
Genetic tests: A new framework for evaluation
Curie Research and Innovation Staff Exchange)
under grant agreement N˚645740, project
“Personalized pREvention of Chronic Diseases
consortium PRECeDI”: http://www.precedi.eu/site/
index.php. EDM reports a grant from Italian
Ministry of Health, project “Definizione e
promozione di programmi per il sostegno
all’attuazione del Piano d’Intesa del 13/3/13 recante
Linee di indirizzo su La Genomica in Sanità
Pubblica”: http://www.ccm-network.it/progetto.
jsp?id=node/1914&idP=740. The funders had no
role in study design, data collection and analysis,
decision to publish, or preparation of the
manuscript.
Competing interests: The authors have declared
that no competing interests exist.
Introduction
The expanding knowledge of the human genome and the consequently growing availability of
genetic and genomic applications in recent decades are expected to revolutionize medicine [1].
The most significant promise lies in precision medicine, where medical decisions are tailored
to an individual’s characteristics, including the patient’s genetic profile [2]. However, there is
still a wide gap between reality and expectations, partly due to the absence of a completely satisfactory framework for distinguishing useful innovations from unwarranted interventions
[3].
The existing frameworks for the evaluation of genetic and genomic applications rely mainly
on the technical and clinical assessments described in the well-known ACCE (Analytic validity, Clinical validity, Clinical utility, Ethical, legal and social implications) model [3, 4]. Nevertheless, such frameworks lack a structured evaluation of the implementation issues and how
these relate to the context in which the test will be used; i.e. how genetic services should be (re)organized to implement and deliver the new test [3]. Now more than ever the precision medicine model is under the spotlight, and it appears clear that an evaluation limited to technical
performance and clinical outcomes is no longer sufficient for effective “research translation”[5,
6]. This is particularly true for publicly funded healthcare systems, where equity and resource
constraints are a major concern.
In Italy, the approval of genetic and genomics applications for funding by the National
Healthcare System is largely unregulated. For this reason, the National Plan for Public Health
Genomics has proposed, as a key objective, the development of a more comprehensive evaluation framework for genetic and genomic applications, which could support decision-makers
in coverage decisions [7]. Our work was conducted as part of a project financed by the Italian
Ministry of Health to implement this plan and aims to develop a comprehensive evaluation
framework that includes an assessment of service delivery.
Materials and methods
A new framework was developed by combining a systematic review and a Delphi survey. The
systematic review was performed to review the need for a new framework and to provide a
basis for its creation. It aimed to identify and compare the existing frameworks, with a particular focus on their evaluation dimensions (i.e. analytic and clinical validity, clinical utility, etc.).
The Delphi survey was designed to support the systematic review and aimed to identify additional frameworks and to further explore the importance of their dimensions.
Throughout the paper, a genetic/genomic test is defined as an analysis of human chromosomes, DNA, RNA, genes, and/or gene products (e.g. enzymes or other proteins) that is primarily used to detect heritable or (...truncated)
