Adverse reaction signal detection methodology in pharmacoepidemiology
European Journal of Epidemiology
Adverse reaction signal detection methodology in pharmacoepidemiology
Bruno H. Stricker 0 1
0 & Bruno H. Stricker
1 Department of Epidemiology, Erasmus Medical Center , P.O. Box 2040, 3000 CA Rotterdam , The Netherlands
In the nineteenth century, the toxicity of chloroform led to
its withdrawal from clinical use [
] and in the period
1920?1940, hepatic injury by cinchophen [
agranulocytosis by amidopyrine and related agents [
recognized. But from the point of view of detection of
important unknown adverse reactions, the thalidomide
disaster with its thousands of fatal and non-fatal cases of
congenital malformations was an absolute hallmark [
a direct consequence, it was made mandatory in the early
sixties of the preceding century to perform extensive
toxicological, preclinical, and clinical studies before
marketing of a drug in Western countries, and national
spontaneous monitoring systems were set up. These
systems in concert with the medical literature, proved to be the
most effective and efficient system for recognizing new
adverse reactions since then [
]. In the years thereafter,
several drugs were recognized as the cause of serious
disease, such as chronic active hepatitis by oxyphenisatin
], sclerosing peritonitis by practolol [
] and many more
since then. Such monitoring consists of manual review of
adverse reaction reports by medical professionals and is
relatively cheap and flexible but suffers from substantial
underreporting, potential false-positive reporting and
absence of reliable usage figures. Also, case-by-case
assessments may lead to a loss of overview when large
numbers of reports are involved and rests heavily on the
quality of the professional.
In 1974, in an attempt to improve adverse reaction
signal detection, Finney proposed to compare the
proportion of reports of a certain drug-event association with the
proportion of reports of that event to all other drugs in the
database and test for significance in a 2 9 2 table [
significantly higher proportion comprised a signal. A
further extension of this principle with the magnitude
expressed as a reporting odds ratio with 95% confidence
limits was first proposed in 1992 [
], and as a proportional
reporting ratio in 2001 [
]. Of these two effect measures,
the reporting odds ratio has certain advantages [
measures are now extensively used by the pharmaceutical
industry as one of the tools of signal detection, in line with
European guidelines [
]. But up till recently, the large
majority of pharmaceutical marketing authorization
holders only check their own database which is limited to those
drugs which are marketed by that particular company. Only
some of them also use the WHO Vigibase or the FDA
Adverse Event Reporting System and since 2018, the
Europeans Medicine Agency?s database Eudravigilance
can be used.
A new development in signal detection is to use not only
adverse reaction reports but complete medical records
healthcare databases for this goal. Elsewhere in this
journal, Hallas et al. [
] describe how a hypothesis-free
screening of large administrative databases can be used for
recognition of new drug-outcome associations. This is one
of the examples of how the strong increase in
computerization in the past decades and the consequent growth of
automated healthcare data can be employed to this end. In
current initiatives such as EU-ADR [
] and the
Observational health data science and informatics (https://ohdsi.
org/), networks of administrative databases were built to
identify drug safety issues by data mining, mainly through
a self-controlled design covering data from up to many
millions of people. The question whether we should be
happy with such a development is completely irrelevant. In
human history, any technical development than can be used
will be used. And data mining has proven very successful
in genetic research. Genome-wide association studies
(GWAs) by consortia of population-based cohort studies
such as CHARGE were very rewarding in finding new
associations between genetic variants and disease [
Especially in Western countries the combination of risk
aversion and legislation is an enormous enforcer to employ
such healthcare information for safety research and as long
as the privacy of patients is guaranteed, there little against
using it. But the consequences are that the number of
falsepositive signals that will have to be tested increases
enormously. This requires a rigorous process of signal
prioritisation and testing as the number of epidemiological
resources is not endless. Apart from the subject itself, there
are a number of important differences between data mining
in genetic epidemiology and in pharmacoepidemiology.
First, in genome-wide association studies Bonferroni
corrections are used. There are many good arguments against
using Bonferroni corrections whatsoever [
] but in GWAs
they are the only workable solutions as using a p value of
0.05 as a cut-off would be very impractical in view of the
abundance of associations when studying millions of single
nucleotide polymorphisms. In data mining with healthcare
databases the number of associations that can be tested is
smaller and Bonferroni corrections are less commonly
used, maybe also because of a fear of litigations for drug
marketing authorization holders for missing associations.
Second, GWAs in consortia often work with identical
platforms. Healthcare databases are, however, very
heterogeneous. Not only do they vary between countries
and healthcare systems but also over time changes in
insurance system and disease coding may complicate
consistent analyses. Moreover, hospital-based and general
practitioner?s healthcare information is structured in a
different way, and mapping towards one analysable dataset
is a cumbersome challenge which has to be repeated again
and again. Third, and maybe most important, genetic
GWAs are driven by scientific interest, rather than for
fulfilling legal obligations. In how far this leads to better
science remains to be seen. But one conclusion, we can
make already now. If we do not improve our ability to
distinguish true-positive from false-positive signals in an
efficient way, we might waste epidemiologic resources for
extensive signal-testing as a consequence of our
increasingly demanding society.
appropriate credit to the original author(s) and the source, provide a
link to the Creative Commons license, and indicate if changes were
1. Wade OL , Beeley L . The dawn of concern . In: Wade OL , Beeley L , editors. Adverse reactions to drugs , vol. 1 . 2nd ed. London: William Heinemann Medical Books Ltd; 1976 .
2. Worster-Draught C . Atophan poisoning . Br Med J. 1923 ; 1 : 148 .
3. Kracke RR , Parker FP . The etiology of granulocytopenia (agranulocytosis). With particular reference to the drugs containing the benzene ring . J Lab Clin Med . 1934 ; 19 : 799 .
4. Taussig HB . A study of the German outbreak of phocomelia . JAMA . 1962 ; 180 : 1106 .
5. Stricker BH , Psaty BM . Detection, verification, and quantification of adverse drug reactions . Br Med J. 2004 ; 329 : 44 - 7 .
6. Reynolds TB , Peters RL , Yamada S. Chronic active and lupoid hepatitis caused by a laxative, oxyphenisatin . N Eng J Med . 1971 ; 285 : 813 .
7. Brown P , Baddeley H , Read AE , et al. Sclerosing peritonitis, an unusual reaction to b-adrenergic blocking drug (Practolol) . Lancet . 1974 ; 2 : 1477 .
8. Finney DJ . Systemic signaling of adverse reactions to drugs . Methods Inf Med . 1974 ; 13 : 1 - 10 .
9. Stricker BH , Tijssen JG . Serum sickness-like reactions to cefaclor . J Clin Epidemiol . 1992 ; 45 : 1177 - 84 .
10. Evans SJW , Waller PC , Davis S . Use of proportional reporting ratios for signal generation from spontaneous adverse drug reaction reports . Pharmacoepidemiol Drug Saf . 2001 ; 10 : 483 - 6 .
11. Rothman KJ , Lanes S , Sacks ST . The reporting odds ratio and its advantages over the proportional reporting ratio . Pharmacoepidemiol Drug Saf . 2004 ; 13 : 519 - 23 .
12. EMA. Guideline on good pharmacovigilance practices, module IX-signal management; 2017 .
13. Hallas J , Wang SV , Gagne JJ , Schneeweiss S , Pratt N , Pottega?rd A. Hypothesis-free screening of large administrative databases for unsuspected drug outcome associations . Eur J Epidemiol . 2018 ; 1:1 (in press).
14. Coloma PM , Schuemie MJ , Trifiro` G, Gini R , Herings R , Hippisley-Cox J , et al. Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EUADR Project . Pharmacoepidemiol Drug Saf . 2011 ; 20 : 1 - 11 .
15. Psaty BM , Hofman A . Genome-wide association studies and large-scale collaborations in epidemiology . Eur J Epidemiol 2010 ; 25 : 525 - 9 .
16. Greenland S , Rothman KJ . Fundamentals in epidemiologic data analysis . Modern epidemiology . 2nd ed. Philadelphia: LippincottRaven; 1998 . p. 201 - 29 .