A Method to Combine Signals from Spontaneous Reporting Systems and Observational Healthcare Data to Detect Adverse Drug Reactions

Drug Saf (2015) 38:895–908 DOI 10.1007/s40264-015-0314-8 ORIGINAL RESEARCH ARTICLE A Method to Combine Signals from Spontaneous Reporting Systems and Observational Healthcare Data to Detect Adverse Drug Reactions Ying Li1 • Patrick B. Ryan1,2,3 • Ying Wei4 • Carol Friedman1 Published online: 8 July 2015 Ó The Author(s) 2015. This article is published with open access at Springerlink.com Abstract Introduction Observational healthcare data contain information useful for hastening detection of adverse drug reactions (ADRs) that may be missed by using data in spontaneous reporting systems (SRSs) alone. There are only several papers describing methods that integrate evidence from healthcare databases and SRSs. We propose a methodology that combines ADR signals from these two sources. Objectives The aim of this study was to investigate whether the proposed method would result in more accurate ADR detection than methods using SRSs or healthcare data alone. Research Design We applied the method to four clinically serious ADRs, and evaluated it using three experiments that involve combining an SRS with a single facility small-scale electronic health record (EHR), a larger scale network-based EHR, and a much larger scale healthcare claims database. The evaluation used a reference standard comprising 165 positive and 234 negative drug–ADR pairs. Measures Area under the receiver operator characteristics curve (AUC) was computed to measure performance. Results There was no improvement in the AUC when the SRS and small-scale HER were combined. The AUC of the combined SRS and large-scale EHR was 0.82 whereas it was 0.76 for each of the individual systems. Similarly, the AUC of the combined SRS and claims system was 0.82 whereas it was 0.76 and 0.78, respectively, for the individual systems. Conclusions The proposed method resulted in a significant improvement in the accuracy of ADR detection when the resources used for combining had sufficient amounts of data, demonstrating that the method could integrate evidence from multiple sources and serve as a tool in actual pharmacovigilance practice. Key Points Y. Wei and C. Friedman contributed equally. Electronic supplementary material The online version of this article (doi:10.1007/s40264-015-0314-8) contains supplementary material, which is available to authorized users. & Ying Li 1 Department of Biomedical Informatics, Columbia University Medical Center, 622 W. 168th Street, Presbyterian Building 20th Floor, New York, NY 10032, USA 2 Janssen Research and Development, 1125 Trenton Harbourton Rd, Titusville, NJ 08560, USA 3 4 Observational Health Data Sciences and Informatics (OHDSI), New York, NY 10032, USA Department of Biostatistics, Columbia University, New York, NY 10032, USA Observational healthcare data can complement spontaneous reporting systems in signal detection through quantitative integration of source-specific signal scores. Signal detection predictive accuracy from each source can be improved by combining signals across sources. 1 Introduction Adverse drug reactions (ADRs) are known to cause high morbidity and mortality and cost several billion dollars annually [1–3]. In addition to the ADRs detected during 896 pre-marketing clinical trials, unanticipated ADRs may occur after a drug has been approved, attributable to its use, which may be prolonged, on large, diverse populations [4]. Therefore, the post-marketing surveillance of drugs is essential for generating more complete drug safety profiles and for providing a decision-making tool to help governmental drug administration agencies take an action on the marketed drugs [5, 6]. Analysis of spontaneous reports of suspected ADRs has traditionally served as a valuable tool in the detection of previously unknown ADRs in post-marketing surveillance [7, 8]. Spontaneous reporting systems (SRSs) can be effective in revealing unusual or rare adverse events that occur with the initial use or short-term use of medications [9]. However, SRSs do not rapidly lead to ADR detection if the adverse event is relatively common but not necessarily drug-related in the general population, and SRSs are also known for limitations such as under-reporting and biased reporting influenced by media coverage or the length of time on the market [10–12]. Electronic healthcare data, such as electronic health records (EHRs) and administrative claims data, are starting to be used to complement SRSs [13–16]. Electronic healthcare data contain longitudinal patient information collected during routine clinical care, and have been used extensively in pharmacoepidemiology and pharmacoeconomics to study the natural history of disease and treatment utilization. Another opportunity for these data is to study the prevalence of a drug and an ADR, to explore the temporal relationship between exposure and outcome, and to reduce the reporting biases of SRSs. The appropriate use of healthcare data has the potential for earlier detection of drug safety signals before healthcare professionals report them to an SRS system [17]. With the ongoing development of the US Food and Drug Administration’s (FDA’s) Sentinel Initiative and similar systems around the world, near real-time active pharmacovigilance may soon be a reality [18]. Since the Sentinel system is based on administrative claims data captured as part of the reimbursement process surrounding routine clinical care, its value can be considered ‘complementary’ to the utility of SRSs. However, electronic healthcare data has its own limitations, which are different from the SRS limitations, since healthcare data usually mention the patient’s medications, symptoms, and diseases individually without mentioning explicit causal relationships, such as the indications for prescribing the medications. Therefore, statistical methods, together with the use of temporality, are needed to infer an estimate of the strength of associations, without the benefit of an explicit reported ADR relationship. For example, a statistical association between a medication and a condition may be a treatment if the condition precedes the medication event, an ADR only Y. Li et al. if the condition follows the medication event, or an indirect association stemming from another event (e.g., a confounder). Considerable systematic studies of the potential value of these databases in post-marketing pharmacovigilance have been undertaken by the Observational Medical Outcomes Partnership (OMOP, http://omop.org) [16, 19, 20] and the European Union project Exploring and Understanding Adverse Drug Reactions (EU-ADR, http:// euadr-project.org) [21]. Currently, research efforts are starting to focus on the use of multiple data sources, such as SRSs, electronic healthcare data, biomedical literature, and chemical information, to detect and validate novel ADRs. For example, Tatonetti et al. discovered a potentially new drug interaction, which can lead to unexpected increases in blood glucose (...truncated)