Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking

Preston et al. Systematic Reviews (2015) 4:82 DOI 10.1186/s13643-015-0074-7 RESEARCH Open Access Improving search efficiency for systematic reviews of diagnostic test accuracy: an exploratory study to assess the viability of limiting to MEDLINE, EMBASE and reference checking Louise Preston1, Christopher Carroll1*, Paolo Gardois2, Suzy Paisley1 and Eva Kaltenthaler1 Abstract Background: Increasing numbers of systematic reviews evaluating the diagnostic test accuracy of technologies are being published. Currently, review teams tend to apply conventional systematic review standards to identify relevant studies for inclusion, for example sensitive searches of multiple bibliographic databases. There has been little evaluation of the efficiency of searching only one or two such databases for this type of review. The aim of this study was to assess the viability of an approach that restricted searches to MEDLINE, EMBASE and the reference lists of included studies. Methods: A convenience sample of nine Health Technology Assessment (HTA) systematic reviews of diagnostic test accuracy, with 302 included citations, was analysed to determine the number and proportion of included citations that were indexed in and retrieved from MEDLINE and EMBASE. An assessment was also made of the number and proportion of citations not retrieved from these databases but that could have been identified from the reference lists of included citations. Results: 287/302 (95 %) of the included citations in the nine reviews were indexed across MEDLINE and EMBASE. The reviews’ searches of MEDLINE and EMBASE accounted for 85 % of the included citations (256/302). Of the forty-six (15 %) included citations not retrieved by the published searches, 24 (8 %) could be found in the reference lists of included citations. Only 22/302 (7 %) of the included citations were not found by the proposed, more efficient approach. Conclusions: The proposed approach would have accounted for 280/302 (93 %) of included citations in this sample of nine systematic reviews. This exploratory study suggests that there might be a case for restricting searches for systematic reviews of diagnostic test accuracy studies to MEDLINE, EMBASE and the reference lists of included citations. The conduct of such reviews might be rendered more efficient by using this approach. Keywords: Systematic reviews, Diagnostic test accuracy, Literature searching, MEDLINE, EMBASE, Health Technology Assessment * Correspondence: 1 School of Health and Related Research (ScHARR), University of Sheffield, Sheffield S1 4DA, England Full list of author information is available at the end of the article © 2015 Preston et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http:// creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Preston et al. Systematic Reviews (2015) 4:82 Background Increasing numbers of systematic reviews evaluating diagnostic technologies are being published in the field of Health Technology Assessment (HTA). In response to the needs of policy-makers in this field, in the last years, the National Institute for Health and Care Excellence (NICE) has established a Diagnostics Assessment Programme and a Diagnostics Advisory Committee, having run a pilot project to develop methods in this area [1, 2]. Systematic reviews or individual studies of diagnostic test accuracy usually compare an index test with the best available test or current standard procedure for making a diagnosis. The methodological challenges of undertaking systematic reviews of diagnostic accuracy studies are well known and have been extensively discussed in the academic literature [3, 4]. Searching for and identifying evidence is one challenge when undertaking such a systematic review. Search filters, including validated filters, are available from various sources, but their use is now not recommended by some organisations because the results from applying these filters are variable [4, 5]. This is due in part to inconsistency in the reporting and indexing of papers. Consequently, diagnostic study filters compare less favourably with other search filters, e.g. for Randomised Controlled Trials [4]. The Cochrane Collaboration Diagnostic Test Accuracy Working Group is working on the publication of diagnostic test accuracy systematic reviews within the Cochrane Library and recognises the challenges of searching for diagnostic studies. The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy has a chapter on searching for studies, which recommends that “a range of databases be considered for searching”, including MEDLINE, EMBASE and regional databases (to account for the differing disease prevalence within different geographical regions) [6]. Within the information science community, there is a growing interest in search efficiency, in particular whether it is possible to identify the same sample of included studies for a systematic review by searching fewer databases than the traditionally large number deemed necessary [7, 8]. This perhaps inevitable move has been driven by several factors, including the improved indexing and searching capabilities of databases and the need to produce high-quality reviews within time and resource constraints [9]. Consequently, it has been argued that a well-structured search undertaken in only two or three databases (supported by additional methods to identify evidence, such as reference list checking, citation searching, contact with manufacturers and experts) might identify evidence more efficiently than a similar search undertaken in more databases [7]. Recent research evaluated whether searches for studies of diagnostic test accuracy for systematic review and meta-analysis could be limited to MEDLINE alone [10]. Page 2 of 10 Appraising 44 reviews of diagnostic test accuracy studies containing 76 meta-analyses, the authors found that in 65 of the 76 meta-analyses (85.5 %), all of the studies were identifiable in MEDLINE. Of the remaining 11 meta-analyses, 87.5–99 % of the studies were identifiable in MEDLINE. Therefore, the authors suggest that extensive searching in databases other than MEDLINE has minimal effect on the identification of studies for inclusion in diagnostic reviews. However, this conclusion assumes that the actual searches undertaken in MEDLINE for all 44 reviews would have had 100 % sensitivity: that is, they would have retrieved all of the relevant studies indexed in that database. In a separate study by the same authors, statistical tests were also undertaken on a subset of those meta-analyses for which not all included studies were indexed in MEDLINE. (...truncated)