Causes behind error rates for predictive biomarker testing: the utility of sending post-EQA surveys

Virchows Archiv https://doi.org/10.1007/s00428-020-02966-7 ORIGINAL ARTICLE Causes behind error rates for predictive biomarker testing: the utility of sending post-EQA surveys Cleo Keppens 1 & Ed Schuuring 2 & Elisabeth M. C. Dequeker 1 Received: 19 March 2020 / Revised: 29 October 2020 / Accepted: 1 November 2020 # The Author(s) 2020 Abstract External quality assessment (EQA) schemes assess the performance of predictive biomarker testing in lung and colorectal cancer and have previously demonstrated variable error rates. No information is currently available on the underlying causes of incorrect EQA results in the laboratories. Participants in EQA schemes by the European Society of Pathology between 2014 and 2018 for lung and colorectal cancer were contacted to complete a survey if they had at least one analysis error or test failure in the provided cases. Of the 791 surveys that were sent, 325 were completed including data from 185 unique laboratories on 514 incorrectly analyzed or failed cases. For the digital cases and immunohistochemistry, the majority of errors were interpretation-related. For fluorescence in situ hybridization, problems with the EQA materials were reported frequently. For variant analysis, the causes were mainly methodological for lung cancer but variable for colorectal cancer. Post-analytical (clerical and interpretation) errors were more likely detected after release of the EQA results compared to pre-analytical and analytical issues. Accredited laboratories encountered fewer reagent problems and more often responded to the survey. A recent change in test methodology resulted in method-related problems. Testing more samples annually introduced personnel errors and lead to a lower performance in future schemes. Participation to quality improvement projects is important to reduce deviating test results in laboratories, as the different error causes differently affect the test performance. EQA providers could benefit from requesting root cause analyses behind errors to offer even more tailored feedback, subschemes, and cases. Keywords External quality assessment . Molecular pathology . Root cause analysis . Quality management . Biomarkers . ISO 15189 . Colorectal cancer . Non-small-cell lung cancer This article is part of the Topical Collection on Quality in Pathology Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s00428-02002966-7. * Elisabeth M. C. Dequeker Cleo Keppens Ed Schuuring 1 Department of Public Health and Primary Care, Biomedical Quality Assurance Research Unit, University of Leuven, Kapucijnenvoer 35 block d, 1st floor, box 7001, 3000 Leuven, Belgium 2 Department of Pathology, University of Groningen, University Medical Center Groningen (UMCG), (HPC EA10), Hanzeplein 1, PO Box 30001, 9700 RB Groningen, The Netherlands Abbreviations ALK ALK receptor tyrosine kinase BRAF B-Raf proto-oncogene CAPA Corrective/preventive action CI Confidence interval EGFR Epidermal growth factor receptor EQA External quality assessment ESP European Society of Pathology FFPE Formalin-fixed paraffin embedded FISH Fluorescence in situ hybridization GEE Generalized estimating equations IHC Immunohistochemistry ISO International Organization for Standardisation KRAS KRAS proto-oncogene mCRC Metastatic colorectal carcinoma NGS Next-generation sequencing NRAS NRAS proto-oncogene NSCLC Non-small-cell lung cancer Virchows Arch OR PD-L1 ROS1 TPS TTP WT Odds ratio Programmed death ligand 1 ROS proto-oncogene 1 Tumor proportion score Total test process Wild-type Introduction The analysis of tumor-specific biomarkers provides information for appropriate targeted treatment decision-making in non-small-cell lung cancer (NSCLC) and metastatic colorectal cancer (mCRC) [1–3]. Predictive biomarker test results should therefore be accurate, reproducible and timely. Several external quality assessment (EQA) schemes, organized on a national or international level, assessed the performance for common biomarkers in NSCLC and mCRC. They revealed varying error rates depending on the evaluated markers and variants, sample types, or scheme rounds [4–13]. Longitudinal analyses of the EQA schemes organized by the European Society of Pathology (ESP) revealed that participation to multiple EQA scheme rounds improved participants’ performances [12, 13]. Over time, error rates decreased for ALK and EGFR analysis but increased for ROS1. Also, error rates were higher for immunohistochemistry (IHC) compared to fluorescence in situ hybridization (FISH) on formalin-fixed paraffin embedded (FFPE) samples and especially compared to digital case interpretation [12]. Remarkably, lower error rates have been described for cell lines compared to resections, for higher variant allele frequencies [13], and for laboratories who are accredited, test more samples or perform research [14]. In mCRC, error rates increased significantly for mutation-positive samples and for methods that do not cover all required variants [11]. Medical laboratories are advised to participate in EQA schemes [1, 3] sometimes part of their quality framework conform the International Organization for Standardization (ISO) standard 15189:2012 [15] or national equivalents like CAP 15189 [16]. Laboratories should have a documented procedure to identify and manage non-conformities when pre-determined performance criteria are not met, both for EQA as in routine practice. The providers of these EQA programs are preferably accredited according to ISO 17043:2010 [17], mimic patient samples as closely as possible, and check the entire examination process [15]. EQA providers could guide laboratories by the provision of feedback, reference material, or methodological advice [18, 19]. Some providers (such as the CAP and UK NEQAS) already request a root cause analysis from poor performers [7, 15], but no data has yet been published. Errors can be systematic (e.g., test method failure) while others can be accidental (e.g., clerical or pipetting errors). The time point of error occurrence in the total test process (TTP) has been reported in clinical chemistry and forensics [20, 21] and were mostly pre- (46–86%) and post-analytical (18–47%) of nature [20]. However, data is still lacking for molecular oncology. Recently, a step-by-step framework for effective EQA results management was proposed for laboratories and EQA providers [22, 23]. A subsequent evaluation of deviating EQA results in clinical chemistry according to this flowchart revealed that most errors (81%) were the laboratory’s responsibility (internal causes) and were mainly clerical errors (i.e., correct outcome entered incorrectly in the results form) (72%) [22]. This study evaluated the feasibility of requesting root causes of deviating EQA results in the ESP schemes for NSCLC and mCRC between 2014 and 2018. The error causes were compared for the different markers, techniques, and sample types (...truncated)