Accuracy and agreement of national spine register data for 474 patients compared to corresponding electronic patient records

European Spine Journal https://doi.org/10.1007/s00586 Accuracy and agreement of national spine register data for 474 patients compared to corresponding electronic patient records Ole Kristian Alhaug 0 1 2 3 4 6 7 Simran Kaur 0 1 2 3 4 6 7 Filip Dolatowski 0 1 2 3 4 6 7 Milada Cvancarova Småstuen 0 1 2 3 4 6 7 Tore K. Solberg 0 1 2 3 4 6 7 Greger Lønne 0 1 2 3 4 6 7 0 OsloMet University , Oslo , Norway 1 Division of Orthopaedic Surgery, Oslo University Hospital , Oslo , Norway 2 Department of Clinical Medicine, The Arctic University of Norway (UiT) , Tromsø , Norway 3 University Hospital of North Norway , Tromsø , Norway 4 Innlandet Hospital Trust , Brumunddal , Norway 5 Ole Kristian Alhaug 6 Norwegian University of Science and Technology , Trondheim , Norway 7 Akershus University Hospital , Nordbyhagen , Norway Purpose Data quality is essential for all types of research, including health registers. However, data quality is rarely reported. We aimed to assess the accuracy of data in a national spine register (NORspine) and its agreement with corresponding data in electronic patient records (EPR). Methods We compared data in NORspine registry against data in (EPR) for 474 patients operated for spinal stenosis in 2015 and 2016 at four public hospitals, using EPR as the gold standard. We assessed accuracy using the proportion correctly classified (PCC) and sensitivity. Agreement was quantified using Kappa statistics or interaclass correlation coefficient (ICC). Results The mean age (SD) was 66 (11) years, and 54% were females. Compared to EPR, surgeon-reported perioperative complications displayed weak agreement (kappa (95% CI) = 0.51 (0.33-0.69)), PCC of 96%, and a sensitivity (95% CI) of 40% (23-58%). ASA classification had a moderate agreement (kappa (95%CI)= 0.73 (0.66-0.80)). Comorbidities were underreported in NORspine. Perioperative details had strong to excellent agreements (kappa (95% CI) ranging from 0.76 ( 0.68-0.84) to 0.98 (0.95-1.00)), PCCs between 93% and 99% and sensitivities (95% CI) between 92% (0.84-1.00%) and 99% (0.98-1.00%). Patient-reported variables (height, weight, smoking) had excellent agreements (kappa (95% CI) between 0.93 (0.89-0.97) and 0.99 (0.98-0.99)). Conclusion Compared to electronic patient records, NORspine displayed weak agreement for perioperative complications, moderate agreement for ASA classification, strong agreement for perioperative details, and excellent agreement for height, weight, and smoking. NORspine underreported perioperative complications and comorbidities when compared to EPRs. Patient-recorded data were more accurate and should be preferred when available. Validation; Accuracy; Agreement; Registry; Lumbar spinal stenosis Introduction In clinical research, it is crucial to question how true and accurate data are; however, data validity and accuracy assessments are rarely published explicitly. National medical registries collect large-scale data during the dynamic workflow of daily clinical practice and have become essential sources of evidence-based medicine and health care policies. Register-based studies reflect everyday practice and have high external validity, and complement randomized control trials (RCTs) that assess smaller populations with lower external validity. Register data are collected and recorded by healthcare personnel, and not by dedicated research assistants. Therefore, it is essential to periodically assess the quality of register data reported by healthcare personnel and patients by validating it against other sources of data [ 1–3 ]. Because systematic errors can lead to bias, register validations may impact the robustness of medical and political conclusions based on register data. The literature on the validity of medical register data is sparse. Some studies are reporting good validity of medical and cancer registries [ 4–6 ]. However, a recent validation study of a German spine registry (DWG) showed high inaccuracy [ 7 ] and the authors recommended against using these register data. Our study aimed to assess the accuracy and agreement of NORspine data by comparing it to electronic patient records (EPR). Such information can aid in identifying pitfalls and conceptual problems related to data collection, not only relevant for other spine registers but also others, routinely recording clinical data. Patients and methods In this cross-sectional study, we reviewed electronic patient records (EPRs) of patients operated for lumbar spinal stenosis (LSS) who consented and responded to NORspine between January 1, 2015, and December 31, 2016. The authors were authorized to access data from four public hospitals within one health region (South-Eastern Norway Regional Health Authority) in Norway. To assess the representativity of our sample, we compared the study population to those treated at the remaining hospitals. In Norway, all 39 hospitals (coverage = 100%) that offer surgery for degenerative spinal disorders are obliged to report data to NORspine. Seventy percent of all patients that undergo elective spine surgery in Norway are included in NORspine, and the proportion that responds one year after surgery is seventy-four percent [ 8 ]. A  NORspine  data set consists of a preoperative form completed by the patient at admission for surgery. This form covers items related to sociodemographic and lifestyle variables (e.g., smoking, height, and weight) and a standard battery of questionnaires assessing pain and disability (Table 5). Immediately after completing surgery, and optimally while still in the operating theater, the surgeon completes a standardized form and reports clinical and radiological diagnosis, relevant comorbidities, ASA classification—usually as graded by the anesthetist, and details about the surgery, e.g., previous surgery, surgical access, surgical methods, and level(s) operated. The surgeon also reports perioperative complications by a predefined list (Table 6). Patients report the clinical outcome at 3 and 12 months after surgery as assessed by standard Patient-Reported Outcome Measures (PROMs). Electronic patient records (EPRs) consist of non-structured text documents (free text) recorded by DIPS® software within predetermined headings. We reviewed the EPRs using a standard empty NORspine form, and the investigators (OKA and SK) had no access to the corresponding data previously recorded in the NORspine. The study group selected a set of NORspine variables that could be recaptured from EPRs. Furthermore, we reviewed EPR documents (e.g., admission and surgeon’s notes) at the same time point as the time of surgery recorded in NORspine. We did not assess variables that were not registered routinely or consistently in EPRs, such as PROMs, symptom duration, marital status, education level, mother tongue, and working capability. The clinical follow-up at the treating centers was not standardized, and it was performed at different time points at the hospitals without (...truncated)