Defining and improving data quality in medical registries: a literature review, case study, and generic framework

The incidence of surgical site infection following major lower limb amputation: A systematic review

Surgical site infections (SSIs) following major lower limb amputation (MLLA) in vascular patients are a major source of morbidity. The objective of this systematic review was to determine the incidence of SSI following MLLA in vascular patients. This review was prospectively registered with the International Prospective Register of Systematic Reviews (CRD42023460645). Databases were searched without date restriction using a pre-defined search strategy. The search identified 1427 articles. Four RCTs and 21 observational studies, reporting on 50 370 MLLAs, were included. Overall SSI incidence per MLLA incision was 7.2% (3628/50370). The incidence of SSI in patients undergoing through-knee amputation (12.9%) and below-knee amputation (7.5%) was higher than the incidence of SSI in patients undergoing above-knee amputation, (3.9%), p < 0.001. The incidence of SSI in studies focusing on patients with peripheral arterial disease (PAD), diabetes or including patients with both was 8.9%, 6.8% and 7.2%, respectively. SSI is a common complication following MLLA in vascular patients. There is a higher incidence of SSI associated with more distal amputation levels. The reported SSI incidence is similar between patients with underlying PAD and diabetes. Further studies are needed to understand the exact incidence of SSI in vascular patients and the factors which influence this.

Table 0; documenting the steps to go from clinical database to research dataset

OBJECTIVES

Data-driven decision support tools have been increasingly recognized to transform health care. However, such tools are often developed on predefined research datasets without adequate knowledge of the origin of this data and how it was selected. How a dataset is extracted from a clinical database can profoundly impact the validity, interpretability and interoperability of the dataset, and downstream analyses, yet is rarely reported. Therefore, we present a case study illustrating how a definitive patient list was extracted from a clinical source database and how this can be reported.

STUDY DESIGN AND SETTING

A single-center observational study was performed at an academic hospital in the Netherlands to illustrate the impact of selecting a definitive patient list for research from a clinical source database, and the importance of documenting this process. All admissions from the critical care database admitted between January 1, 2013, and January 1, 2023, were used.

RESULTS

An interdisciplinary team collaborated to identify and address potential sources of data insufficiency and uncertainty. We demonstrate a stepwise data preparation process, reducing the clinical source database of 54,218 admissions to a definitive patient list of 21,553 admissions. Transparent documentation of the data preparation process improves the quality of the definitive patient list before analysis of the corresponding patient data. This study generated seven important recommendations for preparing observational health-care data for research purposes.

CONCLUSION

Documenting data preparation is essential for understanding a research dataset originating from a clinical source database before analyzing health-care data. The findings contribute to establishing data standards and offer insights into the complexities of preparing health-care data for scientific investigation. Meticulous data preparation and documentation thereof will improve research validity and advance critical care.

The Fight Tumour Blindness Registry: Efficient capture of high-quality real-world data in uveal melanoma

OBJECTIVE

To describe the development of a web-based data collection tool to track the management and outcomes of uveal melanoma patients.

DESIGN

Description of a clinical registry.

PARTICIPANTS

Patients with uveal melanoma.

METHODS

A panel of expert ocular oncologists, with input from other relevant specialties and individuals with expertise in registry development, collaborated to formulate a minimum data set to be collected to track patient centred, real-world outcomes in uveal melanoma. This data set was used to create the Fight Tumour Blindness! (FTB!) registry within Save Sight Registries.

RESULTS

The data set to be collected includes patient demographics and medical history, baseline visit, follow-up visit including tumour treatment, metastatic staging and surveillance, pathology, and patient-reported questionnaires. The inbuilt mechanisms to ensure efficient and complete data collection are described.

CONCLUSIONS

The FTB! registry can be used to monitor outcomes for patients with uveal melanoma. It allows benchmarking of outcomes and comparisons between different clinics and countries.

Outlier analysis for accelerating clinical discovery: An augmented intelligence framework and a systematic review

Clinical discoveries largely depend on dedicated clinicians and scientists to identify and pursue unique and unusual clinical encounters with patients and communicate these through case reports and case series. This process has remained essentially unchanged throughout the history of modern medicine. However, these traditional methods are inefficient, especially considering the modern-day availability of health-related data and the sophistication of computer processing. Outlier analysis has been used in various fields to uncover unique observations, including fraud detection in finance and quality control in manufacturing. We propose that clinical discovery can be formulated as an outlier problem within an augmented intelligence framework to be implemented on any health-related data. Such an augmented intelligence approach would accelerate the identification and pursuit of clinical discoveries, advancing our medical knowledge and uncovering new therapies and management approaches. We define clinical discoveries as contextual outliers measured through an information-based approach and with a novelty-based root cause. Our augmented intelligence framework has five steps: define a patient population with a desired clinical outcome, build a predictive model, identify outliers through appropriate measures, investigate outliers through domain content experts, and generate scientific hypotheses. Recognizing that the field of obstetrics can particularly benefit from this approach, as it is traditionally neglected in commercial research, we conducted a systematic review to explore how outlier analysis is implemented in obstetric research. We identified two obstetrics-related studies that assessed outliers at an aggregate level for purposes outside of clinical discovery. Our findings indicate that using outlier analysis in clinical research in obstetrics and clinical research, in general, requires further development.

Data cleaning for clinician researchers: Application and explanation of a data-quality framework

BACKGROUND

Data cleaning is the series of procedures performed before a formal statistical analysis, with the aim of reducing the number of error values in a dataset and improving the overall quality of subsequent analyses. Several study-reporting guidelines recommend the inclusion of data-cleaning procedures; however, little practical guidance exists for how to conduct these procedures.

OBJECTIVES

This paper aimed to provide practical guidance for how to perform and report rigorous data-cleaning procedures.

METHODS

A previously proposed data-quality framework was identified and used to facilitate the description and explanation of data-cleaning procedures. The broader data-cleaning process was broken down into discrete tasks to create a data-cleaning checklist. Examples of the how the various tasks had been undertaken for a previous study using data from the Australia and New Zealand Intensive Care Society Adult Patient Database were also provided.

RESULTS

Data-cleaning tasks were described and grouped according to four data-quality domains described in the framework: data integrity, consistency, completeness, and accuracy. Tasks described include creation of a data dictionary, checking consistency of values across multiple variables, quantifying and managing missing data, and the identification and management of outlier values. The data-cleaning task checklist provides a practical summary of the various aspects of the data-cleaning process and will assist clinician researchers in performing this process in the future.

CONCLUSIONS

Data cleaning is an integral part of any statistical analysis and helps ensure that study results are valid and reproducible. Use of the data-cleaning task checklist will facilitate the conduct of rigorous data-cleaning processes, with the aim of improving the quality of future research.

Applications of Blockchain-Based Technology for Healthcare Devices Post-market Surveillance

The volume of data analysis for medical device post-market surveillance (PMS) has increased dramatically in recent years. It is the more stringent and intricate regulatory criteria of the health authorities that are meant to improve the medical device safety review. As regulators scrutinize device safety more closely, proactive approaches to PMS processes are becoming crucial. To solve some of the issues brought on by this shifting regulatory landscape, new technologies have been investigated. This study envisages the technical features of blockchain technology (BCT) and its role in enhancing the PMS for medical devices. To address the aforementioned challenges, our model involves the establishment of a secure, permissioned blockchain for PMS data management, utilizing a proof-of-authority consensus mechanism. This blockchain framework will exclusively permit a carefully vetted and designated set of participants to validate transactions and record them in the PMS data ledger. The utilization of BCT holds the potential to introduce enhanced efficiency and provide several advantages to the various stakeholders involved in the PMS procedure, including its potential to support emerging regulatory efforts.

Evaluating the complications of adult groin hernia where there is no hernia registry: a systematic review of Nigerian literature

BACKGROUND

Enumerating the complications of groin hernia repair might help to highlight the need for improvement in the quality of care. This is imperative in a country without a strong post-operative complication surveillance mechanism. Hence, this review aims to determine the complications encountered during the surgical treatment of groin hernias among Nigerian subjects.

METHODS

Databases like Google Scholar, Scopus, and PubMed were searched. Out of the 140 papers found during the search, only 20 were included in this review. Bassini repair was the most common type of hernia repair used, and neither laparoscopic repair nor posterior approach was utilized in any of the patients. Emergency presentations constituted about 18.5% of the cases. Meta-analysis of the studies showed that more prevalent complications were wound/scrotal edema (derived from four studies), surgical site infections (derived from 17 studies), and hematoma (from 19 studies). The rates were 23% (CI 0-46%; I2 = 80.9%), 6% (CI 3-10%; I2 = 87.7%), and 5% (CI 2-8%; I2 = 83.7%), respectively. The rate of complication in giant hernias was higher than the non-giant hernias and was statistically significant [p < 0.05; OR 1.5 (CI 0.9-2.4)]. Although the recurrence rate is low, there was insufficient follow-up of patients.

CONCLUSION

This review has shown that one-fifth of the patients had emergency repair of hernias and giant groin hernias have higher odds of complications after repair compared to normal-sized ones. The most common complication noted was wound/scrotal edema. None of the hernias was repaired with laparoscopy. Perhaps, establishing a registry might improve the detection of late complications in patients who had groin hernia repair.

Frameworks, Dimensions, Definitions of Aspects, and Assessment Methods for the Appraisal of Quality of Health Data for Secondary Use: Comprehensive Overview of Reviews

BACKGROUND

Health care has not reached the full potential of the secondary use of health data because of-among other issues-concerns about the quality of the data being used. The shift toward digital health has led to an increase in the volume of health data. However, this increase in quantity has not been matched by a proportional improvement in the quality of health data.

OBJECTIVE

This review aims to offer a comprehensive overview of the existing frameworks for data quality dimensions and assessment methods for the secondary use of health data. In addition, it aims to consolidate the results into a unified framework.

METHODS

A review of reviews was conducted including reviews describing frameworks of data quality dimensions and their assessment methods, specifically from a secondary use perspective. Reviews were excluded if they were not related to the health care ecosystem, lacked relevant information related to our research objective, and were published in languages other than English.

RESULTS

A total of 22 reviews were included, comprising 22 frameworks, with 23 different terms for dimensions, and 62 definitions of dimensions. All dimensions were mapped toward the data quality framework of the European Institute for Innovation through Health Data. In total, 8 reviews mentioned 38 different assessment methods, pertaining to 31 definitions of the dimensions.

CONCLUSIONS

The findings in this review revealed a lack of consensus in the literature regarding the terminology, definitions, and assessment methods for data quality dimensions. This creates ambiguity and difficulties in developing specific assessment methods. This study goes a step further by assigning all observed definitions to a consolidated framework of 9 data quality dimensions.

Electronic Medical Record Data Missingness and Interruption in Antiretroviral Therapy Among Adults and Children Living With HIV in Haiti: Retrospective Longitudinal Study

Background

Children (aged 0-14 years) living with HIV often experience lower rates of HIV diagnosis, treatment, and viral load suppression. In Haiti, only 63% of children living with HIV know their HIV status (compared to 85% overall), 63% are on treatment (compared to 85% overall), and 48% are virally suppressed (compared to 73% overall). Electronic medical records (EMRs) can improve HIV care and patient outcomes, but these benefits are largely dependent on providers having access to quality and nonmissing data.

Objective

We sought to understand the associations between EMR data missingness and interruption in antiretroviral therapy treatment by age group (pediatric vs adult).

Methods

We assessed associations between patient intake record data missingness and interruption in treatment (IIT) status at 6 and 12 months post antiretroviral therapy initiation using patient-level data drawn from iSanté, the most widely used EMR in Haiti. Missingness was assessed for tuberculosis diagnosis, World Health Organization HIV stage, and weight using a composite score indicator (ie, the number of indicators of interest missing). Risk ratios were estimated using marginal parameters from multilevel modified Poisson models with robust error variances and random intercepts for the facility to account for clustering.

Results

Data were drawn from 50 facilities and comprised 31,457 patient records from people living with HIV, of which 1306 (4.2%) were pediatric cases. Pediatric patients were more likely than adult patients to experience IIT (n=431, 33% vs n=7477, 23.4% at 6 months; P<.001). Additionally, pediatric patient records had higher data missingness, with 581 (44.5%) pediatric records missing at least 1 indicator of interest, compared to 7812 (25.9%) adult records (P<.001). Among pediatric patients, each additional indicator missing was associated with a 1.34 times greater likelihood of experiencing IIT at 6 months (95% CI 1.08-1.66; P=.008) and 1.24 times greater likelihood of experiencing IIT at 12 months (95% CI 1.05-1.46; P=.01). These relationships were not statistically significant for adult patients. Compared to pediatric patients with 0 missing indicators, pediatric patients with 1, 2, or 3 missing indicators were 1.59 (95% CI 1.26-2.01; P<.001), 1.74 (95% CI 1.02-2.97; P=.04), and 2.25 (95% CI 1.43-3.56; P=.001) times more likely to experience IIT at 6 months, respectively. Among adult patients, compared to patients with 0 indicators missing, having all 3 indicators missing was associated with being 1.32 times more likely to experience IIT at 6 months (95% CI 1.03-1.70; P=.03), while there was no association with IIT status for other levels of missingness.

Conclusions

These findings suggest that both EMR data quality and quality of care are lower for children living with HIV in Haiti. This underscores the need for further research into the mechanisms by which EMR data quality impacts the quality of care and patient outcomes among this population. Efforts to improve both EMR data quality and quality of care should consider prioritizing pediatric patients.

Development of an electronic burns register: Digitisation of routinely collected hospital data for global burns surveillance

INTRODUCTION

Burn registers provide important data that can track injury trends and evaluate services. Burn registers are concentrated in high-income countries, but most burn injuries occur in low- and middle-income countries where surveillance data are limited. Injury surveillance guidance recommends utilisation of existing routinely collected data where data quality is adequate, but there is a lack of guidance on how to achieve this. Our aim was to develop a rigorous and reproducible method to establish an electronic burn register from existing routinely collected data that can be implemented in low resource settings.

METHODS

Data quality of handwritten routinely collected records (register books) from a tertiary government hospital burn unit in Mysore, India was assessed prior to digitisation. Process mapping was conducted for burn patient presentations. Register and casualty records were compared to assess the case ascertainment rate. Register books from February 2016 to February 2022 were scanned and anonymised. Scans were quality checked and stored securely. An online data entry form was developed. All data underwent double verification.

RESULTS

Process mapping suggested data were reliable, and case ascertainment was 95%. 1930 presentations were recorded in the registers, representing 0.84% of hospital all-cause admissions. 388 pages were scanned with 4.4% requiring rescanning due to quality problems. Two-step verification estimated there to be errors remaining in 0.06% of fields following data entry.

CONCLUSION

We have described, using the example of a newly established electronic register in India, methods to assess the suitability and reliability of existing routinely collected data for surveillance purposes, to digitise handwritten data, and to quantify error during the digitisation process. The methods are likely to be of particular interest to burn units in countries with no active national burns register. We strongly recommend mobilisation of resources for digitisation of existing high quality routinely collected data as an important step towards developing burn surveillance systems in low resource settings.

Local Validation of a National Orthopaedic Registry

BACKGROUND/OBJECTIVE

Registries are limited by the quality of the data they collect. We aimed to measure the data entry error rate at a regional orthopaedic unit in a national arthroplasty registry and to assess a proposed intervention of restricting data entry to senior trainees.

METHODS AND MATERIALS

A total of 200 primary and revision arthroplasty cases (119 hips, 81 knees) were randomly selected from a single year, 2020. The Irish National Orthopaedic Registry was examined for the grade of the trainee that populated the form and the accuracy of 24 parameters by comparison with data recorded elsewhere in the patient record.

RESULTS

The mean number of errors per form was 2.17 (95% confidence interval (CI): 1.95-2.39), giving an overall error rate of 9% (95% CI: 8%-10.0%). Eighty-seven percent of forms examined contained inaccuracies, ranging from one to nine errors (4%-38%). Some parameters were more prone to errors, ranging from 1% to 28%. There was no evidence of total errors varying by trainee grade (analysis of variance (ANOVA) p-value: 0.34).

CONCLUSIONS

Error rates were in line with the literature. Results did not support restricting data entry to senior trainees.

MBSAQIP database: are the data reliable?

BACKGROUND

The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) database collects data from all accredited centers in the US. A prior study showed data quality issues limiting use of up to 20% of the 2015 database.

OBJECTIVES

To evaluate the completeness and data quality (internal validity, accuracy, and consistency) of the MBSAQIP database between 2015 and 2019.

SETTING

United States.

METHODS

All subsets of data from the MBSAQIP Participant User Data File (PUDF) were compiled into one main file. Completeness, internal validity, accuracy, and consistency were evaluated. Completeness was determined via missing values. Internal validity was assessed using the percentage of patients with a body mass index (BMI) < 30 kg/m2 who underwent primary bariatric surgery. Accuracy was evaluated using reported versus calculated BMI. Consistency was assessed using the percentage of patients with a gain of >5 or a loss of >20 units of BMI change in 30 days. Effects across years were assessed using a chi-squared test.

RESULTS

Missing data for age, BMI, and ASA was consistently low (<2.5%) with no significant difference across years. Only .02% of patients who underwent a primary bariatric procedure had a reported BMI <30 kg/m2. The mean difference between reported versus calculated BMI was -.02 units. A maximum of .33% of patients gained >5 units of BMI, and a maximum of .85% of patients lost > 20 units of BMI in early follow-up.

CONCLUSIONS

While the MBSAQIP is a database with acceptable data quality and minimal changes from 2015-2019, ongoing efforts are needed to improve data.

A Road Map to Creating a High-Quality Clinical Database in Plastic Surgery

BACKGROUND

Detailed in-house databases are a staple of surgical research and a crucial source of data for many studies from which clinical guidelines are built. Despite the importance of generating a clear and thorough developmental design, the literature on database creation and management is limited. In this article, the authors present their stepwise single-institution process of developing a clinical facial fracture database.

METHODS

The authors outline the process of development of a large single-institution clinical pediatric facial fracture database. The authors highlight critical steps from conception, regulatory approval, data safety/integrity, human resource allocation, data collection, quality assurance, and error remediation. The authors recorded patient characteristics, comorbidities, details of the sustained fracture, associated injuries, hospitalization information, treatments, outcomes, and follow-up information on Research Electronic Data Capture. Protocols were created to ensure data quality assurance and control. Error identification analysis was subsequently performed on the database to evaluate the completeness and accuracy of the data.

RESULTS

A total of 4451 records from 3334 patients between 2006 and 2021 were identified and evaluated to generate a clinical database. Overall, there were 259 incorrect entries of 120,177 total entries, yielding a 99.8% completion rate and a 0.216% error rate.

CONCLUSIONS

The quality of clinical research is intrinsically linked to the quality and accuracy of the data collection. Close attention must be paid to quality control at every stage of a database setup. More studies outlining the process of database design are needed to promote transparent, accurate, and replicable research practices.

Quality improvement of Dutch ICUs from 2009 to 2021: A registry based observational study

PURPOSE

To investigate the development in quality of ICU care over time using the Dutch National Intensive Care Evaluation (NICE) registry.

MATERIALS AND METHODS

We included data from all ICU admissions in the Netherlands from those ICUs that submitted complete data between 2009 and 2021 to the NICE registry. We determined median and interquartile range for eight quality indicators. To evaluate changes over time on the indicators, we performed multilevel regression analyses, once without and once with the COVID-19 years 2020 and 2021 included. Additionally we explored between-ICU heterogeneity by calculating intraclass correlation coefficients (ICC).

RESULTS

705,822 ICU admissions from 55 (65%) ICUs were included in the analyses. ICU length of stay (LOS), duration of mechanical ventilation (MV), readmissions, in-hospital mortality, hypoglycemia, and pressure ulcers decreased significantly between 2009 and 2019 (OR <1). After including the COVID-19 pandemic years, the significant change in MV duration, ICU LOS, and pressure ulcers disappeared. We found an ICC ≤0.07 on the quality indicators for all years, except for pressure ulcers with an ICC of 0.27 for 2009 to 2021.

CONCLUSIONS

Quality of Dutch ICU care based on seven indicators significantly improved from 2009 to 2019 and between-ICU heterogeneity is medium to small, except for pressure ulcers. The COVID-19 pandemic disturbed the trend in quality improvement, but unaltered the between-ICU heterogeneity.

A validation study of the 30-day questionnaire in the national Norwegian Tonsil Surgery Register: can we trust the data reported by the patients?

PURPOSE

The aim of this study was to validate the Patient Reported Outcome Measure (PROM) in the Norwegian Tonsil Surgery Register (NTSR) and to examine whether any improvements to the questionnaire could be useful.

METHODS

This is a prospective, descriptive study. NTSR collects data from patients who undergo tonsil surgery and the intention of the register is to improve the quality of treatment and to contribute to research. The patients answers questions about admission due to postoperative haemorrhage, infection and pain 30 days after surgery. 305 patients were contacted on phone 1-2 weeks after answering the questionnaires electronically (ePROM) and asked the same questions. 180 of 305 patients we contacted had some kind of complications after surgery. They were asked additional questions to search for possible points for improvement of the questionnaire.

RESULTS

When comparing the results on the ePROM with the answers on phone, we found that 12 out of 14 variables achieve almost perfect agreement (AC1 ≥ 0.81). Two variables are categorized to be substantial agreement (AC1 = 0.61-0.80). The additional questions showed us that the questionnaire can be improved with more detailed information regarding the severity of the postoperative haemorrhage and the need of better treatment against postoperative pain.

CONCLUSION

This study shows that the information from the 30-day ePROM has high reliability. The questions were understood as they were intended, and the answers reflect what the patients had of complications. Some changes can be done to improve the questionnaire and to open up for more research around the tonsillectomy procedure.

Error Rates of Data Processing Methods in Clinical Research: A Systematic Review and Meta-Analysis of Manuscripts Identified Through PubMed

Background

In clinical research, prevention of systematic and random errors of data collected is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported in the literature. Although there have been reports of data error and discrepancy rates in clinical studies, there has been little systematic synthesis of these results. Further, although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods. We aim to address this gap by evaluating error rates for 4 data processing methods.

Methods

A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials: medical record abstraction (MRA), optical scanning, single-data entry, and double-data entry. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison.

Results

A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively.

Conclusions

Data processing and cleaning methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.

An open-source system for efficient clinical trial support: The COMET study experience

Exercise clinical trials are complex, logistically burdensome, and require a well-coordinated multi-disciplinary approach. Challenges include managing, curating, and reporting on many disparate information sources, while remaining responsive to a variety of stakeholders. The Combined Exercise Trial (COMET, NCT04848038) is a one-year comparison of three exercise modalities delivered in the community. Target enrollment is 280 individuals over 4 years. To support rigorous execution of COMET, the study team has developed a suite of scripts and dashboards to assist study stakeholders in each of their various functions. The result is a highly automated study system that preserves rigor, increases communication, and reduces staff burden. This manuscript describes system considerations and the COMET approach to data management and use, with a goal of encouraging further development and adaptation by other study teams in various fields.

Orthopaedic surgeons' ability to detect pathologic hip fractures: review of 1484 fractures reported to the Norwegian Hip Fracture Register

BACKGROUND

The proximal femur is the most common location of metastases in the appendicular skeleton. Data on pathologic hip fractures, however, are sparse despite it is the most frequently operated pathologic fracture. The aim of this study was to investigate the ability of orthopaedic surgeons to identify pathologic hip fractures in an acute setting and secondly to validate the underlying cause of the pathologic fractures reported to Norwegian Hip Fracture Register (NHFR).

METHODS

In the NHFR dataset between 2005 and 2019, we identified 1484 fractures reported to be pathologic possibly secondary to a malignancy. These fractures were thoroughly validated by reviewing X-rays, the patient journal, the operation description for date, side, why there had been suspicion of pathologic fracture, and implant choice. Pathology reports were reviewed once a biopsy had been performed. Based on this validation, information in the NHFR was corrected, whenever necessary.

RESULTS

Of the 1484 fractures possible secondary to malignancy, 485 (32.7%) were not a pathologic fracture. When reviewing the 999 validated pathologic fractures, 15 patients had a pathologic fracture secondary to a benign lesion. The remaining 984 patients had a pathologic fracture secondary to malignancy. The underlying diagnosis reported was corrected in 442 of the 999 patients. The true rate of pathologic hip fractures secondary to malignancy in our material was 0.8%, and most patients had underlying prostate (30%), breast (20%), or lung (17%) cancer.

CONCLUSION

Orthopaedic surgeons in Norway failed to report correct data on pathologic fractures and the corresponding cancer diagnosis in an acute setting in many patients. The corrected data on pathologic fractures in the NHFR from 2005 to 2019 can now be a valid resource for further studies on the subject.

The Norwegian registry for spine surgery (NORspine): cohort profile

PURPOSE

To review and describe the development, methods and cohort of the lumbosacral part of the Norwegian registry for spine surgery (NORspine).

METHODS

NORspine was established in 2007. It is government funded, covers all providers and captures consecutive cases undergoing operations for degenerative disorders. Patients' participation is voluntary and requires informed consent. A set of baseline-, process- and outcome-variables (3 and 12 months) recommended by the International Consortium for Health Outcome Measurement is reported by surgeons and patients. The main outcome is the Oswestry disability index (ODI) at 12 months.

RESULTS

We show satisfactory data quality assessed by completeness, timeliness, accuracy, relevance and comparability. The coverage rate has been 100% since 2016 and the capture rate has increased to 74% in 2021. The cohort consists of 60,647 (47.6% women) cases with mean age 55.7 years, registered during the years 2007 through 2021. The proportions > 70 years and with an American Society of Anaesthesiologists' Physical Classification System (ASA) score > II has increased gradually to 26.1% and 19.3%, respectively. Mean ODI at baseline was 43.0 (standard deviation 17.3). Most cases were operated with decompression for disc herniation (n = 26,557, 43.8%) or spinal stenosis (n = 26,545, 43.8%), and 7417 (12.2%) with additional or primary fusion. The response rate at 12 months follow-up was 71.6%.

CONCLUSION

NORspine is a well-designed population-based comprehensive national clinical quality registry. The register's methods ensure appropriate data for quality surveillance and improvement, and research.

Data from national health registers as endpoints for the Tromsø Study: Correctness and completeness of stroke diagnoses

AIM

To assess whether stroke diagnoses in national health registers are sufficiently correct and complete to replace manual collection of endpoint data for the Tromsø Study, a population-based epidemiological study.

METHOD

Using the Tromsø Study Cardiovascular Disease Register for 2013-2014 as the gold standard, we calculated correctness (defined as positive predictive value, PPV) and completeness (defined as sensitivity) of stroke cases in four different data subsets derived from the Norwegian Patient Register and the Norwegian Stroke Register. We calculated the sensitivity and PPV with 95% confidence intervals (CIs) assuming a normal approximation of the binomial distribution.

RESULTS

In the Norwegian Stroke Register we found a sensitivity of 79.8% (95% CI 74.2-85.4) and a PPV of 97.5% (95% CI 95.1-99.9). In the Norwegian Patient Register the sensitivity was 86.4% (95% CI 81.6-91.1) and the PPV was 84.2% (95% CI 79.2-89.2). The overall highest levels were found in a subset based on a linkage between the Norwegian Stroke Register and the Norwegian Patient Register, with a sensitivity of 88.9% (95% CI 84.5-93.3), and a PPV of 89.3% (95% CI 85.0-93.6).

CONCLUSIONS

Data from the Norwegian Patient Register and from the linked data set between the Norwegian Patient Register and the Norwegian Stroke Register had acceptable levels of correctness and completeness to be considered as endpoint sources for the Tromsø Study Cardiovascular Disease Register. The benefits of using data from national registers as endpoints in epidemiological studies must be weighed against the impact of potentially decreased data quality.

Quality registry improves the data of chronic ulcers: Validation of Tampere Wound Registry

Quality registries are potential tools for improving health care documentation, but the quality and completeness of each registry should be ensured. This study aimed to evaluate the completion rate (completeness) and accuracy of data, first contact-to-registration time (timeliness), and case coverage of the Tampere Wound Registry (TWR) to assess whether it can be reliably used in clinical practice and for research purposes. Data from all 923 patients registered in the TWR between 5 June 2018 and 31 December 2020 were included in the analysis of data completeness, while data accuracy, timeliness and case coverage were analysed in those registered during the year 2020. In all analyses values over 80% were considered good and values over 90% excellent. The study showed that the overall completeness of the TWR was 81% and overall accuracy was 93%. Timeliness achieved 86% within the first 24 h, and case coverage was found to be 91%. When completion of seven selected variables was compared between TWR and patient medical records, the TWR was found to be more complete in five out of seven variables. In conclusion, the TWR proved to be a reliable tool for health care documentation and an even more reliable data source than patient medical records.

The Impact of Information Quality of Antimicrobial Susceptibility Test Report on the Rational Antimicrobial Use: A Retrospective Study

Background

Antimicrobial susceptibility test (AST) report was important for rational antimicrobial use. However, the reference value of AST report was sometimes limited due to poor information quality (IQ). This study aimed to measure the IQ of AST and evaluate the impact of IQ of AST report on rational antimicrobial use as a reference for antimicrobial therapy.

Methods

The retrospective study included data of AST report, antimicrobials prescribed after reporting AST results and related inpatient information. The inclusion criteria of the AST report included three conditions: 1. The AST reports were from inpatients with diagnosis of infection. 2. The bacteria were extracted from a sterile-site specimen. 3. The interpretive categories (ie sensitive, intermediary or resistance) were firstly reported during one hospitalization. The IQ of AST report was measured by the total IQ and IQ of completeness, usefulness, accuracy and consistency. The rational antimicrobial use was measured by the antimicrobial adherence to the interpretive categories of AST report. Fractional logit regression model (FLRM) was chosen to evaluate the impact of IQ on the rational antimicrobial use.

Results

The median of the total IQ, completeness, usefulness, accuracy and consistency were 0.7345, 0.6082, 0.9167, 0.8966 and 1.0000, respectively. The results of FLRM showed that usefulness, accuracy and consistency had significant positive impacts on the rational antimicrobial use (β = 4.220, P < 0.001; β = 3.987, P < 0.001; β = 0.511, P = 0.001, respectively), while the total IQ and completeness had no statistically significant impacts on the rational antimicrobial use (β = -0.820, P = 0.35; β = -0.793, P = 0.20, respectively).

Conclusion

This study confirmed that usefulness, accuracy and consistency performed well and had positive impacts on the rational antimicrobial use, which indicated that improving IQ especially usefulness, accuracy and consistency would make AST report play a greater role in promoting the rational antimicrobial use.

Validation of the Swedish Quality Register for Ear Surgery - SwedEar

BACKGROUND

The Swedish Quality Register for Ear Surgery (SwedEar) is a national register monitoring surgical procedures and outcomes of ear surgery to facilitate quality improvement. The value of the register is dependent on the quality of its data. SwedEar has never been validated regarding data quality or missing entries. Therefor, the purpose of this study was to assess coverage, completeness and response rate in the register and validate the physicians' reported data accuracy.

METHODS

In this validation study, the completeness, response rate and missing registrations were analysed. Data in SwedEar were compared with the yearly collected statistics of otosurgical procedures in The Swedish Otosurgical Society and the comparison of rates between groups was calculated with Fisher's exact test. Validation of registered data accuracy was performed on every 20th registered case during a five-year period. Data were reabstracted from medical records and compared with the original registration. Interrater agreement, reliability measures, Cohen's kappa, Gwet's AC1 and positive predictive value were calculated.

RESULTS

SwedEar has a coverage of 100%. The completeness of registered cases was 84% and the response rate was 74%. The validation of data accuracy assessed 13 530 variables, including audiograms. Less than 3% of incorrect or missing variables were identified. For most of the pre- and postoperative variables the Kappa and Gwet´s AC1 results show an almost perfect agreement (> 0.80). For audiogram data the ICC shows an excellent reliability (> 0.9) for all but one value.

CONCLUSION

This validation shows that SwedEar has excellent coverage, high completeness, and that the data in the register have almost perfect reliability. The data are suitable for both clinical and research purposes. Further efforts to improve completeness are warranted.

A Quality Assessment of the ARM-Net Registry Design and Data Collection

BACKGROUND

Registries are important in rare disease research. The Anorectal Malformation Network (ARM-Net) registry is a well-established European patient registry collecting demographic, clinical, and functional outcome data. We assessed the quality of this registry through review of the structure, data elements, collected data, and user experience.

MATERIAL AND METHODS

Design and data elements were assessed for completeness, consistency, usefulness, accuracy, validity, and comparability. An intra- and inter-user variability study was conducted through monitoring and re-registration of patients. User experience was assessed via a questionnaire on registration, design of registry, and satisfaction.

RESULTS

We evaluated 119 data elements, of which 107 were utilized and comprised 42 string and 65 numeric elements. A minority (37.0%) of the 2278 included records had complete data, though this improved to 83.5% when follow-up elements were excluded. Intra-observer variability demonstrated 11.7% incongruence, while inter-observer variability was 14.7%. Users were predominantly pediatric surgeons and typically registered patients within 11-30 min. Users did not experience any significant difficulties with data entry and were generally satisfied with the registry, but preferred more longitudinal data and patient-reported outcomes.

CONCLUSIONS

The ARM-Net registry presents one of the largest ARM cohorts. Although its collected data are valuable, they are susceptible to error and user variability. Continuous evaluations are required to maintain relevant and high-quality data and to achieve long-term sustainability. With the recommendations resulting from this study, we call for rare disease patient registries to take example and aim to continuously improve their data quality to enhance the small, but impactful, field of rare disease research.

LEVEL OF EVIDENCE

V.

Excellent agreement of Norwegian trauma registry data compared to corresponding data in electronic patient records

BACKGROUND

The Norwegian Trauma Registry (NTR) is designed to monitor and improve the quality and outcome of trauma care delivered by Norwegian trauma hospitals. Patient care is evaluated through specific quality indicators, which are constructed of variables reported to the registry by certified registrars. Having high-quality data recorded in the registry is essential for the validity, trust and use of data. This study aims to perform a data quality check of a subset of core data elements in the registry by assessing agreement between data in the NTR and corresponding data in electronic patient records (EPRs).

METHODS

We validated 49 of the 118 variables registered in the NTR by comparing those with the corresponding ones in electronic patient records for 180 patients with a trauma diagnosis admitted in 2019 at eight public hospitals. Agreement was quantified by calculating observed agreement, Cohen's Kappa and Gwet's first agreement coefficient (AC1) with 95% confidence intervals (CIs) for 27 nominal variables, quadratic weighted Cohen's Kappa and Gwet's second agreement coefficient (AC2) for five ordinal variables. For nine continuous, one date and seven time variables, we calculated intraclass correlation coefficient (ICC).

RESULTS

Almost perfect agreement (AC1 /AC2/ ICC > 0.80) was observed for all examined variables. Nominal and ordinal variables showed Gwet's agreement coefficients ranging from 0.85 (95% CI: 0.79-0.91) to 1.00 (95% CI: 1.00-1.00). For continuous and time variables there were detected high values of intraclass correlation coefficients (ICC) between 0.88 (95% CI: 0.83-0.91) and 1.00 (CI 95%: 1.00-1.00). While missing values in both the NTR and EPRs were in general negligeable, we found a substantial amount of missing registrations for a continuous "Base excess" in the NTR. For some of the time variables missing values both in the NTR and EPRs were high.

CONCLUSION

All tested variables in the Norwegian Trauma Registry displayed excellent agreement with the corresponding variables in electronic patient records. Variables in the registry that showed missing data need further examination.

Automated identification of patient subgroups: A case-study on mortality of COVID-19 patients admitted to the ICU

BACKGROUND

- Subgroup discovery (SGD) is the automated splitting of the data into complex subgroups. Various SGD methods have been applied to the medical domain, but none have been extensively evaluated. We assess the numerical and clinical quality of SGD methods.

METHOD

- We applied the improved Subgroup Set Discovery (SSD++), Patient Rule Induction Method (PRIM) and APRIORI - Subgroup Discovery (APRIORI-SD) algorithms to obtain patient subgroups on observational data of 14,548 COVID-19 patients admitted to 73 Dutch intensive care units. Hospital mortality was the clinical outcome. Numerical significance of the subgroups was assessed with information-theoretic measures. Clinical significance of the subgroups was assessed by comparing variable importance on population and subgroup levels and by expert evaluation.

RESULTS

- The tested algorithms varied widely in the total number of discovered subgroups (5-62), the number of selected variables, and the predictive value of the subgroups. Qualitative assessment showed that the found subgroups make clinical sense. SSD++ found most subgroups (n = 62), which added predictive value and generally showed high potential for clinical use. APRIORI-SD and PRIM found fewer subgroups (n = 5 and 6), which did not add predictive value and were clinically less relevant.

CONCLUSION

- Automated SGD methods find clinical subgroups that are relevant when assessed quantitatively (yield added predictive value) and qualitatively (intensivists consider the subgroups significant). Different methods yield different subgroups with varying degrees of predictive performance and clinical quality. External validation is needed to generalize the results to other populations and future research should explore which algorithm performs best in other settings.

Validation of forearm fracture diagnoses in administrative patient registers

The validity of forearm fracture diagnoses recorded in five Norwegian hospitals was investigated using image reports and medical records as gold standard. A relatively high completeness and correctness of the diagnoses was found. Algorithms used to define forearm fractures in administrative data should depend on study purpose.

PURPOSE

In Norway, forearm fractures are routinely recorded in the Norwegian Patient Registry (NPR). However, these data have not been validated. Data from patient administrative systems (PAS) at hospitals are sent unabridged to NPR. By using data from PAS, we aimed to examine (1) the validity of the forearm fracture diagnoses and (2) the usefulness of washout periods, follow-up codes, and procedure codes to define incident forearm fracture cases.

METHODS

This hospital-based validation study included women and men aged ≥ 19 years referred to five hospitals for treatment of a forearm fracture during selected periods in 2015. Administrative data for the ICD-10 forearm fracture code S52 (with all subgroups) in PAS and the medical records were reviewed. X-ray and computed tomography (CT) reports from examinations of forearms were reviewed independently and linked to the data from PAS. Sensitivity and positive predictive values (PPVs) were calculated using image reports and/or review of medical records as gold standard.

RESULTS

Among the 8482 reviewed image reports and medical records, 624 patients were identified with an incident forearm fracture during the study period. The sensitivity of PAS registrations was 90.4% (95% CI: 87.8-92.6). The PPV increased from 73.9% (95% CI: 70.6-77.0) in crude data to 90.5% (95% CI: 88.0-92.7) when using a washout period of 6 months. Using procedure codes and follow-up codes in addition to 6-months washout increased the PPV to 94.0%, but the sensitivity fell to 69.0%.

CONCLUSION

A relatively high sensitivity of forearm fracture diagnoses was found in PAS. PPV varied depending on the algorithms used to define cases. Choice of algorithm should therefore depend on study purposes. The results give useful measures of forearm fracture diagnoses from administrative patient registers. Depending on local coding practices and treatment pathways, we infer that the findings are relevant to other fracture diagnoses and registers.

Quality Control for Spine Registries: Development and Application of a New Protocol

Registries are gaining importance both in clinical practice and for research purposes. However, quality control is paramount to ensure that data are consistent and reliable. Quality control protocols have been proposed for arthroplasty registries, but these are not directly applicable to the spine setting. This study aims to develop a new quality control protocol for spine registries. Based on the available protocols for arthroplasty registries, a new protocol for spine registries was developed. The items included in the protocol were completeness (yearly enrollment rate and rate of assessment completion), consistency, and internal validity (coherence between registry data and medical records for blood loss, body mass index, and treated levels). All aspects were then applied to the spine registry of the Institution to verify its quality for each of the 5 years in which the registry has been used (2016-2020). Regarding completeness, the yearly enrollment rate ranged from 78 to 86%; the completion of preoperative assessment from 79% to 100%. The yearly consistency rate varied from 83% to 86%. Considering internal validity, the interclass correlation coefficient ranged from 0.1 to 0.8 for blood loss and from 0.3 to 0.9 for body mass index. The coherency for treated levels ranged from 25% to 82%. Overall, all 3 items showed an improvement over time. All 3 analyzed domains showed good to excellent results. The overall quality of the registered data improved over time.

Iranian Hydatid Disease Registry: Establishment and Implementation of a Neglected Tropical Disease Registry

BACKGROUND

Cystic echinococcosis (CE) or hydatid disease is a global public health concern which imposes considerable economic costs on the communities in endemic regions. CE surveillance data are not adequately reliable. The present study reports the development and outcomes of a CE registry in Iran.

METHODS

Hydatid Registry (HydatidReg) was initially established as a single-center registry in 2014 after the ethical approval of KMU. Following a call from MoHME to promote registry of different diseases and health outcomes, a call for participation was announced and all the Iranian Universities of Medical Sciences were requested to contribute to the registry. Subsequently, a nation-wide registry of hydatid disease was established in 2016. With a global perspective, HydatidReg joined the European Register of Cystic Echinococcosis (ERCE). A data collection form based on minimum dataset was designed and standard operating procedures (SOPs) were prepared to ensure standardized patient enrolment in the registry. A biobank system with two-dimensional barcoding was established along with HydatidReg for management and organization of biological specimens.

RESULTS

As of March 2021, a total of 690 patients were enrolled in the registry. HydatidReg registered 362 (17.3%) out of the total 2097 patients enrolled in ERCE. Quality control (QC) of the data demonstrated 91.2% completeness and 80% timeliness. In the biobank, 322 biological specimens from 184 CE patients have been deposited including 70 blood, 96 sera and 156 parasite materials.

CONCLUSION

High-quality data in the HydatidReg registry provided opportunities for health professionals to improve quality of care and organize meaningful research.

A quality management system aiming to ensure regulatory-grade data quality in a glaucoma registry

BACKGROUND

Disease/patient registries are underutilized despite their multiple advantages over clinical trials in the clinical evaluation of drugs, such as the capacity for long-term curation, provision of patient outcome data in routine clinical practice, and provision of benchmark data for comparison. Ensuring the fit-for-purpose quality of data generated from such registries is important to informing regulatory decision making. Here, we report the construction of a quality management system aiming to ensure regulatory-grade data quality for a registry of Japanese patients with glaucoma to evaluate long-term patient outcomes.

METHODS

The quality management system was established by reference to the risk-based approach in the ICH-E6 (R2) recommendations. The following three-component approach was taken: establishment of governance, computerized system validation (CSV), and implementation of risk assessment and control. Compliance of the system with the recommendations of regulatory guidelines relevant to use of the registry was assessed.

RESULTS

Governance by academic collaboration was established. This was followed by the development of a total of 15 standard operating procedures, including CSV, data management, monitoring, audit, and management of imaging data. The data management system was constructed based on a data management plan, which specified data/paper flow and data management procedures. The electronic data capture (EDC) system was audited by an external vendor, and configured and validated using the V-model framework as recommended in the GAMP5 guideline. Informed consent, eligibility assessment and major ophthalmology measurements were determined as Critical to Quality (CTQ) factors. A total of 22 risk items were identified and classified into three categories, and operationalized in the form of a risk control plan, which included training sessions and risk-based monitoring. The glaucoma registry addressed most quality recommendations in official guidelines issued by multiple health authorities, although two recommendations were not met.

CONCLUSIONS

We established and configured a quality management system for a glaucoma registry to ensure fit-for-purpose data quality for regulatory use, and to curate long-term follow-up data of glaucoma patients in a prospective manner.

The Nordic perioperative and intensive care registries-Collaboration and research possibilities

BACKGROUND

The Nordic perioperative and intensive care registries have been built up during the last 25 years to improve quality in intensive and perioperative care. We aimed to describe the Nordic perioperative and intensive care registries and to highlight possibilities and challenges in future research collaboration between these registries.

MATERIAL AND METHOD

We present an overview of the following Nordic registries: Swedish Perioperative Registry (SPOR), the Danish Anesthesia Database (DAD), the Finnish Perioperative Database (FIN-AN), the Icelandic Anesthesia Database (IS-AN), the Danish Intensive Care Database (DID), the Swedish Intensive Care Registry (SIR), the Finnish Intensive Care Consortium, the Norwegian Intensive Care and Pandemic Registry (NIPaR), and the Icelandic Intensive Care Registry (IS-ICU).

RESULTS

Health care systems and patient populations are similar in the Nordic countries. Despite certain differences in data structure and clinical variables, the perioperative and intensive care registries have enough in common to enable research collaboration. In the future, even a common Nordic registry could be possible.

CONCLUSION

Collaboration between the Nordic perioperative and intensive care registries is both possible and likely to produce research of high quality. Research collaboration between registries may have several add-on effects and stimulate international standardization regarding definitions, scoring systems, and benchmarks, thereby improving overall quality of care.

Opioid overdoses involving xylazine in emergency department patients: a multicenter study

INTRODUCTION

Illicit opioids, consisting largely of fentanyl, novel synthetic opioids, and adulterants, are the primary cause of drug overdose fatality in the United States. Xylazine, an alpha-2 adrenergic agonist and veterinary tranquilizer, is being increasingly detected among decedents following illicit opioid overdose. Clinical outcomes in non-fatal overdose involving xylazine are unexplored. Therefore, among emergency department patients with illicit opioid overdose, we evaluated clinical outcome differences for patients with and without xylazine exposures.

METHODS

This multicenter, prospective cohort study enrolled adult patients with opioid overdose who presented to one of nine United States emergency departments between 21 September 2020, and 17 August 2021. Patients with opioid overdose were screened and included if they tested positive for an illicit opioid (heroin, fentanyl, fentanyl analog, or novel synthetic opioid) or xylazine. Patient serum was analyzed via liquid chromatography quadrupole time-of-flight mass spectroscopy to detect current illicit opioids, novel synthetic opioids, xylazine and adulterants. Overdose severity surrogate outcomes were: (a) cardiac arrest requiring cardiopulmonary resuscitation (primary); and (b) coma within 4 h of arrival (secondary).

RESULTS

Three hundred and twenty-one patients met inclusion criteria: 90 tested positive for xylazine and 231 were negative. The primary outcome occurred in 37 patients, and the secondary outcome occurred in 111 patients. Using multivariable regression analysis, patients positive for xylazine had significantly lower adjusted odds of cardiac arrest (adjusted OR 0.30, 95% CI 0.10-0.92) and coma (adjusted OR 0.52, 95% CI 0.29-0.94).

CONCLUSIONS

In this large multicenter cohort, cardiac arrest and coma in emergency department patients with illicit opioid overdose were significantly less severe in those testing positive for xylazine.

Exploring the similarities and differences of variables collected by burn registers globally: protocol for a data dictionary review study

INTRODUCTION

Burn registers can provide high-quality clinical data that can be used for surveillance, research, planning service provision and clinical quality assessment. Many countrywide and intercountry burn registers now exist. The variables collected by burn registers are not standardised internationally. Few international burn register data comparisons are completed beyond basic morbidity and mortality statistics. Data comparisons across registers require analysis of homogenous variables. Little work has been done to understand whether burn registers have sufficiently similar variables to enable useful comparisons. The aim of this project is to compare the variables collected in countrywide and intercountry burn registers internationally to understand their similarities and differences.

METHODS AND ANALYSIS

Burn register custodians will be invited to participate in the study and to share their register data dictionaries. Study objectives are to compare patient inclusion and exclusion criteria of each participating burn register; determine which variables are collected by each register, and if variables are required or optional, identify common variable themes; and compare a sample of variables to understand how they are defined and measured. All variable names will be extracted from each register and common themes will be identified. Detailed information will be extracted for a sample of variables to give a deeper insight into similarities and differences between registers.

ETHICS AND DISSEMINATION

No patient data will be used in this project. Permission to use each register's data dictionary will be sought from respective register custodians. Results will be presented at international meetings and published in open access journals. These results will be of interest to register custodians and researchers wishing to explore international data comparisons, and countries wishing to establish their own burn register.

Do Patient Outcomes Predict Loss to Long-Term Follow-Up After Spine Surgery?

OBJECTIVE

To determine if spine surgery patients with greater improvement in patient-reported outcomes measures (PROMs) at early postoperative follow-up are more likely to be lost to follow-up at the 1-year and 2-year postoperative visits.

METHODS

All patients older than 18 years who underwent primary or revision decompression or fusion surgery for degenerative spinal conditions at an academic institution were retrospectively identified. Univariate analysis compared patient demographics, surgical factors, and changes in short-term and long-term postoperative PROMs (Neck Disability Index, Oswestry Disability Index, visual analog scale [VAS] neck, VAS arm, VAS back, VAS leg, and Short-Form 12 Physical and Mental Component Scores) across groups with and without 1 year and 2 years follow-up. Logistic regression isolated predictors of loss to follow-up.

RESULTS

A total of 1412 patients were included. Younger patient age, primary surgery, and single surgical approach independently predicted loss at 1 year follow-up. Female sex predicted loss at 2 years follow-up, whereas multilevel fusion surgery predicted attendance at 2 years clinical follow-up. In patients lost at 1 year follow-up compared with those who attended, preoperative to 3-month Mental Component Score and VAS neck pain improvement was significantly greater. When comparing patients based on 2 years follow-up status, VAS back pain improvement at 1 year postoperatively was significantly greater in patients lost to 2 years follow-up. All other changes in PROMs did not differ significantly by 1 or 2 years follow-up attendance.

CONCLUSIONS

Overall patient outcomes were not found to affect loss to long-term follow-up after spine surgery. The general lack of association between postoperative follow-up status and clinical outcome may limit bias introduced in retrospective PROM studies.

A Review of the State of the Art of Data Quality in Healthcare

Effective implementation of strategic data-driven health analysis initiatives is heavily dependent on the quality of the electronic medical records that serve as the foundation from which to improve clinical decisions and, in turn, the quality of care. Although there is a large body of research on the quality of healthcare data, a systematical understanding of the methods used to address the issues of data quality is missing. This study analyzes research articles in health information systems/healthcare informatics on data quality to derive a set of dimensions for understanding data quality. Issues related to each dimension are identified and methods used to address them summarized. The issues and methods can inform healthcare professionals of how to improve data practices.

Consensus statement on data to be entered in the ACL tear registry: SFA-DataLake

INTRODUCTION

Anterior cruciate ligament (ACL) reconstruction is a frequent procedure, with room for improvement by rehabilitation measures and associated peripheral and meniscal surgeries that are currently under assessment, requiring follow-up. Outside France, there have been ACL registries for 20 years now. The French Arthroscopy Society (SFA) decided to set up an ACL tear registry within its SFA DataLake registry platform.

MATERIAL AND METHOD

This article presents the methodology underlying the ACL Tear Registry: i.e., identification, definition and coding of essential and relevant data. A test phase comprised an initial assessment to improve data quality and overall coherence, to optimize data-entry time for patients and practitioners, who are the guarantors of the registry's use and efficacy.

RESULTS

The SFA DataLake ACL Tear Registry was made available to SFA members in December 2021. It aims to enable a review of practices for surgeons, early detection of failure of procedures and implants, with rates of failure and abnormal complications, and identification of prognostic factors for outcome, especially regarding original items that do not figure in previous registries.

CONCLUSION

SFA DataLake strikes a balance between "indispensable" and "original" items. The choice of contents and data quality is founded on a robust methodology with overall coherence, enabling analysis of large cohorts and comparisons with the literature and other registries. However, it remains to assess rates of data entry and item relevance as the Registry progresses.

LEVEL OF EVIDENCE

IV.

Machine Learning Analysis to Identify Data Entry Errors in Prehospital Patient Care Reports: A Case Study of a National Out-of-Hospital Cardiac Arrest Registry

Background: The objective of this study was to develop and validate machine learning models for data entry error detection in a national out-of-hospital cardiac arrest (OHCA) prehospital patient care report database.Methods: Adult OHCAs of presumed cardiac etiology were included. Data entry errors were defined as discrepancies between the coded data and the free-text note documenting the intervention or event; for example, information that was recorded as "absent" in the coded data but "present" in the free-text note. Machine learning models using the extreme gradient boosting, logistic regression, extreme gradient boosting outlier detection, and K-nearest neighbor outlier detection algorithms for error detection within nine core variables were developed and then validated for each variable.Results: Among 12,100 OHCAs, the proportion of cases with at least one error type was 16.2%. The area under the receiver operating characteristic curve (AUC) of the best-performing model (model with the highest AUC for each outcome variable) was 0.71-0.95. Machine learning models detected errors most efficiently for outcome place and initial rhythm errors; 82.6% of place errors and 93.8% of initial rhythm errors could be detected while checking 11 and 35% of data, respectively, compared to the strategy of checking all data.Conclusion: Machine learning models can detect data entry errors in care reports of emergency medical services (EMS) clinicians with acceptable performance and likely can improve the efficiency of the process of data quality control. EMS organizations that provide more prehospital interventions for OHCA patients could have higher error rates and may benefit from the adoption of error-detection models.

Assessing data accuracy in a large multi-institutional quality improvement registry: an update from the Pediatric Cardiac Critical Care Consortium (PC4)

BACKGROUND

The Pediatric Cardiac Critical Care Consortium (PC⁴) is a multi-institutional quality improvement registry focused on the care delivered in the cardiac ICU for patients with CHD and acquired heart disease. To assess data quality, a rigorous procedure of data auditing has been in place since the inception of the consortium.

MATERIALS AND METHODS

This report describes the data auditing process and quantifies the audit results for the initial 39 audits that took place after the transition from version one to version two of the registry's database.

RESULTS

In total, 2219 total encounters were audited for an average of 57 encounters per site. The overall data accuracy rate across all sites was 99.4%, with a major discrepancy rate of 0.52%. A passing score is based on an overall accuracy of >97% (achieved by all sites) and a major discrepancy rate of <1.5% (achieved by 38 of 39 sites, with 35 of 39 sites having a major discrepancy rate of <1%). Fields with the highest discrepancy rates included arrhythmia type, cardiac arrest count, and current surgical status.

CONCLUSIONS

The extensive PC⁴ auditing process, including initial and routinely scheduled follow-up audits of every participating site, demonstrates an extremely high level of accuracy across a broad array of audited fields and supports the continued use of consortium data to identify best practices in paediatric cardiac critical care.

Nurses experience increased clinical and organisational competence by working with a medical quality register, RevNatus - a qualitative study

Background

RevNatus is a consent-based, nationwide medical quality register that collects data on patients with inflammatory rheumatic diseases during pregnancy and one year postpartum. The entering of data takes place in outpatient clinics in rheumatology wards in hospitals. The aim of this study is to explore how rheumatology nurses experience organizing and working with the medical quality register RevNatus in addition to their normal clinical patient-care tasks.

Methods

Qualitative focus group interviews and individual in-depth interviews were conducted in 2018 to gain insights into how nurses organize performing quality register work and clinical work simultaneously. Data were analysed using systematic text condensation.

Results

The informants represented seven different rheumatology outpatient clinics in Norway. The analyses showed that working with RevNatus increased the nurses’ knowledge about pregnancy and rheumatic diseases, improved the content of their nurse consultations and found the ‘register form’ as a useful template to structure the nurse consultations. The nurses took the main responsibility for RevNatus, but lack of routines and uncoordinated collaboration with the rheumatologists and secretaries made the nurses spend too much time verifying the accuracy of data or post-registering missing data.

Conclusion

The nurses experienced work with RevNatus as time-consuming, but the register work increased both their clinical and organisational competences. Routines and collaboration within the registry team are important to ensure the data quality and reduce the workload.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08595-x.

Convolutional Neural Networks in Spinal Magnetic Resonance Imaging: A Systematic Review

OBJECTIVE

Convolutional neural networks (CNNs) are being increasingly used in the medical field, especially for image recognition in high-resolution, large-volume data sets. The study represents the current state of research on the application of CNNs in image segmentation and pathology detection in spine magnetic resonance imaging.

METHODS

For this systematic literature review, the authors performed a systematic initial search of the PubMed/Medline and Web of Science (Core collection) databases for eligible investigations. The authors limited the search to observational studies. Outcome parameters were analyzed according to the inclusion criteria and assigned to 3 groups: 1) segmentation of anatomical structures, 2) segmentation and evaluation of pathologic structures, and 3) specific implementation of CNNs.

RESULTS

Twenty-four retrospectively designed articles met the inclusion criteria. Publication dates ranged from 2017 to 2021. In total, 14,065 patients with 113,110 analyzed images were included. Most authors trained their network with a training-to-testing ratio of 80/20, while all but 2 articles used 5- to 10-fold cross-validation. Nine articles compared their performance results with other neural networks and algorithms, and all 24 articles described outcomes as positive.

CONCLUSIONS

State-of-the-art CNNs can detect and segment-specific anatomical landmarks and pathologies across a wide range, comparable to the skills of radiologists and experienced clinicians. With rapidly evolving network architectures and growing medical image databases, the future is likely to show growth in the development and refinement of these capable networks. However, the aid of automated segmentation and classification by neural networks cannot and should not be expected to replace clinical experts.

Translating periodontal data to knowledge in a learning health system

BACKGROUND

A learning health system (LHS) is a health system in which patients and clinicians work together to choose care on the basis of best evidence and to drive discovery as a natural outgrowth of every clinical encounter to ensure the right care at the right time. An LHS for dentistry is now feasible, as an increased number of oral health care encounters are captured in electronic health records (EHRs).

METHODS

The authors used EHRs data to track periodontal health outcomes at 3 large dental institutions. The 2 outcomes of interest were a new periodontitis case (for patients who had not received a diagnosis of periodontitis previously) and tooth loss due to progression of periodontal disease.

RESULTS

The authors assessed a total of 494,272 examinations (new periodontitis outcome: n = 168,442; new tooth loss outcome: n = 325,830), representing a total of 194,984 patients. Dynamic dashboards displaying performance on both measures over time allow users to compare demographic and risk factors for patients. The incidence of new periodontitis and tooth loss was 4.3% and 1.2%, respectively.

CONCLUSIONS

Periodontal disease, diagnosis, prevention, and treatment are particularly well suited for an LHS model. The results showed the feasibility of automated extraction and interpretation of critical data elements from the EHRs. The 2 outcome measures are being implemented as part of a dental LHS. The authors are using this knowledge to target the main drivers of poorer periodontal outcomes in a specific patient population, and they continue to use clinical health data for the purpose of learning and improvement.

PRACTICAL IMPLICATIONS

Dental institutions of any size can conduct contemporaneous self-evaluation and immediately implement targeted strategies to improve oral health outcomes.

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

PURPOSE

To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data.

METHODS

Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors.

RESULTS

A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors.

CONCLUSION

In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Development and validation of an automated basal cell carcinoma histopathology information extraction system using natural language processing

Introduction

Routinely collected healthcare data are a powerful research resource, but often lack detailed disease-specific information that is collected in clinical free text such as histopathology reports. We aim to use natural Language Processing (NLP) techniques to extract detailed clinical and pathological information from histopathology reports to enrich routinely collected data.

Methods

We used the general architecture for text engineering (GATE) framework to build an NLP information extraction system using rule-based techniques. During validation, we deployed our rule-based NLP pipeline on 200 previously unseen, de-identified and pseudonymised basal cell carcinoma (BCC) histopathological reports from Swansea Bay University Health Board, Wales, UK. The results of our algorithm were compared with gold standard human annotation by two independent and blinded expert clinicians involved in skin cancer care.

Results

We identified 11,224 items of information with a mean precision, recall, and F1 score of 86.0% (95% CI: 75.1-96.9), 84.2% (95% CI: 72.8-96.1), and 84.5% (95% CI: 73.0-95.1), respectively. The difference between clinician annotator F1 scores was 7.9% in comparison with 15.5% between the NLP pipeline and the gold standard corpus. Cohen's Kappa score on annotated tokens was 0.85.

Conclusion

Using an NLP rule-based approach for named entity recognition in BCC, we have been able to develop and validate a pipeline with a potential application in improving the quality of cancer registry data, supporting service planning, and enhancing the quality of routinely collected data for research.

Performance evaluation of a multinational data platform for critical care in Asia

Background: The value of medical registries strongly depends on the quality of the data collected. This must be objectively measured before large clinical databases can be promoted for observational research, quality improvement, and clinical trials. We aimed to evaluate the quality of a multinational intensive care unit (ICU) network of registries of critically ill patients established in seven Asian low- and middle-income countries (LMICs). Methods: The Critical Care Asia federated registry platform enables ICUs to collect clinical, outcome and process data for aggregate and unit-level analysis. The evaluation used the standardised criteria of the Directory of Clinical Databases (DoCDat) and a framework for data quality assurance in medical registries. Six reviewers assessed structure, coverage, reliability and validity of the ICU registry data. Case mix and process measures on patient episodes from June to December 2020 were analysed. Results: Data on 20,507 consecutive patient episodes from 97 ICUs in Afghanistan, Bangladesh, India, Malaysia, Nepal, Pakistan and Vietnam were included. The quality level achieved according to the ten prespecified DoCDat criteria was high (average score 3.4 out of 4) as was the structural and organizational performance -- comparable to ICU registries in high-income countries. Identified strengths were types of variables included, reliability of coding, data completeness and validation. Potential improvements included extension of national coverage, optimization of recruitment completeness validation in all centers and the use of interobserver reliability checks. Conclusions: The Critical Care Asia platform evaluates well using standardised frameworks for data quality and equally to registries in resource-rich settings.

Performance evaluation of a multinational data platform for critical care in Asia

Background: The value of medical registries strongly depends on the quality of the data collected. This must be objectively measured before large clinical databases can be promoted for observational research, quality improvement, and clinical trials. We aimed to evaluate the quality of a multinational intensive care unit (ICU) network of registries of critically ill patients established in seven Asian low- and middle-income countries (LMICs). Methods: The Critical Care Asia federated registry platform enables ICUs to collect clinical, outcome and process data for aggregate and unit-level analysis. The evaluation used the standardised criteria of the Directory of Clinical Databases (DoCDat) and a framework for data quality assurance in medical registries. Six reviewers assessed structure, coverage, reliability and validity of the ICU registry data. Case mix and process measures on patient episodes from June to December 2020 were analysed. Results: Data on 20,507 consecutive patient episodes from 97 ICUs in Afghanistan, Bangladesh, India, Malaysia, Nepal, Pakistan and Vietnam were included. The quality level achieved according to the ten prespecified DoCDat criteria was high (average score 3.4 out of 4) as was the structural and organizational performance -- comparable to ICU registries in high-income countries. Identified strengths were types of variables included, reliability of coding, data completeness and validation. Potential improvements included extension of national coverage, optimization of recruitment completeness validation in all centers and the use of interobserver reliability checks. Conclusions: The Critical Care Asia platform evaluates well using standardised frameworks for data quality and equally to registries in resource-rich settings.

Quality of medical records in public health facilities: A case of Southern Ethiopia, resource limited setting

Facility based retrospective study was conducted in three regions in southern Ethiopia to assess quality of medical records. A total of 2,145 medical records were reviewed from 73 public health facilities. Minimum standards of medical records were considered to assess completeness and legibility of records. The completeness of medical records were judged systematically according to national HMIS formats. From total of 2,145 medical cards reviewed, only 394 (18.4%) records had all complete and readable data. Gaps observed include 29.0% missed at least one of identification data, 14.3% lack chief compliant, 20.1% lack diagnosis, 12.5% lack medication and 60.3% records had no date and/or signature. Moreover, 9.5% cards had at least one non-readable component. Records at health centers were 56.8% less likely to be quality record as compared to records in hospitals. Even though completeness of every single record is must, only less than one fifth of records met quality of national medical record standard. Ministry of health should consider rules and regulation to maintain data quality and switching to electronic record, and finally progress in data quality should be monitored routinely.

Classifying outcomes in secondary and tertiary care clinical quality registries-an organizational case study with the COMET taxonomy

BACKGROUND

The choice of what patient outcomes are included in clinical quality registries is crucial for comparable and relevant data collection. Ideally, a uniform outcome framework could be used to classify the outcomes included in registries, steer the development of outcome measurement, and ultimately enable better patient care through benchmarking and registry research. The aim of this study was to compare clinical quality registry outcomes against the COMET taxonomy to assess its suitability in the registry context.

METHODS

We conducted an organizational case study that included outcomes from 63 somatic clinical quality registries in use at HUS Helsinki University Hospital, Finland. Outcomes were extracted and classified according to the COMET taxonomy and the suitability of the taxonomy was assessed.

RESULTS

HUS clinical quality registries showed great variation in outcome domains and in number of measures. Physiological outcomes were present in 98%, resource use in all, and functioning domains in 62% of the registries. Patient-reported outcome measures were found in 48% of the registries.

CONCLUSIONS

The COMET taxonomy was found to be mostly suitable for classifying the choice of outcomes in clinical quality registries, but improvements are suggested. HUS Helsinki University Hospital clinical quality registries exist at different maturity levels, showing room for improvement in life impact outcomes and in outcome prioritization. This article offers an example of classifying the choice of outcomes included in clinical quality registries and a comparison point for other registry evaluators.

Can we reliably automate clinical prognostic modelling? A retrospective cohort study for ICU triage prediction of in-hospital mortality of COVID-19 patients in the Netherlands

BACKGROUND

Building Machine Learning (ML) models in healthcare may suffer from time-consuming and potentially biased pre-selection of predictors by hand that can result in limited or trivial selection of suitable models. We aimed to assess the predictive performance of automating the process of building ML models (AutoML) in-hospital mortality prediction modelling of triage COVID-19 patients at ICU admission versus expert-based predictor pre-selection followed by logistic regression.

METHODS

We conducted an observational study of all COVID-19 patients admitted to Dutch ICUs between February and July 2020. We included 2,690 COVID-19 patients from 70 ICUs participating in the Dutch National Intensive Care Evaluation (NICE) registry. The main outcome measure was in-hospital mortality. We asessed model performance (at admission and after 24h, respectively) of AutoML compared to the more traditional approach of predictor pre-selection and logistic regression.

FINDINGS

Predictive performance of the autoML models with variables available at admission shows fair discrimination (average AUROC = 0·75-0·76 (sdev = 0·03), PPV = 0·70-0·76 (sdev = 0·1) at cut-off = 0·3 (the observed mortality rate), and good calibration. This performance is on par with a logistic regression model with selection of patient variables by three experts (average AUROC = 0·78 (sdev = 0·03) and PPV = 0·79 (sdev = 0·2)). Extending the models with variables that are available at 24h after admission resulted in models with higher predictive performance (average AUROC = 0·77-0·79 (sdev = 0·03) and PPV = 0·79-0·80 (sdev = 0·10-0·17)).

CONCLUSIONS

AutoML delivers prediction models with fair discriminatory performance, and good calibration and accuracy, which is as good as regression models with expert-based predictor pre-selection. In the context of the restricted availability of data in an ICU quality registry, extending the models with variables that are available at 24h after admission showed small (but significantly) performance increase.