Review: electronic health records and the reliability and validity of quality measures: a review of the literature

Appropriateness of Hysterectomy as Treatment for Benign Gynecological Conditions

Objective: To assess the appropriateness of hysterectomies performed at a large tertiary health system using the 1997 RAND appropriateness classification system and an updated algorithm. Design: We abstracted structured and unstructured data from electronic medical records on patient demographics, primary indication(s) for hysterectomy, diagnosis codes associated with the hysterectomy, previous treatments, and laboratory results. Subjects: Patients aged 18-44 years. Exposure: Receipt of hysterectomy for benign and nonobstetric conditions from October 2014 to December 2017. Main Outcome Measures: Using these data, we provided a RAND-based (dichotomous: inappropriate/appropriate) and Wright-based (3-level: inappropriate/ambiguous/appropriate) appropriateness rating and characterized missing information patterns associated with inappropriate ratings. Results: We analyzed 1,829 hysterectomies across 30 nonmutually exclusive primary indications for surgery. Nearly a third (32.8%) of surgeries had only one primary indication for surgery. Using the RAND-based classifier, 31.3% of hysterectomies were rated as appropriate and 68.7% as inappropriate. Using the Wright-based algorithm, 58.1% of hysterectomies were rated as appropriate, 15.7% as ambiguous, and 26.2% as inappropriate. Missing information on diagnostic procedures was the most common characteristic related to both RAND-based (46.1%) and Wright-based (51.2%) inappropriate ratings. Conclusions: The 1997 RAND classification lacked guidance for several contemporary indications, including gender-affirming care. RAND also has an outdated requirement for diagnostic surgeries such as laparoscopies, which have decreased in practice as diagnostic imaging has improved. Sensitivity analyses suggest that inappropriate surgeries cannot all be attributed to bias from missing electronic medical record data. Accurately documenting care delivery for benign gynecological conditions is key to ensuring quality and equity in gynecological care.

Navigating electronic health record accuracy by examination of sex incongruent conditions

OBJECTIVE

The increasing reliance on electronic health records (EHRs) for research and clinical care necessitates robust methods for assessing data quality and identifying inconsistencies. To address this need, we develop and apply the incongruence rate (IR) using sex-specific medical conditions. We also characterized participants with incongruent records to better understand the scope and nature of data discrepancies.

MATERIALS AND METHODS

In this cross-sectional study, we used the All of Us Research Program's latest version 7 (v7) EHR data to identify prevalent sex-specific conditions and evaluated the occurrence of incongruent cases, quantified as IR.

RESULTS

Among the 92 597 males and 152 551 females with condition occurrence data available from All of Us and sex-conformed gender, we identified 167 prevalent sex-specific conditions. Among the 37 537 biological males and 95 499 biological females with these sex-specific conditions, we detected an overall IR of 0.86%. Attempt to include non-cisgender participants result in inflated overall IR. Additionally, a significant proportion of participants with incongruent conditions also presented with conditions congruent to their biological sex, indicating a mix of accurate and erroneous records. These incongruences were not geographically or temporally isolated, suggesting systematic issues in EHR data integrity.

DISCUSSION

Our findings call attention to the existence of systemic data incongruences in sex-specific conditions and the need for robust validation checks. Extending IR evaluation to non-cisgender participants or non-sex-based conditions remain a challenge.

CONCLUSION

The sex condition-specific IR, when applied to adult populations, provides a valuable metric for data quality assessment in EHRs.

Lessons on the use of real-world data in medical device research: findings from the National Evaluation System for Health Technology Test-Cases

Aim: Although the US FDA encourages manufacturers of medical devices to submit real-world evidence (RWE) to support regulatory decisions, the ability of real-world data (RWD) to generate evidence suitable for decision making remains unclear. The 2017 Medical Device User Fee Amendments (MDUFA IV), authorized the National Evaluation System for health Technology Coordinating Center (NESTcc) to conduct pilot projects, or 'Test-Cases', to assess whether current RWD captures the information needed to answer research questions proposed by industry stakeholders. We synthesized key lessons about the challenges conducting research with RWD and the strategies used by research teams to enhance their ability to generate evidence from RWD based on 18 Test-Cases conducted between 2020 and 2022. Materials & methods: We reviewed study protocols and reports from each Test-Case team and conducted 49 semi-structured interviews with representatives of participating organizations. Interview transcripts were coded and thematically analyzed. Results: Challenges that stakeholders encountered in working with RWD included the lack of unique device identifiers, capturing key data elements and their appropriate meaning in structured data, limited reliability of diagnosis and procedure codes in structured data, extracting information from unstructured electronic health record (EHR) data, limited capture of long-term study end points, missing data and data sharing. Successful strategies included using manufacturer and supply chain data, leveraging clinical registries and registry reporting processes to collect and aggregate data, querying standardized EHR data, implementing natural language processing algorithms and using multidisciplinary research teams. Conclusion: The Test-Cases identified numerous challenges working with RWD but also opportunities to address these challenges and improve researchers' ability to use RWD to generate evidence on medical devices.

The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients' lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.

Accuracy of Electronic Health Record Documentation of Parental Presence: A Data Validation and Quality Improvement Analysis

Parental presence in the neonatal intensive care unit (NICU) is known to improve the health outcomes of an admitted infant. The use of the electronic health record (EHR) to analyze associations between parental presence and sociodemographic factors could provide important insights to families at greatest risk for limited presence during their infant's NICU stay, but there is little evidence about the accuracy of nonvital clinical measures such as parental presence in these datasets. A data validation study was conducted comparing the percentage agreement of an observational log of parental presence to the EHR documentation. Overall, high accuracy values were found when combining two methods of documentation. Additional stratification using a more specific measure, each chart's complete accuracy, instead of overall accuracy, revealed that night shift documentation was more accurate than day shift documentation (76.3% accurate during night shifts, 55.2% accurate during day shifts) and that flowsheet (FS) recordings were more accurate than the free-text plan of care (POC) notes (82.4% accurate for FS, 75.1% accurate for POC notes). This research provides a preliminary look at the accuracy of EHR documentation of nonclinical factors and can serve as a methodological roadmap for other researchers who intend to use EHR data.

Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force

This ISPOR Good Practices report provides a framework for assessing the suitability of electronic health records data for use in health technology assessments (HTAs). Although electronic health record (EHR) data can fill evidence gaps and improve decisions, several important limitations can affect its validity and relevance. The ISPOR framework includes 2 components: data delineation and data fitness for purpose. Data delineation provides a complete understanding of the data and an assessment of its trustworthiness by describing (1) data characteristics; (2) data provenance; and (3) data governance. Fitness for purpose comprises (1) data reliability items, ie, how accurate and complete the estimates are for answering the question at hand and (2) data relevance items, which assess how well the data are suited to answer the particular question from a decision-making perspective. The report includes a checklist specific to EHR data reporting: the ISPOR SUITABILITY Checklist. It also provides recommendations for HTA agencies and policy makers to improve the use of EHR-derived data over time. The report concludes with a discussion of limitations and future directions in the field, including the potential impact from the substantial and rapid advances in the diffusion and capabilities of large language models and generative artificial intelligence. The report's immediate audiences are HTA evidence developers and users. We anticipate that it will also be useful to other stakeholders, particularly regulators and manufacturers, in the future.

The Development of the Municipal Registry of People with Diabetes in Porto Alegre, Brazil

Background/Objective: Diabetes registries that enhance surveillance and improve medical care are uncommon in low- and middle-income countries, where most of the diabetes burden lies. We aimed to describe the methodological and technical aspects adopted in the development of a municipal registry of people with diabetes using local and national Brazilian National Health System databases. Methods: We obtained data between July 2018 and June 2021 based on eight databases covering primary care, specialty and emergency consultations, medication dispensing, outpatient exam management, hospitalizations, and deaths. We identified diabetes using the International Classification of Disease (ICD), International Classification of Primary Care (ICPC), medications for diabetes, hospital codes for the treatment of diabetes complications, and exams for diabetes management. Results: After data processing and database merging using deterministic and probabilistic linkage, we identified 73,185 people with diabetes. Considering that 1.33 million people live in Porto Alegre, the registry captured 5.5% of the population. Conclusions: With additional data processing, the registry can reveal information on the treatment and outcomes of people with diabetes who are receiving publicly financed care in Porto Alegre. It will provide metrics for epidemiologic surveillance, such as the incidence, prevalence, rates, and trends of complications and causes of mortality; identify inadequacies; and provide information. It will enable healthcare providers to monitor the quality of care, identify inadequacies, and provide feedback as needed.

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

OBJECTIVES

This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program.

METHODS

Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation.

RESULTS

The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%).

CONCLUSIONS

The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

Classifying early infant feeding status from clinical notes using natural language processing and machine learning

The objective of this study is to develop and evaluate natural language processing (NLP) and machine learning models to predict infant feeding status from clinical notes in the Epic electronic health records system. The primary outcome was the classification of infant feeding status from clinical notes using Medical Subject Headings (MeSH) terms. Annotation of notes was completed using TeamTat to uniquely classify clinical notes according to infant feeding status. We trained 6 machine learning models to classify infant feeding status: logistic regression, random forest, XGBoost gradient descent, k-nearest neighbors, and support-vector classifier. Model comparison was evaluated based on overall accuracy, precision, recall, and F1 score. Our modeling corpus included an even number of clinical notes that was a balanced sample across each class. We manually reviewed 999 notes that represented 746 mother-infant dyads with a mean gestational age of 38.9 weeks and a mean maternal age of 26.6 years. The most frequent feeding status classification present for this study was exclusive breastfeeding [n = 183 (18.3%)], followed by exclusive formula bottle feeding [n = 146 (14.6%)], and exclusive feeding of expressed mother's milk [n = 102 (10.2%)], with mixed feeding being the least frequent [n = 23 (2.3%)]. Our final analysis evaluated the classification of clinical notes as breast, formula/bottle, and missing. The machine learning models were trained on these three classes after performing balancing and down sampling. The XGBoost model outperformed all others by achieving an accuracy of 90.1%, a macro-averaged precision of 90.3%, a macro-averaged recall of 90.1%, and a macro-averaged F1 score of 90.1%. Our results demonstrate that natural language processing can be applied to clinical notes stored in the electronic health records to classify infant feeding status. Early identification of breastfeeding status using NLP on unstructured electronic health records data can be used to inform precision public health interventions focused on improving lactation support for postpartum patients.

A joint learning method for incomplete and imbalanced data in electronic health record based on generative adversarial networks

Electronic health records (EHR), present challenges of incomplete and imbalanced data in clinical predictions. Previous studies addressed these two issues with two-step separately, which caused the decrease in the performance of prediction tasks. In this paper, we propose a unified framework to simultaneously addresses the challenges of incomplete and imbalanced data in EHR. Based on the framework, we develop a model called Missing Value Imputation and Imbalanced Learning Generative Adversarial Network (MVIIL-GAN). We use MVIIL-GAN to perform joint learning on the imputation process of high missing rate data and the conditional generation process of EHR data. The joint learning is achieved by introducing two discriminators to distinguish the fake data from the generated data at sample-level and variable-level. MVIIL-GAN integrate the missing values imputation and data generation in one step, improving the consistency of parameter optimization and the performance of prediction tasks. We evaluate our framework using the public dataset MIMIC-IV with high missing rates data and imbalanced data. Experimental results show that MVIIL-GAN outperforms existing methods in prediction performance. The implementation of MVIIL-GAN can be found at https://github.com/Peroxidess/MVIIL-GAN.

Healthcare-associated infections and conditions in the era of digital measurement

As the third edition of the Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospitals is released with the latest recommendations for the prevention and management of healthcare-associated infections (HAIs), a new approach to reporting HAIs is just beginning to unfold. This next generation of HAI reporting will be fully electronic and based largely on existing data in electronic health record (EHR) systems and other electronic data sources. It will be a significant change in how hospitals report HAIs and how the Centers for Disease Control and Prevention (CDC) and other agencies receive this information. This paper outlines what that future electronic reporting system will look like and how it will impact HAI reporting.

Database quality assessment in research in paramedicine: a scoping review

BACKGROUND

Research in paramedicine faces challenges in developing research capacity, including access to high-quality data. A variety of unique factors in the paramedic work environment influence data quality. In other fields of healthcare, data quality assessment (DQA) frameworks provide common methods of quality assessment as well as standards of transparent reporting. No similar DQA frameworks exist for paramedicine, and practices related to DQA are sporadically reported. This scoping review aims to describe the range, extent, and nature of DQA practices within research in paramedicine.

METHODS

This review followed a registered and published protocol. In consultation with a professional librarian, a search strategy was developed and applied to MEDLINE (National Library of Medicine), EMBASE (Elsevier), Scopus (Elsevier), and CINAHL (EBSCO) to identify studies published from 2011 through 2021 that assess paramedic data quality as a stated goal. Studies that reported quantitative results of DQA using data that relate primarily to the paramedic practice environment were included. Protocols, commentaries, and similar study types were excluded. Title/abstract screening was conducted by two reviewers; full-text screening was conducted by two, with a third participating to resolve disagreements. Data were extracted using a piloted data-charting form.

RESULTS

Searching yielded 10,105 unique articles. After title and abstract screening, 199 remained for full-text review; 97 were included in the analysis. Included studies varied widely in many characteristics. Majorities were conducted in the United States (51%), assessed data containing between 100 and 9,999 records (61%), or assessed one of three topic areas: data, trauma, or out-of-hospital cardiac arrest (61%). All data-quality domains assessed could be grouped under 5 summary domains: completeness, linkage, accuracy, reliability, and representativeness.

CONCLUSIONS

There are few common standards in terms of variables, domains, methods, or quality thresholds for DQA in paramedic research. Terminology used to describe quality domains varied among included studies and frequently overlapped. The included studies showed no evidence of assessing some domains and emerging topics seen in other areas of healthcare. Research in paramedicine would benefit from a standardized framework for DQA that allows for local variation while establishing common methods, terminology, and reporting standards.

Inverse Probability of Treatment Weighting and Confounder Missingness in Electronic Health Record-based Analyses: A Comparison of Approaches Using Plasmode Simulation

BACKGROUND

Electronic health record (EHR) data represent a critical resource for comparative effectiveness research, allowing investigators to study intervention effects in real-world settings with large patient samples. However, high levels of missingness in confounder variables is common, challenging the perceived validity of EHR-based investigations.

METHODS

We investigated performance of multiple imputation and propensity score (PS) calibration when conducting inverse probability of treatment weights (IPTW)-based comparative effectiveness research using EHR data with missingness in confounder variables and outcome misclassification. Our motivating example compared effectiveness of immunotherapy versus chemotherapy treatment of advanced bladder cancer with missingness in a key prognostic variable. We captured complexity in EHR data structures using a plasmode simulation approach to spike investigator-defined effects into resamples of a cohort of 4361 patients from a nationwide deidentified EHR-derived database. We characterized statistical properties of IPTW hazard ratio estimates when using multiple imputation or PS calibration missingness approaches.

RESULTS

Multiple imputation and PS calibration performed similarly, maintaining ≤0.05 absolute bias in the marginal hazard ratio even when ≥50% of subjects had missing at random or missing not at random confounder data. Multiple imputation required greater computational resources, taking nearly 40 times as long as PS calibration to complete. Outcome misclassification minimally increased bias of both methods.

CONCLUSION

Our results support multiple imputation and PS calibration approaches to missingness in missing completely at random or missing at random confounder variables in EHR-based IPTW comparative effectiveness analyses, even with missingness ≥50%. PS calibration represents a computationally efficient alternative to multiple imputation.

Measuring the quality of skin cancer management in primary care: A scoping review

Skin cancer is a growing global problem and a significant health and economic burden. Despite the practical necessity for skin cancer to be managed in primary care settings, little is known about how quality of care is or should be measured in this setting. This scoping review aimed to capture the breadth and range of contemporary evidence related to the measurement of quality in skin cancer management in primary care settings. Six databases were searched for relevant texts reporting on quality measurement in primary care skin cancer management. Data from 46 texts published since 2011 were extracted, and quality measures were catalogued according to the three domains of the Donabedian model of healthcare quality (structure, process and outcome). Quality measures within each domain were inductively analysed into 13 key emergent groups. These represented what were deemed to be the most relevant components of skin cancer management as related to structure, process or outcomes measurement. Four groups related to the structural elements of care provision (e.g. diagnostic tools and equipment), five related to the process of care delivery (e.g. diagnostic processes) and four related to the outcomes of care (e.g. poor treatment outcomes). A broad range of quality measures have been documented, based predominantly on articles using retrospective cohort designs; systematic reviews and randomised controlled trials were limited.

Physicians' Perspectives on a Multi-Dimensional Model for the Roles of Electronic Health Records in Approaching a Proper Differential Diagnosis

Many healthcare organizations have adopted Electronic Health Records (EHRs) to improve the quality of care and help physicians make proper clinical decisions. The vital roles of EHRs can support the accuracy of diagnosis, suggest, and rationalize the provided care to patients. This study aims to understand the roles of EHRs in approaching proper differential diagnosis and optimizing patient safety. This study utilized a cross-sectional survey-based descriptive research design to assess physicians' perceptions of the roles of EHRs on diagnosis quality and safety. Physicians working in tertiary hospitals in Saudi Arabia were surveyed. Three hundred and fifty-one participants were included in the study, of which 61% were male. The main participants were family/general practice (22%), medicine, general (14%), and OB/GYN (12%). Overall, 66% of the participants ranked themselves as IT competent, most of the participants underwent IT self-guided learning, and 65% of the participants always used the system. The results generally reveal positive physicians' perceptions toward the roles of the EHR system on diagnosis quality and safety. There was a statistically significant relationship between user characteristics and the roles of the EHR by enhancing access to care, patient-physician encounter, clinical reasoning, diagnostic testing and consultation, follow-up, and diagnostic safety functionality. The study participants demonstrate positive perceptions of physicians toward the roles of the EHR system in approaching differential diagnosis. Yet, areas of improvement in the design and using EHRs are emphasized.

Methods for assessing guideline adherence for invasive procedures in the care of chronic coronary artery disease: a scoping review

OBJECTIVES

In the care of coronary artery disease (CAD), evidence questions the adequate application of guidelines for cardiovascular procedures, particularly coronary angiographies (CA) and myocardial revascularisation. This review aims to examine how care providers' guideline adherence for CA and myocardial revascularisation in the care of chronic CAD was assessed in the literature.

DESIGN

Scoping review.

DATA SOURCES

PubMed and EMBASE were searched through in June 2021 (rerun in September 2022).

ELIGIBILITY CRITERIA

We included studies assessing care providers' adherence to evidence-based guidelines for CA or myocardial revascularisation in the care of chronic CAD. Studies had to list the evaluation of guideline adherence as study objective, describe the evaluation methods used and report the underlying guidelines and recommendations.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers used standardised forms to extract study characteristics, methodological aspects such as data sources and variables, definitions of guideline adherence and quantification methods and the extent of guideline adherence. To elucidate the measurement of guideline adherence, the main steps were described.

RESULTS

Twelve studies (311 869 participants) were included, which evaluated guideline adherence by (1) defining guideline adherence, (2) specifying the study population, (3) assigning (classes of) recommendations and (4) quantifying adherence. Thereby, primarily secondary data were used. Studies differed in their definitions of guideline adherence, where six studies each considered only recommendation class I/grade A/strong recommendations as adherent or additionally recommendation classes IIa/IIb. Furthermore, some of the studies reported a priori definitions and allocation rules for the assignment of recommendation classes. Guideline adherence results ranged from 10% for percutaneous coronary intervention with prior heart team discussion to 98% for coronary artery bypass grafting.

CONCLUSION

Due to remarkable inconsistencies in the assessment, a cautious interpretation of the guideline adherence results is required. Future efforts should endeavour to establish a consistent understanding of the concept of guideline adherence.

The Priorities of End Users of Emergency Department Electronic Health Records: Modified Delphi Study

BACKGROUND

The needs of the emergency department (ED) pose unique challenges to modern electronic health record (EHR) systems. A diverse case load of high-acuity, high-complexity presentations, and ambulatory patients, all requiring multiple transitions of care, creates a rich environment through which to critically examine EHRs.

OBJECTIVE

This investigation aims to capture and analyze the perspective of end users of EHR about the strengths, limitations, and future priorities for EHR in the setting of the ED.

METHODS

In the first phase of this investigation, a literature search was conducted to identify 5 key usage categories of ED EHRs. Using key usage categories in the first phase, a modified Delphi study was conducted with a group of 12 panelists with expertise in both emergency medicine and health informatics. Across 3 rounds of surveys, panelists generated and refined a list of strengths, limitations, and key priorities.

RESULTS

The findings from this investigation highlighted the preference of panelists for features maximizing functionality of basic clinical features relative to features of disruptive innovation.

CONCLUSIONS

By capturing the perspectives of end users in the ED, this investigation highlights areas for the improvement or development of future EHRs in acute care settings.

Examining Real-World Therapy Practice of Cognitive Screening and Assessment in Post-Acute Care

OBJECTIVES

Routine implementation of cognitive screening and assessment by therapy providers in post-acute settings may promote improved care coordination. This study examined the frequency of cognitive screening and assessment documentation across post-acute settings, as well as its relationship with contextual factors and outcomes.

DESIGN

Cross-sectional observational study using Medicare claims and electronic health record data from 1 large health system.

SETTING AND PARTICIPANTS

Older adults admitted to post-acute care after an acute hospitalization.

METHODS

Descriptive analysis examined documentation of cognitive screening and assessment. Logistic and hierarchical linear regression evaluated the relationship among patient factors, cognitive screening and assessment, and patient outcomes.

RESULTS

The most common admission diagnoses for the final sample (n=2535) were total hip or knee joint replacement (41.7%) and stroke (15.3%). Following acute hospitalization, patients were discharged to inpatient rehabilitation (22.6%), skilled nursing (9.3%), or home health (68.1%). During the post-acute care stay, 38% of patients had documentation of cognitive screening by any therapy discipline. Patterns of documentation varied across disciplines and post-acute settings. Documentation of standardized cognitive assessments was limited, occurring for less than 2% of patients. Admission for stroke was associated with significantly higher odds of cognitive screening or assessment [odds ratio (OR) 2.07, 95% CI 1.13, 3.82] compared to patients with other diagnoses. There was no significant relationship between documentation of cognitive screening or assessment and 30-day readmissions (OR 0.81, 95% CI 0.53, 1.28).

CONCLUSION AND IMPLICATIONS

The key finding was inconsistent documentation of cognitive screening and assessment across disciplines and post-acute settings, which could be in part due to variation in electronic health record platforms. Future work can expand on these results to understand the degree to which contextual factors facilitate or inhibit routine delivery and documentation of cognitive screening and assessment. Findings can support implementation of standardized data elements to lead to improved care coordination and outcomes.

Genotype first: Clinical genomics research through a reverse phenotyping approach

Although genomic research has predominantly relied on phenotypic ascertainment of individuals affected with heritable disease, the falling costs of sequencing allow consideration of genomic ascertainment and reverse phenotyping (the ascertainment of individuals with specific genomic variants and subsequent evaluation of physical characteristics). In this research modality, the scientific question is inverted: investigators gather individuals with a genomic variant and test the hypothesis that there is an associated phenotype via targeted phenotypic evaluations. Genomic ascertainment research is thus a model of predictive genomic medicine and genomic screening. Here, we provide our experience implementing this research method. We describe the infrastructure we developed to perform reverse phenotyping studies, including aggregating a super-cohort of sequenced individuals who consented to recontact for genomic ascertainment research. We assessed 13 studies completed at the National Institutes of Health (NIH) that piloted our reverse phenotyping approach. The studies can be broadly categorized as (1) facilitating novel genotype-disease associations, (2) expanding the phenotypic spectra, or (3) demonstrating ex vivo functional mechanisms of disease. We highlight three examples of reverse phenotyping studies in detail and describe how using a targeted reverse phenotyping approach (as opposed to phenotypic ascertainment or clinical informatics approaches) was crucial to the conclusions reached. Finally, we propose a framework and address challenges to building collaborative genomic ascertainment research programs at other institutions. Our goal is for more researchers to take advantage of this approach, which will expand our understanding of the predictive capability of genomic medicine and increase the opportunity to mitigate genomic disease.

Development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard

Objective

To describe the development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard.

Materials and methods

Comparative study analyzing a manually extracted and an automatically extracted dataset with 262 patients treated for HNC cancer in a tertiary oncology center in the Netherlands in 2020. The primary outcome measures were the percentage of agreement on data elements required for calculating quality indicators and the difference between indicators results calculated using manually collected and indicators that used automatically extracted data.

Results

The results of this study demonstrate high agreement between manual and automatically collected variables, reaching up to 99.0% agreement. However, some variables demonstrate lower levels of agreement, with one variable showing only a 20.0% agreement rate. The indicator results obtained through manual collection and automatic extraction show high agreement in most cases, with discrepancy rates ranging from 0.3% to 3.5%. One indicator is identified as a negative outlier, with a discrepancy rate of nearly 25%.

Conclusions

This study shows that it is possible to use routinely collected structured data to reliably measure the quality of care in real-time, which could render manual data collection for quality measurement obsolete. To achieve reliable data reuse, it is important that relevant data is recorded as structured data during the care process. Furthermore, the results also imply that data validation is conditional to development of a reliable dashboard.

Determining the requirements of a medical records electronic deficiency management system: a mixed-method study

Purpose

Despite the presence of electronic medical records systems, traditional paper-based methods are often used in many countries to document data and eliminate medical record deficiencies. These methods waste patient and hospital resources. The purpose of this study is to evaluate the traditional deficiency management system and determine the requirements of an electronic deficiency management system in settings that currently use paper records alongside electronic hospital information systems.

Design/methodology/approach

This mixed-method study was performed in three phases. First, the traditional process of medical records deficiency management was qualitatively evaluated. Second, the accuracy of identifying deficiencies by the traditional and redesigned checklists was compared. Third, the requirements for an electronic deficiency management system were discussed in focus group sessions.

Findings

Problems in the traditional system include inadequate guidelines, incomplete procedures for evaluating sheets and subsequent delays in activities. Problems also included the omission of some vital data elements and a lack of feedback about the documentation deficiencies of each documenter. There was a significant difference between the mean number of deficiencies identified by traditional and redesigned checklists (p < 0.0001). The authors proposed an electronic deficiency management system based on redesigned checklists with improved functionalities such as discriminating deficiencies based on the documenter’s role, providing systematic feedback and generating automatic reports.

Originality/value

Previous studies only examined the positive effect of audit and feedback methods to enhance the documentation of data elements in electronic and paper medical records. The authors propose an electronic deficiency management system for medical records to solve those problems. Health-care policymakers, hospital managers and health information systems developers can use the proposed system to manage deficiencies and improve medical records documentation.

Developing Automated Computer Algorithms to Phenotype Periodontal Disease Diagnoses in Electronic Dental Records

OBJECTIVE

 Our objective was to phenotype periodontal disease (PD) diagnoses from three different sections (diagnosis codes, clinical notes, and periodontal charting) of the electronic dental records (EDR) by developing two automated computer algorithms.

METHODS

 We conducted a retrospective study using EDR data of patients (n = 27,138) who received care at Temple University Maurice H. Kornberg School of Dentistry from January 1, 2017 to August 31, 2021. We determined the completeness of patient demographics, periodontal charting, and PD diagnoses information in the EDR. Next, we developed two automated computer algorithms to automatically diagnose patients' PD statuses from clinical notes and periodontal charting data. Last, we phenotyped PD diagnoses using automated computer algorithms and reported the improved completeness of diagnosis.

RESULTS

 The completeness of PD diagnosis from the EDR was as follows: periodontal diagnosis codes 36% (n = 9,834), diagnoses in clinical notes 18% (n = 4,867), and charting information 80% (n = 21,710). After phenotyping, the completeness of PD diagnoses improved to 100%. Eleven percent of patients had healthy periodontium, 43% were with gingivitis, 3% with stage I, 36% with stage II, and 7% with stage III/IV periodontitis.

CONCLUSIONS

 We successfully developed, tested, and deployed two automated algorithms on big EDR datasets to improve the completeness of PD diagnoses. After phenotyping, EDR provided 100% completeness of PD diagnoses of 27,138 unique patients for research purposes. This approach is recommended for use in other large databases for the evaluation of their EDR data quality and for phenotyping PD diagnoses and other relevant variables.

Operationalizing a real-time scoring model to predict fall risk among older adults in the emergency department

Predictive models are increasingly being developed and implemented to improve patient care across a variety of clinical scenarios. While a body of literature exists on the development of models using existing data, less focus has been placed on practical operationalization of these models for deployment in real-time production environments. This case-study describes challenges and barriers identified and overcome in such an operationalization for a model aimed at predicting risk of outpatient falls after Emergency Department (ED) visits among older adults. Based on our experience, we provide general principles for translating an EHR-based predictive model from research and reporting environments into real-time operation.

The Increasing Population Movements in the 21st Century: A Call for the E-Register of Health-Related Data Integrating Health Care Systems in Europe

The escalating mass influx of people to Europe in the 21st century due to geopolitical and economic reasons as well as food crises ignites significant challenges for national health care services. The lack or disruption of cross-border, e-transferred, health-related data negatively affects the health outcome and continuous care, particularly in medically compromised individuals with an unsettled status. Proposal: The urgent need of a structured database, in the form of a health-related data register funded by the European Union that allows a swift exchange of crucial medical data, was discussed to flag ever-increasing migrants' health problems, with a primary aim to support an adequate health care provision for underserved people who are at risk of deteriorating health. The data security information technology aspects, with a proposed and drafted structure of an e-health register, were succinctly highlighted. Conclusions: Focusing on long-term benefits and considering future waves of mass relocation, an investment in a health-related data register in Europe could vastly reduce health care disparities between minority groups and improve epidemiological situations with regard to major illnesses, including common, communicable diseases as well as oncological and infectious conditions. Commissioners, policymakers, and stakeholders are urged to continue a collective action to ensure vulnerable people can access health services by responding to the ongoing global migration crisis.

Performance of EHR classifiers for patient eligibility in a clinical trial of precision screening

BACKGROUND

Validated computable eligibility criteria use real-world data and facilitate the conduct of clinical trials. The Genomic Medicine at VA (GenoVA) Study is a pragmatic trial of polygenic risk score testing enrolling patients without known diagnoses of 6 common diseases: atrial fibrillation, coronary artery disease, type 2 diabetes, breast cancer, colorectal cancer, and prostate cancer. We describe the validation of computable disease classifiers as eligibility criteria and their performance in the first 16 months of trial enrollment.

METHODS

We identified well-performing published computable classifiers for the 6 target diseases and validated these in the target population using blinded physician review. If needed, classifiers were refined and then underwent a subsequent round of blinded review until true positive and true negative rates ≥80% were achieved. The optimized classifiers were then implemented as pre-screening exclusion criteria; telephone screens enabled an assessment of their real-world negative predictive value (NPV-RW).

RESULTS

Published classifiers for type 2 diabetes and breast and prostate cancer achieved desired performance in blinded chart review without modification; the classifier for atrial fibrillation required two rounds of refinement before achieving desired performance. Among the 1077 potential participants screened in the first 16 months of enrollment, NPV-RW of the classifiers ranged from 98.4% for coronary artery disease to 99.9% for colorectal cancer. Performance did not differ by gender or race/ethnicity.

CONCLUSIONS

Computable disease classifiers can serve as efficient and accurate pre-screening classifiers for clinical trials, although performance will depend on the trial objectives and diseases under study.

Not all phenotypes are created equal: covariates of success in e-phenotype specification

BACKGROUND

Electronic (e)-phenotype specification by noninformaticist investigators remains a challenge. Although validation of each patient returned by e-phenotype could ensure accuracy of cohort representation, this approach is not practical. Understanding the factors leading to successful e-phenotype specification may reveal generalizable strategies leading to better results.

MATERIALS AND METHODS

Noninformaticist experts (n = 21) were recruited to produce expert-mediated e-phenotypes using i2b2 assisted by a honest data-broker and a project coordinator. Patient- and visit-sets were reidentified and a random sample of 20 charts matching each e-phenotype was returned to experts for chart-validation. Attributes of the queries and expert characteristics were captured and related to chart-validation rates using generalized linear regression models.

RESULTS

E-phenotype validation rates varied according to experts' domains and query characteristics (mean = 61%, range 20-100%). Clinical domains that performed better included infectious, rheumatic, neonatal, and cancers, whereas other domains performed worse (psychiatric, GI, skin, and pulmonary). Match-rate was negatively impacted when specification of temporal constraints was required. In general, the increase in e-phenotype specificity contributed positively to match-rate.

DISCUSSIONS AND CONCLUSIONS

Clinical experts and informaticists experience a variety of challenges when building e-phenotypes, including the inability to differentiate clinical events from patient characteristics or appropriately configure temporal constraints; a lack of access to available and quality data; and difficulty in specifying routes of medication administration. Biomedical query mediation by informaticists and honest data-brokers in designing e-phenotypes cannot be overstated. Although tools such as i2b2 may be widely available to noninformaticists, successful utilization depends not on users' confidence, but rather on creating highly specific e-phenotypes.

A Review of Potential National Chronic Pain Surveillance Systems in the United States

Pain has been established as a major public health problem in the United States (U.S.) with 50 million adults experiencing chronic pain and 20 million afflicted with high-impact chronic pain (ie, chronic pain that interferes with life or work activities). High financial and social costs are associated with chronic pain. Over the past 2 decades, pain management has been complicated by the marked increase in opioids prescribed to treat chronic noncancer pain and by the concurrent opioid crisis. Monitoring the prevalence of chronic pain and pain management is especially important because pain management is changing in uncertain ways. We review potential U.S. chronic pain surveillance systems, present potential difficulties of chronic pain surveillance, and explore how to address chronic pain surveillance in the current opioid era. We consider case definitions, severity, anatomic site, and varieties of chronic pain management strategies in reviewing and evaluating national surveys for chronic pain surveillance. Based on the criteria evaluated, the National Health Interview Survey offers the best single source for pain surveillance as the pain-related questions administered are brief, valid, and cover a broad scope of pain-related phenomena. PERSPECTIVE: This review article describes data sources that can be leveraged to conduct national chronic pain surveillance in the United States, explores case defining or pain-related questions administered, and evaluates them against 8 surveillance attributes.

Visual Acuity – Assessment of Data Quality and Usability in an Electronic Health Record System

Objective

To examine the data quality and usability of visual acuity (VA) data extracted from an electronic health record (EHR) system during ophthalmology encounters and provide recommendations for consideration of relevant VA end points in retrospective analyses.

Design

Retrospective, EHR data analysis.

Participants

All patients with eyecare office encounters at any 1 of the 9 locations of a large academic medical center between August 1, 2013, and December 31, 2015.

Methods

Data from 13 of the 21 VA fields (accounting for 93% VA data) in EHR encounters were extracted, categorized, recoded, and assessed for conformance and plausibility using an internal data dictionary, a 38-item listing of VA line measurements and observations including 28 line measurements (e.g., 20/30, 20/400) and 10 observations (e.g., no light perception). Entries were classified into usable and unusable data. Usable data were further categorized based on conformance to the internal data dictionary: (1) exact match; (2) conditional conformance, letter count (e.g., 20/30^+2-3); (3) convertible conformance (e.g., 5/200 to 20/800); (4) plausible but cannot be conformed (e.g., 5/400). Data were deemed unusable when they were not plausible.

Main Outcome Measures

Proportions of usable and unusable VA entries at the overall and subspecialty levels.

Results

All VA data from 513 036 encounters representing 166 212 patients were included. Of the 1 573 643 VA entries, 1 438 661 (91.4%) contained usable data. There were 1 196 720 (76.0%) exact match (category 1), 185 692 (11.8%) conditional conformance (category 2), 40 270 (2.6%) convertible conformance (category 3), and 15 979 (1.0%) plausible but not conformed entries (category 4). Visual acuity entries during visits with providers from retina (17.5%), glaucoma (14.0%), neuro-ophthalmology (8.9%), and low vision (8.8%) had the highest rates of unusable data. Documented VA entries with providers from comprehensive eyecare (86.7%), oculoplastics (81.5%), and pediatrics/strabismus (78.6%) yielded the highest proportions of exact match with the data dictionary.

Conclusions

Electronic health record VA data quality and usability vary across documented VA measures, observations, and eyecare subspecialty. We proposed a checklist of considerations and recommendations for planning, extracting, analyzing, and reporting retrospective study outcomes using EHR VA data. These are important first steps to standardize analyses enabling comparative research.

Early COVID-19 respiratory risk stratification using machine learning

Background

COVID-19 has strained healthcare systems globally. In this and future pandemics, providers with limited critical care experience must distinguish between moderately ill patients and those who will require aggressive care, particularly endotracheal intubation. We sought to develop a machine learning-informed Early COVID-19 Respiratory Risk Stratification (ECoRRS) score to assist in triage, by providing a prediction of intubation within the next 48 hours based on objective clinical parameters.

Methods

Electronic health record data from 3447 COVID-19 hospitalizations, 20.7% including intubation, were extracted. 80% of these records were used as the derivation cohort. The validation cohort consisted of 20% of the total 3447 records. Multiple randomizations of the training and testing split were used to calculate confidence intervals. Data were binned into 4-hour blocks and labeled as cases of intubation or no intubation within the specified time frame. A LASSO (least absolute shrinkage and selection operator) regression model was tuned for sensitivity and sparsity.

Results

Six highly predictive parameters were identified, the most significant being fraction of inspired oxygen. The model achieved an area under the receiver operating characteristic curve of 0.789 (95% CI 0.785 to 0.812). At 90% sensitivity, the negative predictive value was 0.997.

Discussion

The ECoRRS score enables non-specialists to identify patients with COVID-19 at risk of intubation within 48 hours with minimal undertriage and enables health systems to forecast new COVID-19 ventilator needs up to 48 hours in advance.

Level of evidence

IV.

Profiling hearing aid users through big data explainable artificial intelligence techniques

Debilitating hearing loss (HL) affects ~6% of the human population. Only 20% of the people in need of a hearing assistive device will eventually seek and acquire one. The number of people that are satisfied with their Hearing Aids (HAids) and continue using them in the long term is even lower. Understanding the personal, behavioral, environmental, or other factors that correlate with the optimal HAid fitting and with users' experience of HAids is a significant step in improving patient satisfaction and quality of life, while reducing societal and financial burden. In SMART BEAR we are addressing this need by making use of the capacity of modern HAids to provide dynamic logging of their operation and by combining this information with a big amount of information about the medical, environmental, and social context of each HAid user. We are studying hearing rehabilitation through a 12-month continuous monitoring of HL patients, collecting data, such as participants' demographics, audiometric and medical data, their cognitive and mental status, their habits, and preferences, through a set of medical devices and wearables, as well as through face-to-face and remote clinical assessments and fitting/fine-tuning sessions. Descriptive, AI-based analysis and assessment of the relationships between heterogeneous data and HL-related parameters will help clinical researchers to better understand the overall health profiles of HL patients, and to identify patterns or relations that may be proven essential for future clinical trials. In addition, the future state and behavioral (e.g., HAids Satisfiability and HAids usage) of the patients will be predicted with time-dependent machine learning models to assist the clinical researchers to decide on the nature of the interventions. Explainable Artificial Intelligence (XAI) techniques will be leveraged to better understand the factors that play a significant role in the success of a hearing rehabilitation program, constructing patient profiles. This paper is a conceptual one aiming to describe the upcoming data collection process and proposed framework for providing a comprehensive profile for patients with HL in the context of EU-funded SMART BEAR project. Such patient profiles can be invaluable in HL treatment as they can help to identify the characteristics making patients more prone to drop out and stop using their HAids, using their HAids sufficiently long during the day, and being more satisfied by their HAids experience. They can also help decrease the number of needed remote sessions with their Audiologist for counseling, and/or HAids fine tuning, or the number of manual changes of HAids program (as indication of poor sound quality and bad adaptation of HAids configuration to patients' real needs and daily challenges), leading to reduced healthcare cost.

Optometry independent prescribing during COVID lockdown in Wales

INTRODUCTION

During the COVID-19 lockdown, primary care optometry services in Wales moved to a hub model of provision. Three independent prescribing models were available in different areas: a commissioned Independent Prescribing Optometry Service (IPOS), independent prescribers that were not commissioned and no independent prescribers available. This allowed a unique opportunity for comparison.

METHOD

Optometry practices completed an online survey for each patient episode. Analysis of the data gave insight into patient presentation to urgent eye services and the drugs prescribed by optometrists. Medicines prescribed, sold or given and onward referral were compared between areas with an IPOS service (n = 2), those with prescribers but no commissioned service (n = 2) and those with no prescribers (n = 2).

RESULTS

Data from 22,434 reported patient episodes from 81 optometry practices in six health boards between 14 April 2020 and 30 June 2020 were analysed. Urgent care accounted for 10,997 (49.02%) first appointments and 1777 (7.92%) follow-ups. Most (18,006, 80.26%) patients self-referred. The most common presenting symptom was 'Eye pain/discomfort' (4818, 43.81% of urgent attendances). Anterior segment pathology was the most reported finding at first (6078, 55.27%) and follow-up (1316, 74.06%) urgent care appointments. Topical steroids (373, 25.99% of prescriptions) were the most prescribed medications. More medications were prescribed in areas with an IPOS service (1136, 79.16% of prescriptions) than areas with prescribers but no commissioned service. There were more follow-up appointments in optometric practice and fewer urgent referrals to ophthalmology in IPOS areas.

CONCLUSION

Urgent care services were most utilised by patients with discomfort caused by anterior eye conditions. IPOS services enabled optometrists to manage conditions to resolution without referral and without reduction in medications sold or given. Commissioners should recognise the value in reducing burden in urgent ophthalmology and the need for follow-up as part of a commissioned independent prescribing service.

Assessing risk factors for latent and active tuberculosis among persons living with HIV in Florida: A comparison of self-reports and medical records

PURPOSE

This study examined factors associated with TB among persons living with HIV (PLWH) in Florida and the agreement between self-reported and medically documented history of tuberculosis (TB) in assessing the risk factors.

METHODS

Self-reported and medically documented data of 655 PLWH in Florida were analyzed. Data on sociodemographic factors such as age, race/ethnicity, place of birth, current marital status, education, employment, homelessness in the past year and 'ever been jailed' and behavioural factors such as excessive alcohol use, marijuana, injection drug use (IDU), substance and current cigarette use were obtained. Health status information such as health insurance status, adherence to HIV antiretroviral therapy (ART), most recent CD4 count, HIV viral load and comorbid conditions were also obtained. The associations between these selected factors with self-reported TB and medically documented TB diagnosis were compared using Chi-square and logistic regression analyses. Additionally, the agreement between self-reports and medical records was assessed.

RESULTS

TB prevalence according to self-reports and medical records was 16.6% and 7.5% respectively. Being age ≥55 years, African American and homeless in the past 12 months were statistically significantly associated with self-reported TB, while being African American homeless in the past 12 months and not on antiretroviral therapy (ART) were statistically significantly associated with medically documented TB. African Americans compared to Whites had odds ratios of 3.04 and 4.89 for self-reported and medically documented TB, respectively. There was moderate agreement between self-reported and medically documented TB (Kappa = 0.41).

CONCLUSIONS

TB prevalence was higher based on self-reports than medical records. There was moderate agreement between the two data sources, showing the importance of self-reports. Establishing the true prevalence of TB and associated risk factors in PLWH for developing policies may therefore require the use of self-reports and confirmation by screening tests, clinical signs and/or microbiologic data.

Database quality assessment in research in paramedicine: a scoping review protocol

INTRODUCTION

The paramedic practice environment presents unique challenges to data documentation and access, as well as linkage to other parts of the healthcare system. Variable or unknown data quality can influence the validity of research in paramedicine. A number of database quality assessment (DQA) frameworks have been developed and used to evaluate data quality in other areas of healthcare. The extent these or other DQA practices have been applied to paramedic research is not known. Accordingly, this scoping review aims to describe the range, extent and nature of DQA practices within research in paramedicine.

METHODS AND ANALYSIS

This scoping review will follow established methods for the conduct (Johanna Briggs Institute; Arksey and O'Malley) and reporting (Preferred Reporting Items in Systematic Reviews and Meta-Analyses extension for scoping reviews) of scoping reviews. In consultation with a professional librarian, a search strategy was developed representing the applicable population, concept and context. This strategy will be applied to MEDLINE (National Library of Medicine), Embase (Elsevier), Scopus (Elsevier) and CINAHL (EBSCO) to identify studies published from 2011 through 2021 that assess paramedic data quality as a stated goal. Studies will be included if they report quantitative results of DQA using data that relate primarily to the paramedic practice environment. Protocols, commentaries, case studies, interviews, simulations and experimental data-processing techniques will be excluded. No restrictions will be placed on language. Study selection will be performed by two reviewers, with a third available to resolve conflicts. Data will be extracted from included studies using a data-charting form piloted and iteratively revised based on studies known to be relevant. Results will be summarised in a chart of study characteristics, DQA-specific outcomes and key findings.

ETHICS AND DISSEMINATION

Ethical approval is not required. Results will be submitted to relevant conferences and peer-reviewed journals.

TRIAL REGISTRATION

10.17605/OSF.IO/Z287T.

Infants With Congenital Muscular Torticollis: Demographic Factors, Clinical Characteristics, and Physical Therapy Episode of Care

PURPOSE

To describe demographic factors, baseline characteristics, and physical therapy episodes in infants with congenital muscular torticollis (CMT), examine groups based on physical therapy completion, and identify implications for clinical practice.

METHODS

Retrospective data were extracted from a single-site registry of 445 infants with CMT.

RESULTS

Most infants were male (57%), Caucasian (63%), and firstborn (50%), with torticollis detected by 3 months old (89%) with a left (51%), mild (72%) CMT presentation. Cervical range of motion (ROM) limitations were greatest in passive lateral flexion and active rotation. Sixty-seven percent of infants completed an episode of physical therapy, 25% completed a partial episode, and 8% did not attend visits following the initial examination. Age at examination, ROM, and muscle function differed significantly between groups.

CONCLUSIONS

Physical therapists may use clinical registry data to inform practice for timing of referral, frequency of care, and clinician training to manage infants with CMT.

Inaccurate recording of routinely collected data items influences identification of COVID-19 patients

Background

During the Coronavirus disease 2019 (COVID-19) pandemic it became apparent that it is difficult to extract standardized Electronic Health Record (EHR) data for secondary purposes like public health decision-making. Accurate recording of, for example, standardized diagnosis codes and test results is required to identify all COVID-19 patients. This study aimed to investigate if specific combinations of routinely collected data items for COVID-19 can be used to identify an accurate set of intensive care unit (ICU)-admitted COVID-19 patients.

Methods

The following routinely collected EHR data items to identify COVID-19 patients were evaluated: positive reverse transcription polymerase chain reaction (RT-PCR) test results; problem list codes for COVID-19 registered by healthcare professionals and COVID-19 infection labels. COVID-19 codes registered by clinical coders retrospectively after discharge were also evaluated. A gold standard dataset was created by evaluating two datasets of suspected and confirmed COVID-19-patients admitted to the ICU at a Dutch university hospital between February 2020 and December 2020, of which one set was manually maintained by intensivists and one set was extracted from the EHR by a research data management department. Patients were labeled ‘COVID-19′ if their EHR record showed diagnosing COVID-19 during or right before an ICU-admission. Patients were labeled ‘non-COVID-19′ if the record indicated no COVID-19, exclusion or only suspicion during or right before an ICU-admission or if COVID-19 was diagnosed and cured during non-ICU episodes of the hospitalization in which an ICU-admission took place. Performance was determined for 37 queries including real-time and retrospective data items. We used the F₁ score, which is the harmonic mean between precision and recall. The gold standard dataset was split into one subset including admissions between February and April and one subset including admissions between May and December to determine accuracy differences.

Results

The total dataset consisted of 402 patients: 196 ‘COVID-19′ and 206 ‘non-COVID-19′ patients. F₁ scores of search queries including EHR data items that can be extracted real-time ranged between 0.68 and 0.97 and for search queries including the data item that was retrospectively registered by clinical coders F₁ scores ranged between 0.73 and 0.99. F₁ scores showed no clear pattern in variability between the two time periods.

Conclusions

Our study showed that one cannot rely on individual routinely collected data items such as coded COVID-19 on problem lists to identify all COVID-19 patients. If information is not required real-time, medical coding from clinical coders is most reliable. Researchers should be transparent about their methods used to extract data. To maximize the ability to completely identify all COVID-19 cases alerts for inconsistent data and policies for standardized data capture could enable reliable data reuse.

Neurocritical Care Performance Measures Derived from Electronic Health Record Data are Feasible and Reveal Site-Specific Variation: A CHoRUS Pilot Project

BACKGROUND

We evaluated the feasibility and discriminability of recently proposed Clinical Performance Measures for Neurocritical Care (Neurocritical Care Society) and Quality Indicators for Traumatic Brain Injury (Collaborative European NeuroTrauma Effectiveness Research in TBI; CENTER-TBI) extracted from electronic health record (EHR) flowsheet data.

METHODS

At three centers within the Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI consortium, we examined consecutive neurocritical care admissions exceeding 24 h (03/2015-02/2020) and evaluated the feasibility, discriminability, and site-specific variation of five clinical performance measures and quality indicators: (1) intracranial pressure (ICP) monitoring (ICPM) within 24 h when indicated, (2) ICPM latency when initiated within 24 h, (3) frequency of nurse-documented neurologic assessments, (4) intermittent pneumatic compression device (IPCd) initiation within 24 h, and (5) latency to IPCd application. We additionally explored associations between delayed IPCd initiation and codes for venous thromboembolism documented using the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) system. Median (interquartile range) statistics are reported. Kruskal-Wallis tests were measured for differences across centers, and Dunn statistics were reported for between-center differences.

RESULTS

A total of 14,985 admissions met inclusion criteria. ICPM was documented in 1514 (10.1%), neurologic assessments in 14,635 (91.1%), and IPCd application in 14,175 (88.5%). ICPM began within 24 h for 1267 (83.7%), with site-specific latency differences among sites 1-3, respectively, (0.54 h [2.82], 0.58 h [1.68], and 2.36 h [4.60]; p < 0.001). The frequency of nurse-documented neurologic assessments also varied by site (17.4 per day [5.97], 8.4 per day [3.12], and 15.3 per day [8.34]; p < 0.001) and diurnally (6.90 per day during daytime hours vs. 5.67 per day at night, p < 0.001). IPCds were applied within 24 h for 12,863 (90.7%) patients meeting clinical eligibility (excluding those with EHR documentation of limiting injuries, actively documented as ambulating, or refusing prophylaxis). In-hospital venous thromboembolism varied by site (1.23%, 1.55%, and 5.18%; p < 0.001) and was associated with increased IPCd latency (overall, 1.02 h [10.4] vs. 0.97 h [5.98], p = 0.479; site 1, 2.25 h [10.27] vs. 1.82 h [7.39], p = 0.713; site 2, 1.38 h [5.90] vs. 0.80 h [0.53], p = 0.216; site 3, 0.40 h [16.3] vs. 0.35 h [11.5], p = 0.036).

CONCLUSIONS

Electronic health record-derived reporting of neurocritical care performance measures is feasible and demonstrates site-specific variation. Future efforts should examine whether performance or documentation drives these measures, what outcomes are associated with performance, and whether EHR-derived measures of performance measures and quality indicators are modifiable.

Annual Research Review: A systematic review of mental health services for emerging adults - moulding a precipice into a smooth passage

BACKGROUND

The transition between child and adult services should aim to support young people into the next stage of their life in a way that optimises their function. Yet financial, organisational and procedural barriers to continuity of care often hamper smooth transition between child and adult services.

AIM AND METHOD

We reviewed studies of transition from child to adult mental health services, focusing on: (a) rates of referrals and referral acceptance; (b) barriers and facilitators of successful transition; (c) continuity of care during and post-transition and (d) service users' experience of transition. Studies were identified through systematic searches of electronic databases: PsycINFO, Medline, Embase and Child Development and Adolescent Studies.

FINDINGS

Forty-seven papers describing 43 unique studies met inclusion criteria. Service provision is influenced by previous history and funding processes, and the presence or absence of strong primary care, specialist centres of excellence and coordination between specialist and primary care. Provision varies between and within countries, particularly whether services are restricted to 'core' mental health or broader needs. Unsupportive organisational culture, fragmentation of resources, skills and knowledge base undermine the collaborative working essential to optimise transition. Stigma and young people's concerns about peers' evaluation often prompt disengagement and discontinuation of care during transition, leading to worsening of symptoms and later, to service re-entry. Qualitative studies reveal that young people and families find the transition process frustrating and difficult, mainly because of lack of advanced planning and inadequate preparation.

CONCLUSIONS

Despite increasing research interest over the last decade, transition remains 'poorly planned, executed and experienced'. Closer collaboration between child and adult services is needed to improve the quality of provision for this vulnerable group at this sensitive period of development.

Prevalence and Factors Associated with Patient-Requested Corrections to the Medical Record through Use of a Patient Portal: Findings from a National Survey

BACKGROUND

Providing patients with medical records access is one strategy that health systems can utilize to reduce medical errors. However, how often patients request corrections to their records on a national scale is unknown.

OBJECTIVES

We aimed to develop population-level estimates of patients who request corrections to their medical records using national-level data. We also identified patient-level correlates of requesting corrections.

METHODS

We used the 2017 and 2019 Health Information National Trends Survey and examined all patient portal adopters. We applied jackknife replicate weights to develop population-representative estimates of the prevalence of requesting medical record corrections. We conducted a multivariable logistic regression analysis to identify correlates of requesting corrections while controlling for demographic factors, health care utilization patterns, health status, technology/internet use patterns, and year.

RESULTS

Across 1,657 respondents, 125 (weighted estimate: 6.5%) reported requesting corrections to their medical records. In unadjusted models, greater odds of requesting corrections were observed among patients who reported their race/ethnicity as non-Hispanic black (odds ratio [OR]: 2.20, 95% confidence interval [CI]: 1.10-4.43), had frequent portal visits (OR: 3.92, 95% CI: 1.51-10.23), and had entered data into the portal (OR: 7.51, 95% CI: 4.08-13.81). In adjusted models, we found greater odds of requesting corrections among those who reported frequent portal visits (OR: 3.39, 95% CI: 1.24-9.33) and those who reported entering data into the portal (OR: 6.43, 95% CI: 3.20-12.94). No other significant differences were observed.

CONCLUSION

Prior to the Information Blocking Final Rule in April 2021, approximately 6.5% of patients requested corrections of errors in their medical records at the national level. Those who reported higher engagement with their health, as proxied by portal visit frequency and entering data into the portal, were more likely to request corrections.

Extraction of Active Medications and Adherence Using Natural Language Processing for Glaucoma Patients

Accuracy of medication data in electronic health records (EHRs) is crucial for patient care and research, but many studies have shown that medication lists frequently contain errors. In contrast, physicians often pay more attention to the clinical notes and record medication information in them. The medication information in notes may be used for medication reconciliation to improve the medication lists' accuracy. However, accurately extracting patient's current medications from free-text narratives is challenging. In this study, we first explored the discrepancies between medication documentation in medication lists and progress notes for glaucoma patients by manually reviewing patients' charts. Next, we developed and validated a named entity recognition model to identify current medication and adherence from progress notes. Lastly, a prototype tool for medication reconciliation using the developed model was demonstrated. In the future, the model has the potential to be incorporated into the EHR system to help with realtime medication reconciliation.

Comparison of electronic versus manual abstraction for 2 standardized perinatal care measures

OBJECTIVE

Given that electronic clinical quality measures (eCQMs) are playing a central role in quality improvement applications nationwide, a stronger evidence base demonstrating their reliability is critically needed. To assess the reliability of electronic health record-extracted data elements and measure results for the Elective Delivery and Exclusive Breast Milk Feeding measures (vs manual abstraction) among a national sample of US acute care hospitals, as well as common sources of discrepancies and change over time.

MATERIALS AND METHODS

eCQM and chart-abstracted data for the same patients were matched and compared at the data element and measure level for hospitals submitting both sources of data to The Joint Commission between 2017 and 2019. Sensitivity, specificity, and kappa statistics were used to assess reliability.

RESULTS

Although eCQM denominator reliability had moderate to substantial agreement for both measures and both improved over time (Elective Delivery: kappa = 0.59 [95% confidence interval (CI), 0.58-0.61] in 2017 and 0.84 [95% CI, 083-0.85] in 2019; Exclusive Breast Milk Feeding: kappa = 0.58 [95% CI, 0.54-0.62] in 2017 and 0.70 [95% CI, 0.67-0.73] in 2019), the numerator status reliability was poor for Elective Delivery (kappa = 0.08 [95% CI, 0.03-0.12] in 2017 and 0.10 [95% CI, 0.05-0.15] in 2019) but near perfect for Exclusive Breast Milk Feeding (kappa = 0.85 [0.83, 0.87] in 2017 and 0.84 [0.83, 0.85] in 2019). The failure of the eCQM to accurately capture estimated gestational age, conditions possibly justifying elective delivery, active labor, and medical induction were the main reasons for the discrepancies.

CONCLUSIONS

Although eCQM denominator reliability for the Elective Delivery and Exclusive Breast Milk Feeding measures had moderate agreement when compared to medical record review, the numerator status reliability was poor for Elective Delivery, but near perfect for Exclusive Breast Milk Feeding. Improvements in eCQM data capture of some key data elements would greatly improve the reliability.

How to assure the quality of clinical records? A 7-year experience in a large academic hospital

INTRODUCTION

Clinical record (CR) is the primary tool used by healthcare workers (HCWs) to record clinical information and its completeness can help achieve safer practices. CR is the most appropriate source in order to measure and evaluate the quality of care. In order to achieve a safety climate is fundamental to involve a responsive healthcare workforce thorough peer-review and feedbacks. This study aims to develop a peer-review tool for clinical records quality assurance, presenting the seven-year experience in the evolution of it; secondary aims are to describe the CR completeness and HCWs' diligence toward recording information in it.

METHODS

To assess the completeness of CRs a peer-review tool was developed in a large Academic Hospital of Northern Italy. This tool included measurable items that examined different themes, moments and levels of the clinical process. Data were collected every three months between 2010 and 2016 by appointed and trained HCWs from 42 Units; the hospital Quality Unit was responsible for of processing and validating them. Variations in the proportion of CR completeness were assessed using Cochran-Armitage test for trends.

RESULTS

A total of 9,408 CRs were evaluated. Overall CR completeness improved significantly from 79.6% in 2010 to 86.5% in 2016 (p<0.001). Doctors' attitude showed a trend similar to the overall completeness, while nurses improved more consistently (p<0.001). Most items exploring themes, moments and levels registered a significant improvement in the early years, then flattened in last years. Results of the validation process were always above the cut-off of 75%.

CONCLUSIONS

This peer-review tool enabled the Quality Unit and hospital leadership to obtain a reliable picture of CRs completeness, while involving the HCWs in the quality evaluation. The completeness of CR showed an overall positive and significant trend during these seven years.

Identification of predictors and model for predicting prolonged length of stay in dengue patients

PURPOSE

Our objective is to identify the predictive factors and predict hospital length of stay (LOS) in dengue patients, for efficient utilization of hospital resources.

METHODS

We collected 1360 medical patient records of confirmed dengue infection from 2012 to 2017 at Max group of hospitals in India. We applied two different data mining algorithms, logistic regression (LR) with elastic-net, and random forest to extract predictive factors and predict the LOS. We used an area under the curve (AUC), sensitivity, and specificity to evaluate the performance of the classifiers.

RESULTS

The classifiers performed well, with logistic regression (LR) with elastic-net providing an AUC score of 0.75 and random forest providing a score of 0.72. Out of 1148 patients, 364 (32%) patients had prolonged length of stay (LOS) (> 5 days) and overall hospitalization mean was 4.03 ± 2.44 days (median ± IQR). The highest number of dengue cases belonged to the age group of 10-20 years (21.1%) with a male predominance. Moreover, the study showed that blood transfusion, emergency admission, assisted ventilation, low haemoglobin, high total leucocyte count (TLC), low or high haematocrit, and low lymphocytes have a significant correlation with prolonged LOS.

CONCLUSION

Our findings demonstrated that the logistic regression with elastic-net was the best fit with an AUC of 0.75 and there is a significant association between LOS greater than five days and identified patient-specific variables. This method can identify the patients at highest risks and help focus time and resources.

Expert Consensus on Currently Accepted Measures of Harm

BACKGROUND

Twenty-five years after the seminal work of the Harvard Medical Practice Study, the numbers and specific types of health care measures of harm have evolved and expanded. Using the World Café method to derive expert consensus, we sought to generate a contemporary list of triggers and adverse event measures that could be used for chart review to determine the current incidence of inpatient and outpatient adverse events.

METHODS

We held a modified World Café event in March 2018, during which content experts were divided into 10 tables by clinical domain. After a focused discussion of a prepopulated list of literature-based triggers and measures relevant to that domain, they were asked to rate each measure on clinical importance and suitability for chart review and electronic extraction (very low, low, medium, high, very high).

RESULTS

Seventy-one experts from 9 diverse institutions attended (primary acceptance rate, 72%). Of 525 total triggers and measures, 67% of 391 measures and 46% of 134 triggers were deemed to have high or very high clinical importance. For those triggers and measures with high or very high clinical importance, 218 overall were deemed to be highly amenable to chart review and 198 overall were deemed to be suitable for electronic surveillance.

CONCLUSIONS

The World Café method effectively prioritized measures/triggers of high clinical importance including those that can be used in chart review, which is considered the gold standard. A future goal is to validate these measures using electronic surveillance mechanisms to decrease the need for chart review.

Association of chiropractic integration in an Ontario community health centre with prescription opioid use for chronic non-cancer pain: a mixed methods study protocol

INTRODUCTION

Emerging evidence from a number of primary care centres suggests that integration of chiropractic services into chronic pain management is associated with improved clinical outcomes and high patient satisfaction as well as with reductions in physician visits, specialist referrals use of advanced imaging and prescribing of analgesics. However, formal assessments of the integration of chiropractic services into primary care settings are sparse, and the impact of such integration on prescription opioid use in chronic pain management remains uncertain. To help address this knowledge gap, we will conduct a mixed methods health service evaluation of an integrated chiropractic back pain programme in an urban community health centre in Ontario, Canada. This centre provides services to vulnerable populations with high unemployment rates, multiple comorbidities and musculoskeletal disorders that are commonly managed with prescription opioids.

METHODS AND ANALYSIS

We will use a sequential explanatory mixed methods design, which consists of a quantitative phase followed by a qualitative phase. In the quantitative phase, we will conduct a retrospective chart review and evaluate whether receipt of chiropractic services is associated with reduced opioid use among patients already prescribed opioid therapy for chronic pain. We will measure opioid prescriptions (ie, opioid fills, number of refills and dosages) by reviewing electronic medical records of recipients and non-recipients of chiropractic services between 1 January 2014 and 31 December 2020 and use multivariable regression analysis to examine the association. In the qualitative phase, we will conduct in-depth, one-on-one interviews of patients and their general practitioners to explore perceptions of chiropractic integration and its impact on opioid use.

ETHICS AND DISSEMINATION

This study was approved by the Hamilton Integrated Research Ethics Board at McMaster University (approval number 2021-10930). The results will be disseminated via peer-reviewed publications, conference presentations and in-person or webinar presentations to community members and healthcare professionals.

Incision Location Predicts 30-Day Major Adverse Events after Cosmetic Breast Augmentation: An Analysis of the Tracking Outcomes and Operations for Plastic Surgeons Database

BACKGROUND

Enhanced understanding of early postoperative adverse events will improve patient counseling and preoperative risk modification to decrease complications in implant-based breast augmentation. This study seeks to evaluate the early major adverse events following cosmetic breast augmentation.

METHODS

A retrospective cohort analysis of the Tracking Outcomes and Operations for Plastic Surgeons database was performed to identify any women undergoing augmentation mammaplasty with an implant between 2008 and 2016.

RESULTS

A total of 84,296 patients were studied. Major adverse events were identified in 0.37 percent. Seroma requiring drainage was observed in 0.08 percent, hematoma requiring drainage was observed in 0.15 percent, deep wound disruption was observed in 0.09 percent, and implant loss was observed in 0.11 percent. The authors identified multiple independent predictors of major adverse events, including body mass index greater than 30 kg/m2 (relative risk, 2.05; p < 0.001), tobacco use (relative risk, 2.25; p < 0.001), and diabetes mellitus (relative risk, 1.8; p < 0.05). Use of a periareolar incision significantly increased the risk of developing an early postoperative complication (relative risk, 1.77; p < 0.001).

CONCLUSIONS

The findings of this study indicate an early major adverse event rate following cosmetic breast augmentation with implants of 0.37 percent. The authors identified multiple independent predictors of major adverse events, including body mass index greater than 30 kg/m2, tobacco use, and diabetes mellitus. In addition, when controlling for other factors, periareolar incision significantly increased the risk for major adverse events, when compared to an inframammary incision.

CLINICAL QUESTION/LEVEL OF EVIDENCE

Therapeutic, III.

Assessing the change in disease severity based on depressive symptoms in real-world psoriasis patients

Aim: To evaluate whether the presence of a history of depression hinders psoriasis response to systemic therapies and to delineate baseline characteristics of patients whose depressive symptoms improved on systemic treatment. Methods: We studied patients within the Corrona^® Psoriasis Registry, a prospective, multicenter observational disease-based registry, that were enrolled through September 2018, comparing changes from enrollment to 12-month visit. Results: There was a statistically significant improvement in all disease characteristics and most patient-reported outcomes in patients reporting a history of depression and in those that did not while there was no statistically significant difference in the degree of change comparing these two cohorts. Patients who noted improvement in depressive symptoms had more severe baseline disease characteristics and reported overall worse baseline patient-reported outcomes. Conclusions: History of depression does not portend a differential response to systemic treatment. Patients with improvement in depressive symptoms had worse baseline characteristics.