Early Identification of Childhood Asthma: The Role of Informatics in an Era of Electronic Health Records

Tracking persistent postoperative opioid use: a proof-of-concept study demonstrating a use case for natural language processing

BACKGROUND

Large language models have been gaining tremendous popularity since the introduction of ChatGPT in late 2022. Perioperative pain providers should leverage natural language processing (NLP) technology and explore pertinent use cases to improve patient care. One example is tracking persistent postoperative opioid use after surgery. Since much of the relevant data may be 'hidden' within unstructured clinical text, NLP models may prove to be advantageous. The primary objective of this proof-of-concept study was to demonstrate the ability of an NLP engine to review clinical notes and accurately identify patients who had persistent postoperative opioid use after major spine surgery.

METHODS

Clinical documents from all patients that underwent major spine surgery during July 2015-August 2021 were extracted from the electronic health record. The primary outcome was persistent postoperative opioid use, defined as continued use of opioids greater than or equal to 3 months after surgery. This outcome was ascertained via manual clinician review from outpatient spine surgery follow-up notes. An NLP engine was applied to these notes to ascertain the presence of persistent opioid use-this was then compared with results from clinician manual review.

RESULTS

The final study sample consisted of 965 patients, in which 705 (73.1%) were determined to have persistent opioid use following surgery. The NLP engine correctly determined the patients' opioid use status in 92.9% of cases, in which it correctly identified persistent opioid use in 95.6% of cases and no persistent opioid use in 86.1% of cases.

DISCUSSION

Access to unstructured data within the perioperative history can contextualize patients' opioid use and provide further insight into the opioid crisis, while at the same time improve care directly at the patient level. While these goals are in reach, future work is needed to evaluate how to best implement NLP within different healthcare systems for use in clinical decision support.

Ascertaining asthma status in epidemiologic studies: a comparison between administrative health data and self-report

BACKGROUND

Studies have suggested that agreement between administrative health data and self-report for asthma status ranges from fair to good, but few studies benefited from administrative health data over a long period. We aimed to (1) evaluate agreement between asthma status ascertained in administrative health data covering a period of 30 years and from self-report, and (2) identify determinants of agreement between the two sources.

METHODS

We used administrative health data (1983-2012) from the Quebec Birth Cohort on Immunity and Health, which included 81,496 individuals born in the province of Quebec, Canada, in 1974. Additional information, including self-reported asthma, was collected by telephone interview with 1643 participants in 2012. By design, half of them had childhood asthma based on health services utilization. Results were weighted according to the inverse of the sampling probabilities. Five algorithms were applied to administrative health data (having ≥ 2 physician claims over a 1-, 2-, 3-, 5-, or 30-year interval or ≥ 1 hospitalization), to enable comparisons with previous studies. We estimated the proportion of overall agreement and Kappa, between asthma status derived from algorithms and self-reports. We used logistic regression to identify factors associated with agreement.

RESULTS

Applying the five algorithms, the prevalence of asthma ranged from 49 to 55% among the 1643 participants. At interview (mean age = 37 years), 49% and 47% of participants respectively reported ever having asthma and asthma diagnosed by a physician. Proportions of agreement between administrative health data and self-report ranged from 88 to 91%, with Kappas ranging from 0.57 (95% CI: 0.52-0.63) to 0.67 (95% CI: 0.62-0.72); the highest values were obtained with the [≥ 2 physician claims over a 30-year interval or ≥ 1 hospitalization] algorithm. Having sought health services for allergic diseases other than asthma was related to lower agreement (Odds ratio = 0.41; 95% CI: 0.25-0.65 comparing ≥ 1 health services to none).

CONCLUSIONS

These findings indicate good agreement between asthma status defined from administrative health data and self-report. Agreement was higher than previously observed, which may be due to the 30-year lookback window in administrative data. Our findings support using both administrative health data and self-report in population-based epidemiological studies.

Validation of Diagnostic Coding for Asthma in an Electronic Health Record System in Hong Kong

Background

Electronic health record (EHR) databases can facilitate epidemiology research into various diseases including asthma. Given the diagnostic challenges of asthma, the validity of the coding in EHR requires clarification. We aimed to assess the validity of International Classification of Diseases, 9th Revision (ICD-9) code algorithms for identifying asthma in the territory-wide electronic medical health record system of the Clinical Data Analysis and Reporting System (CDARS) in Hong Kong.

Methods

Adult patients who had the diagnosis of asthma input from all public hospitals in Hong Kong and those from Queen Mary Hospital in 2011-2020 were identified using the ICD-9 code of 493 (493.0, 493.1, 493.2, and 493.9) by CDARS. Patients' clinical record and spirometry were reviewed by two respiratory specialists to confirm the presence of asthma in the randomly selected cases.

Results

There were 43,454 patients who had the diagnostic code of asthma among all public hospitals in Hong Kong and 1852 in Queen Mary Hospital in the same period. A total of 200 cases were randomly selected and validated using medical record and spirometry review by a respiratory specialist. The overall positive predictive value (PPV) was 85.0% (95% CI 80.1-89.9%).

Conclusion

This was the first ICD-9 code validation for CDARS (EHR) in Hong Kong on asthma. Our study demonstrated that using ICD-9 code (493.0, 493.1, 493.2 and 493.9) to identify asthma can result in a PPV that was reliable to support the utility of the CDARS database for further research on asthma among the Hong Kong population.

RoBERTa-Assisted Outcome Prediction in Ovarian Cancer Cytoreductive Surgery Using Operative Notes

INTRODUCTION

Contemporary efforts to predict surgical outcomes focus on the associations between traditional discrete surgical risk factors. We aimed to determine whether natural language processing (NLP) of unstructured operative notes improves the prediction of residual disease in women with advanced epithelial ovarian cancer (EOC) following cytoreductive surgery.

METHODS

Electronic Health Records were queried to identify women with advanced EOC including their operative notes. The Term Frequency - Inverse Document Frequency (TF-IDF) score was used to quantify the discrimination capacity of sequences of words (n-grams) regarding the existence of residual disease. We employed the state-of-the-art RoBERTa-based classifier to process unstructured surgical notes. Discrimination was measured using standard performance metrics. An XGBoost model was then trained on the same dataset using both discrete and engineered clinical features along with the probabilities outputted by the RoBERTa classifier.

RESULTS

The cohort consisted of 555 cases of EOC cytoreduction performed by eight surgeons between January 2014 and December 2019. Discrete word clouds weighted by n-gram TF-IDF score difference between R0 and non-R0 resection were identified. The words 'adherent' and 'miliary disease' best discriminated between the two groups. The RoBERTa model reached high evaluation metrics (AUROC .86; AUPRC .87, precision, recall, and F1 score of .77 and accuracy of .81). Equally, it outperformed models that used discrete clinical and engineered features and outplayed the performance of other state-of-the-art NLP tools. When the probabilities from the RoBERTa classifier were combined with commonly used predictors in the XGBoost model, a marginal improvement in the overall model's performance was observed (AUROC and AUPRC of .91, with all other metrics the same).

CONCLUSION/IMPLICATIONS

We applied a sui generis approach to extract information from the abundant textual surgical data and demonstrated how it can be effectively used for classification prediction, outperforming models relying on conventional structured data. State-of-art NLP applications in biomedical texts can improve modern EOC care.

A Technical Performance Study and Proposed Systematic and Comprehensive Evaluation of an ML-based CDS Solution for Pediatric Asthma

Achieving optimal care for pediatric asthma patients depends on giving clinicians efficient access to pertinent patient information. Unfortunately, adherence to guidelines or best practices has shown to be challenging, as relevant information is often scattered throughout the patient record in both structured data and unstructured clinical notes. Furthermore, in the absence of supporting tools, the onus of consolidating this information generally falls upon the clinician. In this study, we propose a machine learning-based clinical decision support (CDS) system focused on pediatric asthma care to alleviate some of this burden. This framework aims to incorporate a machine learning model capable of predicting asthma exacerbation risk into the clinical workflow, emphasizing contextual data, supporting information, and model transparency and explainability. We show that this asthma exacerbation model is capable of predicting exacerbation with an 0.8 AUC-ROC. This model, paired with a comprehensive informatics-based process centered on clinical usability, emphasizes our focus on meeting the needs of the clinical practice with machine learning technology.

Risk, Mechanisms and Implications of Asthma-Associated Infectious and Inflammatory Multimorbidities (AIMs) among Individuals With Asthma: a Systematic Review and a Case Study

Our prior work and the work of others have demonstrated that asthma increases the risk of a broad range of both respiratory (e.g., pneumonia and pertussis) and non-respiratory (e.g., zoster and appendicitis) infectious diseases as well as inflammatory diseases (e.g., celiac disease and myocardial infarction [MI]), suggesting the systemic disease nature of asthma and its impact beyond the airways. We call these conditions asthma-associated infectious and inflammatory multimorbidities (AIMs). At present, little is known about why some people with asthma are at high-risk of AIMs, and others are not, to the extent to which controlling asthma reduces the risk of AIMs and which specific therapies mitigate the risk of AIMs. These questions represent a significant knowledge gap in asthma research and unmet needs in asthma care, because there are no guidelines addressing the identification and management of AIMs. This is a systematic review on the association of asthma with the risk of AIMs and a case study to highlight that 1) AIMs are relatively under-recognized conditions, but pose major health threats to people with asthma; 2) AIMs provide insights into immunological and clinical features of asthma as a systemic inflammatory disease beyond a solely chronic airway disease; and 3) it is time to recognize AIMs as a distinctive asthma phenotype in order to advance asthma research and improve asthma care. An improved understanding of AIMs and their underlying mechanisms will bring valuable and new perspectives improving the practice, research, and public health related to asthma.

Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial

RATIONALE

Clinical decision support (CDS) tools leveraging electronic health records (EHRs) have been an approach for addressing challenges in asthma care but remain under-studied through clinical trials.

OBJECTIVES

To assess the effectiveness and efficiency of Asthma-Guidance and Prediction System (A-GPS), an Artificial Intelligence (AI)-assisted CDS tool, in optimizing asthma management through a randomized clinical trial (RCT).

METHODS

This was a single-center pragmatic RCT with a stratified randomization design conducted for one year in the primary care pediatric practice of the Mayo Clinic, MN. Children (<18 years) diagnosed with asthma receiving care at the study site were enrolled along with their 42 primary care providers. Study subjects were stratified into three strata (based on asthma severity, asthma care status, and asthma diagnosis) and were blinded to the assigned groups.

MEASUREMENTS

Intervention was a quarterly A-GPS report to clinicians including relevant clinical information for asthma management from EHRs and machine learning-based prediction for risk of asthma exacerbation (AE). Primary endpoint was the occurrence of AE within 1 year and secondary outcomes included time required for clinicians to review EHRs for asthma management.

MAIN RESULTS

Out of 555 participants invited to the study, 184 consented for the study and were randomized (90 in intervention and 94 in control group). Median age of 184 participants was 8.5 years. While the proportion of children with AE in both groups decreased from the baseline (P = 0.042), there was no difference in AE frequency between the two groups (12% for the intervention group vs. 15% for the control group, Odds Ratio: 0.82; 95%CI 0.374-1.96; P = 0.626) during the study period. For the secondary end points, A-GPS intervention, however, significantly reduced time for reviewing EHRs for asthma management of each participant (median: 3.5 min, IQR: 2-5), compared to usual care without A-GPS (median: 11.3 min, IQR: 6.3-15); p<0.001). Mean health care costs with 95%CI of children during the trial (compared to before the trial) in the intervention group were lower than those in the control group (-$1,036 [-$2177, $44] for the intervention group vs. +$80 [-$841, $1000] for the control group), though there was no significant difference (p = 0.12). Among those who experienced the first AE during the study period (n = 25), those in the intervention group had timelier follow up by the clinical care team compared to those in the control group but no significant difference was found (HR = 1.93; 95% CI: 0.82-1.45, P = 0.10). There was no difference in the proportion of duration when patients had well-controlled asthma during the study period between the intervention and the control groups.

CONCLUSIONS

While A-GPS-based intervention showed similar reduction in AE events to usual care, it might reduce clinicians' burden for EHRs review resulting in efficient asthma management. A larger RCT is needed for further studying the findings.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02865967.

Artificial intelligence approaches using natural language processing to advance EHR-based clinical research

The wide adoption of electronic health record systems in health care generates big real-world data that open new venues to conduct clinical research. As a large amount of valuable clinical information is locked in clinical narratives, natural language processing techniques as an artificial intelligence approach have been leveraged to extract information from clinical narratives in electronic health records. This capability of natural language processing potentially enables automated chart review for identifying patients with distinctive clinical characteristics in clinical care and reduces methodological heterogeneity in defining phenotype, obscuring biological heterogeneity in research concerning allergy, asthma, and immunology. This brief review discusses the current literature on the secondary use of electronic health record data for clinical research concerning allergy, asthma, and immunology and highlights the potential, challenges, and implications of natural language processing techniques.