Natural language processing for the surveillance of postoperative venous thromboembolism

Machine learning in cancer-associated thrombosis: hype or hope in untangling the clot

The goal of machine learning (ML) is to create informative signals and useful tasks by leveraging large datasets to derive computational algorithms. ML has the potential to revolutionize the healthcare industry by boosting productivity, enhancing safe and effective patient care, and lightening the load on clinicians. In addition to gaining mechanistic insights into cancer-associated thrombosis (CAT), ML can be used to improve patient outcomes, streamline healthcare delivery, and spur innovation. Our review paper delves into the present and potential applications of this cutting-edge technology, encompassing three areas: i) computer vision-assisted diagnosis of thromboembolism from radiology data; ii) case detection from electronic health records using natural language processing; iii) algorithms for CAT prediction and risk stratification. The availability of large, well-annotated, high-quality datasets, overfitting, limited generalizability, the risk of propagating inherent bias, and a lack of transparency among patients and clinicians are among the challenges that must be overcome in order to effectively develop ML in the health sector. To guarantee that this powerful instrument can be utilized to maximize innovation in CAT, clinicians can collaborate with stakeholders such as computer scientists, regulatory bodies, and patient groups.

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

Background

There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date.

Objective

We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey.

Methods

Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis.

Results

The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve > 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI.

Conclusion

Mining EHR notes with NLP holds promise for improving early IEI patient detection.

Case Sampling for Evaluating Hospital Postoperative Morbidity in US Surgical Quality Improvement Programs

Importance

US surgical quality improvement (QI) programs use data from a systematic sample of surgical cases, rather than universal review of all cases, to assess and compare risk-adjusted hospital postoperative complication rates. Given decreasing postoperative complication rates over time and the types of cases eligible for abstraction, it is unclear whether case sampling is robust for identifying hospitals with higher than expected complications.

Objective

To compare the assessment of hospital 30-day complication rates derived from sampling strategy used by some US surgical QI programs relative to universal review of all cases.

Design, Setting, and Participants

This US hospital-level analysis took place from January 1, 2016, through September 30, 2020. Data analysis was performed from July 1, 2022, through December 21, 2022. Quarterly, risk-adjusted, 30-day complication observed to expected (O-E) ratios were calculated for each hospital using the sample (n = 502 730) and universal review (n = 1 725 364). Outlier hospitals (ie, those with higher than expected mortality) were identified using an O-E ratio significantly greater than 1.0. Patients 18 years and older who underwent a noncardiac operation at US Department of Veterans Affairs (VA) hospitals with a record in the VA Surgical Quality Improvement Program (systematic sample) and the VA Corporate Data Warehouse surgical domain (100% of surgical cases) were included.

Main Outcome Measure

Thirty-day complications.

Results

Most patients in both the representative sample and the universal sample were men (90.2% vs 91.2%) and White (74.7% vs 74.5%). Overall, 30-day complication rates were 7.6% and 5.3% for the sample and universal review cohorts, respectively (P < .001). Over 2145 hospital quarters of data, hospitals were identified as an outlier in 15.0% of quarters using the sample and 18.2% with universal review. Average hospital quarterly complication rates were 4.7%, 7.2%, and 7.4% for outliers identified using the sample only, universal review only, and concurrent identification in both data sources, respectively. For nonsampled cases, average hospital quarterly complication rates were 7.0% at outliers and 4.4% at nonoutliers. Among outlier hospital quarters in the sample, 54.2% were concurrently identified with universal review. For those identified with universal review, 44.6% were concurrently identified using the sample.

Conclusion

In this observational study, case sampling identified less than half of hospitals with excess risk-adjusted postoperative complication rates. Future work is needed to ascertain how to best use currently collected data and whether alternative data collection strategies may be needed to better inform local QI efforts.

ClotCatcher: a novel natural language model to accurately adjudicate venous thromboembolism from radiology reports

INTRODUCTION

Accurate identification of venous thromboembolism (VTE) is critical to develop replicable epidemiological studies and rigorous predictions models. Traditionally, VTE studies have relied on international classification of diseases (ICD) codes which are inaccurate - leading to misclassification bias. Here, we developed ClotCatcher, a novel deep learning model that uses natural language processing to detect VTE from radiology reports.

METHODS

Radiology reports to detect VTE were obtained from patients admitted to Emory University Hospital (EUH) and Grady Memorial Hospital (GMH). Data augmentation was performed using the Google PEGASUS paraphraser. This data was then used to fine-tune ClotCatcher, a novel deep learning model. ClotCatcher was validated on both the EUH dataset alone and GMH dataset alone.

RESULTS

The dataset contained 1358 studies from EUH and 915 studies from GMH (n = 2273). The dataset contained 1506 ultrasound studies with 528 (35.1%) studies positive for VTE, and 767 CT studies with 91 (11.9%) positive for VTE. When validated on the EUH dataset, ClotCatcher performed best (AUC = 0.980) when trained on both EUH and GMH dataset without paraphrasing. When validated on the GMH dataset, ClotCatcher performed best (AUC = 0.995) when trained on both EUH and GMH dataset with paraphrasing.

CONCLUSION

ClotCatcher, a novel deep learning model with data augmentation rapidly and accurately adjudicated the presence of VTE from radiology reports. Applying ClotCatcher to large databases would allow for rapid and accurate adjudication of incident VTE. This would reduce misclassification bias and form the foundation for future studies to estimate individual risk for patient to develop incident VTE.

Natural Language Processing in a Clinical Decision Support System for the Identification of Venous Thromboembolism: Algorithm Development and Validation

BACKGROUND

It remains unknown whether capturing data from electronic health records (EHRs) using natural language processing (NLP) can improve venous thromboembolism (VTE) detection in different clinical settings.

OBJECTIVE

The aim of this study was to validate the NLP algorithm in a clinical decision support system for VTE risk assessment and integrated care (DeVTEcare) to identify VTEs from EHRs.

METHODS

All inpatients aged ≥18 years in the Sixth Medical Center of the Chinese People's Liberation Army General Hospital from January 1 to December 31, 2021, were included as the validation cohort. The sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR-, respectively), area under the receiver operating characteristic curve (AUC), and F1-scores along with their 95% CIs were used to analyze the performance of the NLP tool, with manual review of medical records as the reference standard for detecting deep vein thrombosis (DVT) and pulmonary embolism (PE). The primary end point was the performance of the NLP approach embedded into the EHR for VTE identification. The secondary end points were the performances to identify VTE among different hospital departments with different VTE risks. Subgroup analyses were performed among age, sex, and the study season.

RESULTS

Among 30,152 patients (median age 56 [IQR 41-67] years; 14,247/30,152, 47.3% females), the prevalence of VTE, PE, and DVT was 2.1% (626/30,152), 0.6% (177/30,152), and 1.8% (532/30,152), respectively. The sensitivity, specificity, LR+, LR-, AUC, and F1-score of NLP-facilitated VTE detection were 89.9% (95% CI 87.3%-92.2%), 99.8% (95% CI 99.8%-99.9%), 483 (95% CI 370-629), 0.10 (95% CI 0.08-0.13), 0.95 (95% CI 0.94-0.96), and 0.90 (95% CI 0.90-0.91), respectively. Among departments of surgery, internal medicine, and intensive care units, the highest specificity (100% vs 99.7% vs 98.8%, respectively), LR+ (3202 vs 321 vs 77, respectively), and F1-score (0.95 vs 0.89 vs 0.92, respectively) were in the surgery department (all P<.001). Among low, intermediate, and high VTE risks in hospital departments, the low-risk department had the highest AUC (1.00 vs 0.94 vs 0.96, respectively) and F1-score (0.97 vs 0.90 vs 0.90, respectively) as well as the lowest LR- (0.00 vs 0.13 vs 0.08, respectively) (DeLong test for AUC; all P<.001). Subgroup analysis of the age, sex, and season demonstrated consistently good performance of VTE detection with >87% sensitivity and specificity and >89% AUC and F1-score. The NLP algorithm performed better among patients aged ≤65 years than among those aged >65 years (F1-score 0.93 vs 0.89, respectively; P<.001).

CONCLUSIONS

The NLP algorithm in our DeVTEcare identified VTE well across different clinical settings, especially in patients in surgery units, departments with low-risk VTE, and patients aged ≤65 years. This algorithm can help to inform accurate in-hospital VTE rates and enhance risk-classified VTE integrated care in future research.

Practice-Based Learning and Improvement: Improving Morbidity and Mortality Review Using Natural Language Processing

INTRODUCTION

Practice-Based Learning and Improvement, a core competency identified by the Accreditation Council for Graduate Medical Education, carries importance throughout a physician's career. Practice-Based Learning and Improvement is cultivated by a critical review of complications, yet methods to accurately identify complications are inadequate. Machine-learning algorithms show promise in improving identification of complications. We compare a manual-supplemented natural language processing (ms-NLP) methodology against a validated electronic morbidity and mortality (MM) database, the Morbidity and Mortality Adverse Event Reporting System (MARS) to understand the utility of NLP in MM review.

METHODS

The number and severity of complications were compared between MARS and ms-NLP of surgical hospitalization discharge summaries among three academic medical centers. Clavien-Dindo (CD) scores were assigned to cases with identified complications and classified into minor (CD I-II) or major (CD III-IV) harm.

RESULTS

Of 7774 admissions, 987 cases were identified to have 1659 complications by MARS and 1296 by ms-NLP. MARS identified 611 (62%) cases, whereas ms-NLP identified 670 (68%) cases. Less than one-third of cases (299, 30.3%) were detected by both methods. MARS identified a greater number of complications with major harm (457, 46.30%) than did ms-NLP (P < 0.0001).

CONCLUSIONS

Both a prospectively maintained MM database and ms-NLP review of discharge summaries fail to identify a significant proportion of postoperative complications and overlap 1/3 of the time. ms-NLP more frequently identifies cases with minor complications, whereas prospective voluntary reporting more frequently identifies major complications. The educational benefit of reporting and analysis of complication data may be supplemented by ms-NLP but not replaced by it at this time.

Automated extraction of incidental adrenal nodules from electronic health records

BACKGROUND

Many adrenal incidentalomas do not undergo appropriate biochemical testing and complete imaging characterization to assess for hormone hypersecretion and malignancy. With the growing availability of clinical narratives in the electronic medical record, automated surveillance using advanced data analytic techniques may represent a promising method to improve management.

METHODS

A data provisioning process using a series of structured query language scripts was used to abstract all chest and abdominal computed tomography and magnetic resonance imaging reports from an academic health care system in 2018. The narratives and impressions were queried for key text relating to the identification of adrenal incidentalomas. Patients with a history of extra-adrenal malignancy undergoing staging or surveillance imaging were excluded. The prevalence and radiographic characteristics were analyzed. Patients with adrenal incidentalomas newly identified in 2018 were assessed for biochemical testing and nodule stability through August 2021.

RESULTS

Of 36,618 patients queried, 8,557 were excluded owing to a history of extra-adrenal malignancy. Data from 447 patients were flagged by the structured query language scripts and electronically abstracted. On internal validation, 307/447 (69%) patients were correctly identified as having adrenal nodules (1.1% overall prevalence). The median patient age was 67 years, and 56% were female. The median nodule size was 1.7 (IQR 1.3-2.5) cm, 9% were bilateral, and 63% were low density (unenhanced Hounsfield units <10). Adrenal carcinoma was identified in 10 patients. In 2018, 121 patients were diagnosed with a new adrenal incidentaloma. Of 32 (27%) patients who had follow-up imaging at a median of 1.9 years, 97% of nodules were stable in size. Biochemical testing was performed in 53 patients (44%), of which 31 (26%) had complete hormonal assessment; 14 (26%) were functional nodules: 7 aldosterone-secreting, 4 cortisol-secreting, and 3 pheochromocytoma.

CONCLUSION

Only one-fourth of patients received appropriate biochemical testing after incidental diagnosis of an adrenal nodule, and most nodules with indeterminate imaging characteristics did not undergo follow-up imaging. Advanced data analytic techniques on electronic imaging reports may aid in the clinical identification and improved management of patients with adrenal incidentalomas.