Adapting Bidirectional Encoder Representations from Transformers (BERT) to Assess Clinical Semantic Textual Similarity: Algorithm Development and Validation Study

Development and validation of HBV surveillance models using big data and machine learning

BACKGROUND

The construction of a robust healthcare information system is fundamental to enhancing countries' capabilities in the surveillance and control of hepatitis B virus (HBV). Making use of China's rapidly expanding primary healthcare system, this innovative approach using big data and machine learning (ML) could help towards the World Health Organization's (WHO) HBV infection elimination goals of reaching 90% diagnosis and treatment rates by 2030. We aimed to develop and validate HBV detection models using routine clinical data to improve the detection of HBV and support the development of effective interventions to mitigate the impact of this disease in China.

METHODS

Relevant data records extracted from the Family Medicine Clinic of the University of Hong Kong-Shenzhen Hospital's Hospital Information System were structuralized using state-of-the-art Natural Language Processing techniques. Several ML models have been used to develop HBV risk assessment models. The performance of the ML model was then interpreted using the Shapley value (SHAP) and validated using cohort data randomly divided at a ratio of 2:1 using a five-fold cross-validation framework.

RESULTS

The patterns of physical complaints of patients with and without HBV infection were identified by processing 158,988 clinic attendance records. After removing cases without any clinical parameters from the derivation sample (n = 105,992), 27,392 cases were analysed using six modelling methods. A simplified model for HBV using patients' physical complaints and parameters was developed with good discrimination (AUC = 0.78) and calibration (goodness of fit test p-value >0.05).

CONCLUSIONS

Suspected case detection models of HBV, showing potential for clinical deployment, have been developed to improve HBV surveillance in primary care setting in China. (Word count: 264).

Predicting prognostic factors in kidney transplantation using a machine learning approach to enhance outcome predictions: a retrospective cohort study

BACKGROUND

Accurate forecasting of clinical outcomes after kidney transplantation is essential for improving patient care and increasing the success rates of transplants. The authors' study employs advanced machine learning (ML) algorithms to identify crucial prognostic indicators for kidney transplantation. By analyzing complex datasets with ML models, the authors aim to enhance prediction accuracy and provide valuable insights to support clinical decision-making.

MATERIALS AND METHODS

Analyzing data from 4077 KT patients (June 1990-May 2015) at a single center, this research included 27 features encompassing recipient/donor traits and peri-transplant data. The dataset was divided into training (80%) and testing (20%) sets. Four ML models-eXtreme Gradient Boosting (XGBoost), Feedforward Neural Network, Logistic Regression, And Support Vector Machine-were trained on carefully selected features to predict the success of graft survival. Performance was assessed by precision, sensitivity, F1 score, area under the receiver operating characteristic (AUROC), and area under the precision-recall curve.

RESULTS

XGBoost emerged as the best model, with an AUROC of 0.828, identifying key survival predictors like T-cell flow crossmatch positivity, creatinine levels two years post-transplant and human leukocyte antigen mismatch. The study also examined the prognostic importance of histological features identified by the Banff criteria for renal biopsy, emphasizing the significance of intimal arteritis, interstitial inflammation, and chronic glomerulopathy.

CONCLUSION

The study developed ML models that pinpoint clinical factors crucial for KT graft survival, aiding clinicians in making informed post-transplant care decisions. Incorporating these findings with the Banff classification could improve renal pathology diagnosis and treatment, offering a data-driven approach to prioritizing pathology scores.

Language model and its interpretability in biomedicine: A scoping review

With advancements in large language models, artificial intelligence (AI) is undergoing a paradigm shift where AI models can be repurposed with minimal effort across various downstream tasks. This provides great promise in learning generally useful representations from biomedical corpora, at scale, which would empower AI solutions in healthcare and biomedical research. Nonetheless, our understanding of how they work, when they fail, and what they are capable of remains underexplored due to their emergent properties. Consequently, there is a need to comprehensively examine the use of language models in biomedicine. This review aims to summarize existing studies of language models in biomedicine and identify topics ripe for future research, along with the technical and analytical challenges w.r.t. interpretability. We expect this review to help researchers and practitioners better understand the landscape of language models in biomedicine and what methods are available to enhance the interpretability of their models.

RECAP-KG: Mining Knowledge Graphs from Raw Primary Care Physician Notes for Remote COVID-19 Assessment in Primary Care

Building Clinical Decision Support Systems, whether from regression models or machine learning requires clinical data either in standard terminology or as text for Natural Language Processing (NLP). Unfortunately, many clinical notes are written quickly during the consultation and contain many abbreviations, typographical errors, and a lack of grammar and punctuation Processing these highly unstructured clinical notes is an open challenge for NLP that we address in this paper. We present RECAP-KG - a knowledge graph construction frame workfrom primary care clinical notes. Our framework extracts structured knowledge graphs from the clinical record by utilising the SNOMED-CT ontology both the entire finding hierarchy and a COVID-relevant curated subset. We apply our framework to consultation notes in the UK COVID-19 Clinical Assessment Service (CCAS) dataset and provide a quantitative evaluation of our framework demonstrating that our approach has better accuracy than traditional NLP methods when answering questions about patients.

Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives

Importance

Despite the unmet need, many deceased-donor kidneys are discarded or not recovered. Inefficient allocation and prolonged ischemia time are contributing factors, and early detection of high-risk donors may reduce organ loss.

Objective

To evaluate the feasibility of machine learning (ML) and natural language processing (NLP) classification of donors with kidneys that are used vs not used for organ transplant.

Design, Setting, and Participants

This retrospective cohort study used donor information (structured donor characteristics and unstructured donor narratives) from the United Network for Organ Sharing (UNOS). All donor offers to a single transplant center between January 2015 and December 2020 were used to train and validate ML models to predict donors who had at least 1 kidney transplanted (at our center or another center). The donor data from 2021 were used to test each model.

Exposures

Donor information was provided by UNOS to the transplant centers with potential transplant candidates. Each center evaluated the donor and decided within an allotted time whether to accept the kidney for organ transplant.

Main Outcomes and Measures

Outcome metrics of the test cohort included area under the receiver operating characteristic curve (AUROC), F1 score, accuracy, precision, and recall of each ML classifier. Feature importance and Shapley additive explanation (SHAP) summaries were assessed for model explainability.

Results

The training/validation cohort included 9555 donors (median [IQR] age, 50 [36-58] years; 5571 male [58.3%]), and the test cohort included 2481 donors (median [IQR] age, 52 [40-59] years; 1496 male [60.3%]). Only 20% to 30% of potential donors had at least 1 kidney transplanted. The ML model with a single variable (Kidney Donor Profile Index) showed an AUROC of 0.69, F1 score of 0.42, and accuracy of 0.64. Multivariable ML models based on basic a priori structured donor data showed similar metrics (logistic regression: AUROC = 0.70; F1 score = 0.42; accuracy = 0.62; random forest classifier: AUROC = 0.69; F1 score = 0.42; accuracy = 0.64). The classic NLP model (bag-of-words model) showed its best metrics (AUROC = 0.60; F1 score = 0.35; accuracy = 0.59) by the logistic regression classifier. The advanced Bidirectional Encoder Representations From Transformers model showed comparable metrics (AUROC = 0.62; F1 score = 0.39; accuracy = 0.69) only after appending basic donor information. Feature importance and SHAP detected the variables (and words) that affected the models most.

Conclusions and Relevance

Results of this cohort study suggest that models using ML can be applied to predict donors with high-risk kidneys not used for organ transplant, but the models still need further elaboration. The use of unstructured data is likely to expand the possibilities; further exploration of new approaches will be necessary to develop models with better predictive metrics.

The Utility of Language Models in Cardiology: A Narrative Review of the Benefits and Concerns of ChatGPT-4

Artificial intelligence (AI) and language models such as ChatGPT-4 (Generative Pretrained Transformer) have made tremendous advances recently and are rapidly transforming the landscape of medicine. Cardiology is among many of the specialties that utilize AI with the intention of improving patient care. Generative AI, with the use of its advanced machine learning algorithms, has the potential to diagnose heart disease and recommend management options suitable for the patient. This may lead to improved patient outcomes not only by recommending the best treatment plan but also by increasing physician efficiency. Language models could assist physicians with administrative tasks, allowing them to spend more time on patient care. However, there are several concerns with the use of AI and language models in the field of medicine. These technologies may not be the most up-to-date with the latest research and could provide outdated information, which may lead to an adverse event. Secondly, AI tools can be expensive, leading to increased healthcare costs and reduced accessibility to the general population. There is also concern about the loss of the human touch and empathy as AI becomes more mainstream. Healthcare professionals would need to be adequately trained to utilize these tools. While AI and language models have many beneficial traits, all healthcare providers need to be involved and aware of generative AI so as to assure its optimal use and mitigate any potential risks and challenges associated with its implementation. In this review, we discuss the various uses of language models in the field of cardiology.

Users' Reactions to Announced Vaccines Against COVID-19 Before Marketing in France: Analysis of Twitter Posts

BACKGROUND

Within a few months, the COVID-19 pandemic had spread to many countries and had been a real challenge for health systems all around the world. This unprecedented crisis has led to a surge of online discussions about potential cures for the disease. Among them, vaccines have been at the heart of the debates and have faced lack of confidence before marketing in France.

OBJECTIVE

This study aims to identify and investigate the opinions of French Twitter users on the announced vaccines against COVID-19 through sentiment analysis.

METHODS

This study was conducted in 2 phases. First, we filtered a collection of tweets related to COVID-19 available on Twitter from February 2020 to August 2020 with a set of keywords associated with vaccine mistrust using word embeddings. Second, we performed sentiment analysis using deep learning to identify the characteristics of vaccine mistrust. The model was trained on a hand-labeled subset of 4548 tweets.

RESULTS

A set of 69 relevant keywords were identified as the semantic concept of the word "vaccin" (vaccine in French) and focused mainly on conspiracies, pharmaceutical companies, and alternative treatments. Those keywords enabled us to extract nearly 350,000 tweets in French. The sentiment analysis model achieved 0.75 accuracy. The model then predicted 16% of positive tweets, 41% of negative tweets, and 43% of neutral tweets. This allowed us to explore the semantic concepts of positive and negative tweets and to plot the trends of each sentiment. The main negative rhetoric identified from users' tweets was that vaccines are perceived as having a political purpose and that COVID-19 is a commercial argument for the pharmaceutical companies.

CONCLUSIONS

Twitter might be a useful tool to investigate the arguments for vaccine mistrust because it unveils political criticism contrasting with the usual concerns on adverse drug reactions. As the opposition rhetoric is more consistent and more widely spread than the positive rhetoric, we believe that this research provides effective tools to help health authorities better characterize the risk of vaccine mistrust.

Predicting Postoperative Mortality With Deep Neural Networks and Natural Language Processing: Model Development and Validation

Background

Machine learning (ML) achieves better predictions of postoperative mortality than previous prediction tools. Free-text descriptions of the preoperative diagnosis and the planned procedure are available preoperatively. Because reading these descriptions helps anesthesiologists evaluate the risk of the surgery, we hypothesized that deep learning (DL) models with unstructured text could improve postoperative mortality prediction. However, it is challenging to extract meaningful concept embeddings from this unstructured clinical text.

Objective

This study aims to develop a fusion DL model containing structured and unstructured features to predict the in-hospital 30-day postoperative mortality before surgery. ML models for predicting postoperative mortality using preoperative data with or without free clinical text were assessed.

Methods

We retrospectively collected preoperative anesthesia assessments, surgical information, and discharge summaries of patients undergoing general and neuraxial anesthesia from electronic health records (EHRs) from 2016 to 2020. We first compared the deep neural network (DNN) with other models using the same input features to demonstrate effectiveness. Then, we combined the DNN model with bidirectional encoder representations from transformers (BERT) to extract information from clinical texts. The effects of adding text information on the model performance were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Statistical significance was evaluated using P<.05.

Results

The final cohort contained 121,313 patients who underwent surgeries. A total of 1562 (1.29%) patients died within 30 days of surgery. Our BERT-DNN model achieved the highest AUROC (0.964, 95% CI 0.961-0.967) and AUPRC (0.336, 95% CI 0.276-0.402). The AUROC of the BERT-DNN was significantly higher compared to logistic regression (AUROC=0.952, 95% CI 0.949-0.955) and the American Society of Anesthesiologist Physical Status (ASAPS AUROC=0.892, 95% CI 0.887-0.896) but not significantly higher compared to the DNN (AUROC=0.959, 95% CI 0.956-0.962) and the random forest (AUROC=0.961, 95% CI 0.958-0.964). The AUPRC of the BERT-DNN was significantly higher compared to the DNN (AUPRC=0.319, 95% CI 0.260-0.384), the random forest (AUPRC=0.296, 95% CI 0.239-0.360), logistic regression (AUPRC=0.276, 95% CI 0.220-0.339), and the ASAPS (AUPRC=0.149, 95% CI 0.107-0.203).

Conclusions

Our BERT-DNN model has an AUPRC significantly higher compared to previously proposed models using no text and an AUROC significantly higher compared to logistic regression and the ASAPS. This technique helps identify patients with higher risk from the surgical description text in EHRs.