Sequential Data-Based Patient Similarity Framework for Patient Outcome Prediction: Algorithm Development

A Patient Similarity Network (CHDmap) to Predict Outcomes After Congenital Heart Surgery: Development and Validation Study

Background

Although evidence-based medicine proposes personalized care that considers the best evidence, it still fails to address personal treatment in many real clinical scenarios where the complexity of the situation makes none of the available evidence applicable. "Medicine-based evidence" (MBE), in which big data and machine learning techniques are embraced to derive treatment responses from appropriately matched patients in real-world clinical practice, was proposed. However, many challenges remain in translating this conceptual framework into practice.

Objective

This study aimed to technically translate the MBE conceptual framework into practice and evaluate its performance in providing general decision support services for outcomes after congenital heart disease (CHD) surgery.

Methods

Data from 4774 CHD surgeries were collected. A total of 66 indicators and all diagnoses were extracted from each echocardiographic report using natural language processing technology. Combined with some basic clinical and surgical information, the distances between each patient were measured by a series of calculation formulas. Inspired by structure-mapping theory, the fusion of distances between different dimensions can be modulated by clinical experts. In addition to supporting direct analogical reasoning, a machine learning model can be constructed based on similar patients to provide personalized prediction. A user-operable patient similarity network (PSN) of CHD called CHDmap was proposed and developed to provide general decision support services based on the MBE approach.

Results

Using 256 CHD cases, CHDmap was evaluated on 2 different types of postoperative prognostic prediction tasks: a binary classification task to predict postoperative complications and a multiple classification task to predict mechanical ventilation duration. A simple poll of the k-most similar patients provided by the PSN can achieve better prediction results than the average performance of 3 clinicians. Constructing logistic regression models for prediction using similar patients obtained from the PSN can further improve the performance of the 2 tasks (best area under the receiver operating characteristic curve=0.810 and 0.926, respectively). With the support of CHDmap, clinicians substantially improved their predictive capabilities.

Conclusions

Without individual optimization, CHDmap demonstrates competitive performance compared to clinical experts. In addition, CHDmap has the advantage of enabling clinicians to use their superior cognitive abilities in conjunction with it to make decisions that are sometimes even superior to those made using artificial intelligence models. The MBE approach can be embraced in clinical practice, and its full potential can be realized.

The predictive value of machine learning for mortality risk in patients with acute coronary syndromes: a systematic review and meta-analysis

BACKGROUND

Acute coronary syndromes (ACS) are the leading cause of global death. Optimizing mortality risk prediction and early identification of high-risk patients is essential for developing targeted prevention strategies. Many researchers have built machine learning (ML) models to predict the mortality risk in ACS patients. Our meta-analysis aimed to evaluate the predictive value of various ML models in predicting death in ACS patients at different times.

METHODS

PubMed, Embase, Web of Science, and Cochrane Library were searched systematically from database establishment to March 12, 2022 for studies developing or validating at least one ML predictive model for death in ACS patients. We used PROBAST to assess the risk of bias in the reported predictive models and a random-effects model to assess the pooled C-index and accuracy of these models.

RESULTS

Fifty papers were included, involving 216 ML prediction models, 119 of which were externally validated. The combined C-index of the ML models in the validation cohort predicting the in-hospital mortality, 30-day mortality, 3- or 6-month mortality, and 1 year or above mortality in ACS patients were 0.8633 (95% CI 0.8467-0.8802), 0.8296 (95% CI 0.8134-0.8462), 0.8205 (95% CI 0.7881-0.8541), and 0.8197 (95% CI 0.8042-0.8354), respectively, with the corresponding combined accuracy of 0.8569 (95% CI 0.8411-0.8715), 0.8282 (95% CI 0.7922-0.8591), 0.7303 (95% CI 0.7184-0.7418), and 0.7837 (95% CI 0.7455-0.8175), indicating that the ML models were relatively excellent in predicting ACS mortality at different times. Furthermore, common predictors of death in ML models included age, sex, systolic blood pressure, serum creatinine, Killip class, heart rate, diastolic blood pressure, blood glucose, and hemoglobin.

CONCLUSIONS

The ML models had excellent predictive power for mortality in ACS, and the methodologies may need to be addressed before they can be used in clinical practice.

Year 2022 in Medical Natural Language Processing: Availability of Language Models as a Step in the Democratization of NLP in the Biomedical Area

OBJECTIVES

To analyse the content of publications within the medical Natural Language Processing (NLP) domain in 2022.

METHODS

Automatic and manual preselection of publications to be reviewed, and selection of the best NLP papers of the year. Analysis of the important issues.

RESULTS

Three best papers have been selected. We also propose an analysis of the content of the NLP publications in 2022, stressing on some of the topics.

CONCLUSION

The main trend in 2022 is certainly related to the availability of large language models, especially those based on Transformers, and to their use by non-NLP researchers. This leads to the democratization of the NLP methods. We also observe the renewal of interest to languages other than English, the continuation of research on information extraction and prediction, the massive use of data from social media, and the consideration of needs and interests of patients.

Representation of time-varying and time-invariant EMR data and its application in modeling outcome prediction for heart failure patients

OBJECTIVE

To represent a patient record with both time-invariant and time-varying features as a single vector using an end-to-end deep learning model, and further to predict the kidney failure (KF) status and mortality of heart failure (HF) patients.

MATERIALS AND METHODS

The time-invariant EMR data included demographic information and comorbidities, and the time-varying EMR data were lab tests. We used a Transformer encoder module to represent the time-invariant data, and refined a long short-term memory (LSTM) with a Transformer encoder attached to the top to represent the time-varying data, taking the original measured values and their corresponding embedding vectors, masking vectors, and two types of time intervals as inputs. The proposed representations of patients with time-invariant and time-varying data were used to predict KF status (949 out of 5268 HF patients diagnosed with KF) and mortality (463 in-hospital deaths) for HF patients. Comparative experiments were conducted between the proposed model and some representative machine learning models. Ablation experiments were also performed around the time-varying data representation, including replacing the refined LSTM with the standard LSTM, GRU-D and T-LSTM, respectively, and removing the Transformer encoder and the time-varying data representation module, respectively. The visualization of the attention weights of the time-invariant and time-varying features was used to clinically interpret the predictive performance. We used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the F1-score to evaluate the predictive performance of the models.

RESULTS

The proposed model achieved superior performance, with average AUROCs, AUPRCs and F1-scores of 0.960, 0.610 and 0.759 for KF prediction and 0.937, 0.353 and 0.537 for mortality prediction, respectively. Predictive performance improved with the addition of time-varying data from longer time periods. The proposed model outperformed the comparison and ablation references in both prediction tasks.

CONCLUSIONS

Both time-invariant and time-varying EMR data of patients could be efficiently represented by the proposed unified deep learning model, which shows higher performance in clinical prediction tasks. The way to use time-varying data in the current study is hopeful to be used in other kinds of time-varying data and other clinical tasks.

Predicting outcomes at the individual patient level: what is the best method?

OBJECTIVE

When developing prediction models, researchers commonly employ a single model which uses all the available data (end-to-end approach). Alternatively, a similarity-based approach has been previously proposed, in which patients with similar clinical characteristics are first grouped into clusters, then prediction models are developed within each cluster. The potential advantage of the similarity-based approach is that it may better address heterogeneity in patient characteristics. However, it remains unclear whether it improves the overall predictive performance. We illustrate the similarity-based approach using data from people with depression and empirically compare its performance with the end-to-end approach.

METHODS

We used primary care data collected in general practices in the UK. Using 31 predefined baseline variables, we aimed to predict the severity of depressive symptoms, measured by Patient Health Questionnaire-9, 60 days after initiation of antidepressant treatment. Following the similarity-based approach, we used k-means to cluster patients based on their baseline characteristics. We derived the optimal number of clusters using the Silhouette coefficient. We used ridge regression to build prediction models in both approaches. To compare the models' performance, we calculated the mean absolute error (MAE) and the coefficient of determination (R2) using bootstrapping.

RESULTS

We analysed data from 16 384 patients. The end-to-end approach resulted in an MAE of 4.64 and R2 of 0.20. The best-performing similarity-based model was for four clusters, with MAE of 4.65 and R2 of 0.19.

CONCLUSIONS

The end-to-end and the similarity-based model yielded comparable performance. Due to its simplicity, the end-to-end approach can be favoured when using demographic and clinical data to build prediction models on pharmacological treatments for depression.

Improving Performance of Outcome Prediction for In-patients with Acute Myocardial Infarction Based on Embedding Representation Learned from Electronic Medical Records: Development and Validation Study (Preprint)

Background

The widespread secondary use of electronic medical records (EMRs) promotes health care quality improvement. Representation learning that can automatically extract hidden information from EMR data has gained increasing attention.

Objective

We aimed to propose a patient representation with more feature associations and task-specific feature importance to improve the outcome prediction performance for inpatients with acute myocardial infarction (AMI).

Methods

Medical concepts, including patients’ age, gender, disease diagnoses, laboratory tests, structured radiological features, procedures, and medications, were first embedded into real-value vectors using the improved skip-gram algorithm, where concepts in the context windows were selected by feature association strengths measured by association rule confidence. Then, each patient was represented as the sum of the feature embeddings weighted by the task-specific feature importance, which was applied to facilitate predictive model prediction from global and local perspectives. We finally applied the proposed patient representation into mortality risk prediction for 3010 and 1671 AMI inpatients from a public data set and a private data set, respectively, and compared it with several reference representation methods in terms of the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1-score.

Results

Compared with the reference methods, the proposed embedding-based representation showed consistently superior predictive performance on the 2 data sets, achieving mean AUROCs of 0.878 and 0.973, AUPRCs of 0.220 and 0.505, and F1-scores of 0.376 and 0.674 for the public and private data sets, respectively, while the greatest AUROCs, AUPRCs, and F1-scores among the reference methods were 0.847 and 0.939, 0.196 and 0.283, and 0.344 and 0.361 for the public and private data sets, respectively. Feature importance integrated in patient representation reflected features that were also critical in prediction tasks and clinical practice.

Conclusions

The introduction of feature associations and feature importance facilitated an effective patient representation and contributed to prediction performance improvement and model interpretation.