Use of electronic medical records in development and validation of risk prediction models of hospital readmission: systematic review

Nursing Variables Predicting Readmissions in Patients With a High Risk: A Scoping Review

Unplanned readmission endangers patient safety and increases unnecessary healthcare expenditure. Identifying nursing variables that predict patient readmissions can aid nurses in providing timely nursing interventions that help patients avoid readmission after discharge. We aimed to provide an overview of the nursing variables predicting readmission of patients with a high risk. The authors searched five databases-PubMed, CINAHL, EMBASE, Cochrane Library, and Scopus-for publications from inception to April 2023. Search terms included "readmission" and "nursing records." Eight studies were included for review. Nursing variables were classified into three categories-specifically, nursing assessment, nursing diagnosis, and nursing intervention. The nursing assessment category comprised 75% of the nursing variables; the proportions of the nursing diagnosis (25%) and nursing intervention categories (12.5%) were relatively low. Although most variables of the nursing assessment category focused on the patients' physical aspect, emotional and social aspects were also considered. This study demonstrated how nursing care contributes to patients' adverse outcomes. The findings can assist nurses in identifying the essential nursing assessment, diagnosis, and interventions, which should be provided from the time of patients' admission. This can mitigate preventable readmissions of patients with a high risk and facilitate their safe transition from an acute care setting to the community.

Enhancing readmission prediction models by integrating insights from home healthcare notes: Retrospective cohort study

BACKGROUND

Hospital readmission is an important indicator of inpatient care quality and a significant driver of increasing medical costs. Therefore, it is important to explore the effects of postdischarge information, particularly from home healthcare notes, on enhancing readmission prediction models. Despite the use of Natural Language Processing (NLP) and machine learning in prediction model development, current studies often overlook insights from home healthcare notes.

OBJECTIVE

This study aimed to develop prediction models for 30-day readmissions using home healthcare notes and structured data. In addition, it explored the development of 14- and 180-day prediction models using variables in the 30-day model.

DESIGN

A retrospective observational cohort study.

SETTING(S)

This study was conducted at Ajou University School of Medicine in South Korea.

PARTICIPANTS

Data from electronic health records, encompassing demographic characteristics of 1819 participants, along with information on conditions, drug, and home healthcare, were utilized.

METHODS

Two distinct models were developed for each prediction window (30-, 14-, 180-day): the traditional model, which utilized structured variables alone, and the common data model (CDM)-NLP model, which incorporated structured and topic variables extracted from home healthcare notes. BERTopic facilitated topic generation and risk probability, representing the likelihood of documents being assigned to specific topics. Feature selection involved experimenting with various algorithms. The best-performing algorithm, determined using the area under the receiver operating characteristic curve (AUROC), was used for model development. Model performance was assessed using various learning metrics including AUROC.

RESULTS

Among 1819 patients, 251 (13.80 %) experienced 30-day readmission. The least absolute shrinkage and selection operator was used for feature extraction and model development. The 15 structured features were used in the traditional model. Moreover, five additional topic variables from the home healthcare notes were applied in the CDM-NLP model. The AUROC of the traditional model was 0.739 (95 % CI: 0.672-0.807). The AUROC of the CDM-NLP model was high at 0.824 (95 % CI: 0.768-0.880), which indicated an outstanding performance. The topics in the CDM-NLP model included emotional distress, daily living functions, nutrition, postoperative status, and cardiorespiratory issues. In extended prediction model development for 14- and 180-day readmissions, the CDM-NLP consistently outperformed the traditional model.

CONCLUSIONS

This study developed effective prediction models using both structured and unstructured data, thereby emphasizing the significance of postdischarge information from home healthcare notes in readmission prediction.

Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis

BACKGROUND

The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited.

OBJECTIVE

This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics.

METHODS

We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations.

RESULTS

The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations.

CONCLUSIONS

Caution must be taken when interpreting fairness measures' face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities.

Hospital Readmissions for Fluid Overload among Individuals with Diabetes and Diabetic Kidney Disease: Risk Factors and Multivariable Prediction Models

AIMS

Hospital readmissions due to recurrent fluid overload in diabetes and diabetic kidney disease can be avoided with evidence-based interventions. We aimed to identify at-risk patients who can benefit from these interventions by developing risk prediction models for readmissions for fluid overload in people living with diabetes and diabetic kidney disease.

METHODS

This was a single-center retrospective cohort study of 1,531 adults with diabetes and diabetic kidney disease hospitalized for fluid overload, congestive heart failure, pulmonary edema, and generalized edema between 2015 and 2017. The multivariable regression models for 30-day and 90-day readmission for fluid overload were compared with the LACE score for discrimination, calibration, sensitivity, specificity, and net reclassification index (NRI).

RESULTS

Readmissions for fluid overload within 30 days and 90 days occurred in 8.6% and 17.2% of patients with diabetes, and 8.2% and 18.3% of patients with diabetic kidney disease, respectively. After adjusting for demographics, comorbidities, clinical parameters, and medications, a history of alcoholism (HR 3.85, 95% CI: 1.41-10.55) and prior hospitalization for fluid overload (HR 2.50, 95% CI: 1.26-4.96) were independently associated with 30-day readmission in patients with diabetic kidney disease, as well as in individuals with diabetes. Additionally, current smoking, absence of hypertension, and high-dose intravenous furosemide were also associated with 30-day readmission in individuals with diabetes. Prior hospitalization for fluid overload (HR 2.43, 95% CI: 1.50-3.94), cardiovascular disease (HR 1.44, 95% CI: 1.03-2.02), eGFR ≤45 mL/min/1.73 m2 (HR 1.39, 95% CI: 1.003-1.93) was independently associated with 90-day readmissions in individuals with diabetic kidney disease. Additionally, thiazide prescription at discharge reduced 90-day readmission in diabetic kidney disease, while the need for high-dose intravenous furosemide predicted 90-day readmission in diabetes. The clinical and clinico-psychological models for 90-day readmission in individuals with diabetes and diabetic kidney disease had better discrimination and calibration than the LACE score. The NRI for the clinico-psychosocial models to predict 30- and 90-day readmissions in diabetes was 22.4% and 28.9%, respectively. The NRI for the clinico-psychosocial models to predict 30- and 90-day readmissions in diabetic kidney disease was 5.6% and 38.9%, respectively.

CONCLUSION

The risk models can potentially be used to identify patients at risk of readmission for fluid overload for evidence-based interventions, such as patient education or transitional care programs to reduce preventable hospitalizations.

An overview of reviews and synthesis across 440 studies examines the importance of hospital readmission predictors across various patient populations

OBJECTIVES

The scientific literature contains an abundance of prediction models for hospital readmissions. However, no review has yet synthesized their predictors across various patient populations. Therefore, our aim was to examine predictors of hospital readmissions across 13 patient populations.

STUDY DESIGN AND SETTING

An overview of systematic reviews was combined with a meta-analytical approach. Two thousand five hundred four different predictors were categorized using common ontologies to pool and examine their odds ratios and frequencies of use in prediction models across and within different patient populations.

RESULTS

Twenty-eight systematic reviews with 440 primary studies were included. Numerous predictors related to prior use of healthcare services (odds ratio; 95% confidence interval: 1.64; 1.42-1.89), diagnoses (1.41; 1.31-1.51), health status (1.35; 1.20-1.52), medications (1.28; 1.13-1.44), administrative information about the index hospitalization (1.23; 1.14-1.33), clinical procedures (1.20; 1.07-1.35), laboratory results (1.18; 1.11-1.25), demographic information (1.10; 1.06-1.14), and socioeconomic status (1.07; 1.02-1.11) were analyzed. Diagnoses were frequently used (in 37.38%) and displayed large effect sizes across all populations. Prior use of healthcare services showed the largest effect sizes but were seldomly used (in 2.57%), whereas demographic information (in 13.18%) was frequently used but displayed small effect sizes.

CONCLUSION

Diagnoses and patients' prior use of healthcare services showed large effects both across and within different populations. These results can serve as a foundation for future prediction modeling.

Early repeat hospitalization for fluid overload in individuals with cardiovascular disease and risks: a retrospective cohort study

AIMS

Fluid overload is a common manifestation of cardiovascular and kidney disease and a leading cause of hospitalizations. To identify patients at risk of recurrent severe fluid overload, we evaluated the incidence and risk factors associated with early repeat hospitalization for fluid overload among individuals with cardiovascular disease and risks.

METHODS

Single-center retrospective cohort study of 3423 consecutive adults with an index hospitalization for fluid overload between January 2015 and December 2017 and had cardiovascular risks (older age, diabetes mellitus, hypertension, dyslipidemia, kidney disease, known cardiovascular disease), but excluded if lost to follow-up or eGFR < 15 ml/min/1.73 m2. The outcome was early repeat hospitalization for fluid overload within 30 days of discharge.

RESULTS

The mean age was 73.9 ± 11.6 years and eGFR was 54.1 ± 24.6 ml/min/1.73 m2 at index hospitalization. Early repeat hospitalization for fluid overload occurred in 291 patients (8.5%). After adjusting for demographics, comorbidities, clinical parameters during index hospitalization and medications at discharge, cardiovascular disease (adjusted odds ratio, OR 1.66, 95% CI 1.27-2.17), prior hospitalization for fluid overload within 3 months (OR 2.52, 95% CI 1.17-5.44), prior hospitalization for any cause in within 6 months (OR 1.33, 95% CI 1.02-1.73) and intravenous furosemide use (OR 1.58, 95% CI 1.10-2.28) were associated with early repeat hospitalization for fluid overload. Higher systolic BP on admission (OR 0.992, 95% 0.986-0.998) and diuretic at discharge (OR 0.50, 95% CI 0.26-0.98) reduced early hospitalization for fluid overload.

CONCLUSION

Patients at-risk of early repeat hospitalization for fluid overload may be identified using these risk factors for targeted interventions.

Clinical Impact of "Real World Data" and Blockchain on Public Health: A Scoping Review

The digitisation of healthcare has allowed a significant rethinking of the previous clinical protocols, improving their interoperability through substantial standardisation. These technological advances have ensured that data are comparable, as they are obtained from 'reliable' and certified processes; however, there are billions of data that are neither structured nor quality-controlled. They are collectively referred to as 'Real World Data' (RWD). Blockchain (BC) is a procedure with specific characteristics and algorithms that ensure that the stored data cannot be tampered with. Nowadays, there is an increasing need to rethink blockchain in a one-health vision, making it more than just a 'repository' of data and information, but rather an active player in the process of data sharing. In this landscape, several scholars have analysed the potential benefits of BC in healthcare, focusing on the sharing and safety of clinical data and its contact tracing applications. There is limited research on this matter; moreover, there is a strategic interest in managing RWD in a reliable and comparable way, despite the lack of knowledge on this topic. Our work aims to analyse systematically the most impacting literature, highlighting the main aspects of BC within the context of the new digital healthcare, and speculating on the unexpressed potential of RWD.

Chinese experts' consensus on the application of intensive care big data

The development of intensive care medicine is inseparable from the diversified monitoring data. Intensive care medicine has been closely integrated with data since its birth. Critical care research requires an integrative approach that embraces the complexity of critical illness and the computational technology and algorithms that can make it possible. Considering the need of standardization of application of big data in intensive care, Intensive Care Medicine Branch of China Health Information and Health Care Big Data Society, Standard Committee has convened expert group, secretary group and the external audit expert group to formulate Chinese Experts' Consensus on the Application of Intensive Care Big Data (2022). This consensus makes 29 recommendations on the following five parts: Concept of intensive care big data, Important scientific issues, Standards and principles of database, Methodology in solving big data problems, Clinical application and safety consideration of intensive care big data. The consensus group believes this consensus is the starting step of application big data in the field of intensive care. More explorations and big data based retrospective research should be carried out in order to enhance safety and reliability of big data based models of critical care field.

Advanced Sampling Technique in Radiology Free-Text Data for Efficiently Building Text Mining Models by Deep Learning in Vertebral Fracture

This study aims to establish advanced sampling methods in free-text data for efficiently building semantic text mining models using deep learning, such as identifying vertebral compression fracture (VCF) in radiology reports. We enrolled a total of 27,401 radiology free-text reports of X-ray examinations of the spine. The predictive effects were compared between text mining models built using supervised long short-term memory networks, independently derived by four sampling methods: vector sum minimization, vector sum maximization, stratified, and simple random sampling, using four fixed percentages. The drawn samples were applied to the training set, and the remaining samples were used to validate each group using different sampling methods and ratios. The predictive accuracy was measured using the area under the receiver operating characteristics (AUROC) to identify VCF. At the sampling ratios of 1/10, 1/20, 1/30, and 1/40, the highest AUROC was revealed in the sampling methods of vector sum minimization as confidence intervals of 0.981 (95%CIs: 0.980-0.983)/0.963 (95%CIs: 0.961-0.965)/0.907 (95%CIs: 0.904-0.911)/0.895 (95%CIs: 0.891-0.899), respectively. The lowest AUROC was demonstrated in the vector sum maximization. This study proposes an advanced sampling method, vector sum minimization, in free-text data that can be efficiently applied to build the text mining models by smartly drawing a small amount of critical representative samples.

Biomedical Big Data Technologies, Applications, and Challenges for Precision Medicine: A Review

The explosive growth of biomedical Big Data presents both significant opportunities and challenges in the realm of knowledge discovery and translational applications within precision medicine. Efficient management, analysis, and interpretation of big data can pave the way for groundbreaking advancements in precision medicine. However, the unprecedented strides in the automated collection of large-scale molecular and clinical data have also introduced formidable challenges in terms of data analysis and interpretation, necessitating the development of novel computational approaches. Some potential challenges include the curse of dimensionality, data heterogeneity, missing data, class imbalance, and scalability issues. This overview article focuses on the recent progress and breakthroughs in the application of big data within precision medicine. Key aspects are summarized, including content, data sources, technologies, tools, challenges, and existing gaps. Nine fields-Datawarehouse and data management, electronic medical record, biomedical imaging informatics, Artificial intelligence-aided surgical design and surgery optimization, omics data, health monitoring data, knowledge graph, public health informatics, and security and privacy-are discussed.

The Impact of Documentation Workflow on the Accuracy of the Coded Diagnoses in the Electronic Health Record

Objective

To determine the impact of documentation workflow on the accuracy of coded diagnoses in electronic health records (EHRs).

Design

Cross-sectional study.

Participants

All patients who completed visits at the Casey Eye Institute Retina Division faculty clinic between April 7, 2022 and April 13, 2022.

Main Outcome Measures

Agreement between coded diagnoses and clinical notes.

Methods

We assessed the rate of agreement between the diagnoses in the clinical notes and the coded diagnosis in the EHR using manual review and examined the impact of the documentation workflow on the rate of agreement in an academic retina practice.

Results

In 202 visits by 8 physicians, 78% (range, 22%-100%) had an agreement between the coded diagnoses and the clinical notes. When physicians integrated the diagnosis code entry and note composition, the rate of agreement was 87.9% (range, 62%-100%). For those who entered the diagnosis codes separately from writing notes, the agreement was 44.4% (22%-50%, P < 0.0001).

Conclusion

The visit-specific agreement between the coded diagnosis and the progress note can vary widely by workflow. The workflow and EHR design may be an important part of understanding and improving the quality of EHR data.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Novel Prediction Equations for Absolute Risk Assessment of Total Cardiovascular Disease Incorporating Cardiovascular-Kidney-Metabolic Health: A Scientific Statement From the American Heart Association

Cardiovascular-kidney-metabolic (CKM) syndrome is a novel construct recently defined by the American Heart Association in response to the high prevalence of metabolic and kidney disease. Epidemiological data demonstrate higher absolute risk of both atherosclerotic cardiovascular disease (CVD) and heart failure as an individual progresses from CKM stage 0 to stage 3, but optimal strategies for risk assessment need to be refined. Absolute risk assessment with the goal to match type and intensity of interventions with predicted risk and expected treatment benefit remains the cornerstone of primary prevention. Given the growing number of therapies in our armamentarium that simultaneously address all 3 CKM axes, novel risk prediction equations are needed that incorporate predictors and outcomes relevant to the CKM context. This should also include social determinants of health, which are key upstream drivers of CVD, to more equitably estimate and address risk. This scientific statement summarizes the background, rationale, and clinical implications for the newly developed sex-specific, race-free risk equations: PREVENT (AHA Predicting Risk of CVD Events). The PREVENT equations enable 10- and 30-year risk estimates for total CVD (composite of atherosclerotic CVD and heart failure), include estimated glomerular filtration rate as a predictor, and adjust for competing risk of non-CVD death among adults 30 to 79 years of age. Additional models accommodate enhanced predictive utility with the addition of CKM factors when clinically indicated for measurement (urine albumin-to-creatinine ratio and hemoglobin A1c) or social determinants of health (social deprivation index) when available. Approaches to implement risk-based prevention using PREVENT across various settings are discussed.

The unfulfilled promises of electronic health records

We provide a brief update on the current evidence on electronic health records' benefits, risks, and potential harms through a rapid narrative review. Many of the promised benefits of electronic health records have not yet been realised. Electronic health records are often not user-friendly. To enhance their potential, electronic health record platforms should be continuously evaluated and enhanced by carefully considering feedback from all stakeholders.

Natural language processing to identify social determinants of health in Alzheimer's disease and related dementia from electronic health records

OBJECTIVE

To develop a natural language processing (NLP) algorithm that identifies social determinants of health (SDoH), including housing, transportation, food, and medication insecurities, social isolation, abuse, neglect, or exploitation, and financial difficulties for patients with Alzheimer's disease and related dementias (ADRD) from unstructured electronic health records (EHRs).

DATA SOURCES AND STUDY SETTING

We leveraged 1000 medical notes randomly selected from 7401 emergency department and inpatient social worker notes generated between 2015 and 2019 for 231 unique patients diagnosed with ADRD at Michigan Medicine.

STUDY DESIGN

We developed a rule-based NLP algorithm for the identification of seven domains of SDoH noted above. We also compared the rule-based algorithm with deep learning and regularized logistic regression approaches. These models were compared using accuracy, sensitivity, specificity, F1 score, and the area under the receiver operating characteristic curve (AUC). All notes were split into 700 notes for training NLP algorithms, and 300 notes for validation.

DATA COLLECTION/EXTRACTION METHODS

Social worker notes used in this study were extracted from the Michigan Medicine EHR database.

PRINCIPAL FINDINGS

Of the 700 notes for training, F1 and AUC for the rule-based algorithm were at least 0.94 and 0.95, respectively, for all SDoH categories. Of the 300 notes for validation, F1 and AUC were at least 0.80 and 0.97, respectively, for all SDoH except housing and medication insecurities. The deep learning and regularized logistic regression algorithms had unsatisfactory performance.

CONCLUSIONS

The rule-based algorithm can accurately extract SDoH information in all seven domains of SDoH except housing and medication insecurities. Findings from the algorithm can be used by clinicians and social workers to proactively address social needs of patients with ADRD and other vulnerable patient populations.

Factors associated with readmission in chronic kidney disease: Systematic review and meta-analysis

BACKGROUND

Risk factors associated with all-cause hospital readmission are poorly characterised in patients with chronic kidney disease.

OBJECTIVE

A systematic review and meta-analysis were conducted to identify risk factors and protectors of hospital readmission in chronic kidney disease.

DESIGN, PARTICIPANTS & MEASUREMENTS

Studies involving adult patients were identified from four databases from inception to 31/03/2020. Random-effects meta-analyses were conducted to determine factors associated with all-cause 30-day hospital readmission in general chronic kidney disease, in dialysis and in kidney transplant recipient groups.

RESULTS

Eighty relevant studies (chronic kidney disease, n = 14 studies; dialysis, n = 34 studies; and transplant, n = 32 studies) were identified. Meta-analysis revealed that in both chronic kidney disease and transplant groups, increasing age in years and days spent at the hospital during the initial stay were associated with a higher risk of 30-day readmission. Other risk factors identified included increasing body mass index (kg/m2 ) in the transplant group, and functional impairment and discharge destination in the dialysis group. Within the chronic kidney disease ​​​​group, having an outpatient follow-up appointment with a nephrologist within 14 days of discharge was protective against readmission but this was not protective if provided by a primary care provider or a cardiologist.

CONCLUSION

Risk-reduction interventions that can be implemented include a nephrologist appointment within 14 days of hospital discharge, rehabilitation programme for functional improvement in the dialysis group and meal plans in the transplant group. Future risk analysis should focus on modifiable factors to ensure that strategies can be tested and implemented in those who are more at risk.

Finding the best trade-off between performance and interpretability in predicting hospital length of stay using structured and unstructured data

OBJECTIVE

This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data.

METHODS

The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines.

RESULTS

The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery.

CONCLUSIONS

Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.

Using the Elixhauser risk adjustment model to predict outcomes among patients hospitalized in internal medicine at a large, tertiary-care hospital in Israel

BACKGROUND

In Israel, internal medicine admissions are currently reimbursed without accounting for patient complexity. This is at odds with most other developed countries and has the potential to lead to market distortions such as avoiding sicker patients. Our objective was to apply a well-known, freely available risk adjustment model, the Elixhauser model, to predict relevant outcomes among patients hospitalized on the internal medicine service of a large, Israeli tertiary-care hospital.

METHODS

We used data from the Shaare Zedek Medical Center, a large tertiary referral hospital in Jerusalem. The study included 55,946 hospitalizations between 01.01.2016 and 31.12.2019. We modeled four patient outcomes: in-hospital mortality, escalation of care (intensive care unit (ICU) transfer, mechanical ventilation, daytime bi-level positive pressure ventilation, or vasopressors), 30-day readmission, and length of stay (LOS). We log-transformed LOS to address right skew. As is usual with the Elixhauser model, we identified 29 comorbid conditions using international classification of diseases codes, clinical modification, version 9. We derived and validated the coefficients for these 29 variables using split-sample derivation and validation. We checked model fit using c-statistics and R2, and model calibration using a Hosmer-Lemeshow test.

RESULTS

The Elixhauser model achieved acceptable prediction of the three binary outcomes, with c-statistics of 0.712, 0.681, and 0.605 to predict in-hospital mortality, escalation of care, and 30-day readmission respectively. The c-statistic did not decrease in the validation set (0.707, 0.687, and 0.603, respectively), suggesting that the models are not overfitted. The model to predict log length of stay achieved an R2 of 0.102 in the derivation set and 0.101 in the validation set. The Hosmer-Lemeshow test did not suggest issues with model calibration.

CONCLUSION

We demonstrated that a freely-available risk adjustment model can achieve acceptable prediction of important clinical outcomes in a dataset of patients admitted to a large, Israeli tertiary-care hospital. This model could potentially be used as a basis for differential payment by patient complexity.

Machine learning risk estimation and prediction of death in continuing care facilities using administrative data

In this study, we aimed to identify the factors that were associated with mortality among continuing care residents in Alberta, during the coronavirus disease 2019 (COVID-19) pandemic. We achieved this by leveraging and linking various administrative datasets together. Then, we examined pre-processing methods in terms of prediction performance. Finally, we developed several machine learning models and compared the results of these models in terms of performance. We conducted a retrospective cohort study of all continuing care residents in Alberta, Canada, from March 1, 2020, to March 31, 2021. We used a univariable and a multivariable logistic regression (LR) model to identify predictive factors of 60-day all-cause mortality by estimating odds ratios (ORs) with a 95% confidence interval. To determine the best sensitivity-specificity cut-off point, the Youden index was employed. We developed several machine learning models to determine the best model regarding performance. In this cohort study, increased age, male sex, symptoms, previous admissions, and some specific comorbidities were associated with increased mortality. Machine learning and pre-processing approaches offer a potentially valuable method for improving risk prediction for mortality, but more work is needed to show improvement beyond standard risk factors.

The Impact of Machine Learning on Total Joint Arthroplasty Patient Outcomes: A Systemic Review

BACKGROUND

Supervised machine learning techniques have been increasingly applied to predict patient outcomes after hip and knee arthroplasty procedures. The purpose of this study was to systematically review the applications of supervised machine learning techniques to predict patient outcomes after primary total hip and knee arthroplasty.

METHODS

A comprehensive literature search using the electronic databases MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews was conducted in July of 2021. The inclusion criteria were studies that utilized supervised machine learning techniques to predict patient outcomes after primary total hip or knee arthroplasty.

RESULTS

Search criteria yielded n = 30 relevant studies. Topics of study included patient complications (n = 6), readmissions (n = 1), revision (n = 2), patient-reported outcome measures (n = 4), patient satisfaction (n = 4), inpatient status and length of stay (LOS) (n = 9), opioid usage (n = 3), and patient function (n = 1). Studies involved TKA (n = 12), THA (n = 11), or a combination (n = 7). Less than 35% of predictive outcomes had an area under the receiver operating characteristic curve (AUC) in the excellent or outstanding range. Additionally, only 9 of the studies found improvement over logistic regression, and only 9 studies were externally validated.

CONCLUSION

Supervised machine learning algorithms are powerful tools that have been increasingly applied to predict patient outcomes after total hip and knee arthroplasty. However, these algorithms should be evaluated in the context of prognostic accuracy, comparison to traditional statistical techniques for outcome prediction, and application to populations outside the training set. While machine learning algorithms have been received with considerable interest, they should be critically assessed and validated prior to clinical adoption.

Performance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis

Objectives

Machine learning algorithms are being increasingly used for predicting hospital readmissions. This meta-analysis evaluated the performance of logistic regression (LR) and machine learning (ML) models for the prediction of 30-day hospital readmission among patients in the US.

Methods

Electronic databases (i.e., Medline, PubMed, and Embase) were searched from January 2015 to December 2019. Only studies in the English language were included. Two reviewers performed studies screening, quality appraisal, and data collection. The quality of the studies was assessed using the Quality in Prognosis Studies (QUIPS) tool. Model performance was evaluated using the Area Under the Curve (AUC). A random-effects meta-analysis was performed using STATA 16.

Results

Nine studies were included based on the selection criteria. The most common ML techniques were tree-based methods such as boosting and random forest. Most of the studies had a low risk of bias (8/9). The AUC was greater with ML to predict 30-day all-cause hospital readmission compared with LR [Mean Difference (MD): 0.03; 95% Confidence Interval (CI) 0.01-0.05]. Subgroup analyses found that deep-learning methods had a better performance compared with LR (MD 0.06; 95% CI, 0.04-0.09), followed by neural networks (MD: 0.03; 95% CI, 0.03-0.03), while the AUCs of the tree-based (MD: 0.02; 95% CI -0.00-0.04) and kernel-based (MD: 0.02; 95% CI 0.02 (-0.13-0.16) methods were no different compared to LR. More than half of the studies evaluated heart failure-related rehospitalization (N = 5). For the readmission prediction among heart failure patients, ML performed better compared with LR, with a mean difference in AUC of 0.04 (95% CI, 0.01-0.07). The leave-one-out sensitivity analysis confirmed the robustness of the findings.

Conclusion

Multiple ML methods were used to predict 30-day all-cause hospital readmission. Performance varied across the ML methods, with deep-learning methods showing the best performance over the LR.

Predicting acute clinical deterioration with interpretable machine learning to support emergency care decision making

The emergency department (ED) is a fast-paced environment responsible for large volumes of patients with varied disease acuity. Operational pressures on EDs are increasing, which creates the imperative to efficiently identify patients at imminent risk of acute deterioration. The aim of this study is to systematically compare the performance of machine learning algorithms based on logistic regression, gradient boosted decision trees, and support vector machines for predicting imminent clinical deterioration for patients based on cross-sectional patient data extracted from electronic patient records (EPR) at the point of entry to the hospital. We apply state-of-the-art machine learning methods to predict early patient deterioration, based on their first recorded vital signs, observations, laboratory results, and other predictors documented in the EPR. Clinical deterioration in this study is measured by in-hospital mortality and/or admission to critical care. We build on prior work by incorporating interpretable machine learning and fairness-aware modelling, and use a dataset comprising 118, 886 unplanned admissions to Salford Royal Hospital, UK, to systematically compare model variations for predicting mortality and critical care utilisation within 24 hours of admission. We compare model performance to the National Early Warning Score 2 (NEWS2) and yield up to a 0.366 increase in average precision, up to a [Formula: see text] reduction in daily alert rate, and a median 0.599 reduction in differential bias amplification across the protected demographics of age and sex. We use Shapely Additive exPlanations to justify the models' outputs, verify that the captured data associations align with domain knowledge, and pair predictions with the causal context of each patient's most influential characteristics. Introducing our modelling to clinical practice has the potential to reduce alert fatigue and identify high-risk patients with a lower NEWS2 that might be missed currently, but further work is needed to trial the models in clinical practice. We encourage future research to follow a systematised approach to data-driven risk modelling to obtain clinically applicable support tools.

Simulation-Based Education in the Artificial Intelligence Era

Simulation-based medical education (SBME) has been widely implemented in skill training in various clinical specialties. SBME has contributed not only to patient and medical safety but also to undergraduate and specialist education in the healthcare field. In this review, we discuss the challenges and future directions of SBME in the artificial intelligence (AI) era. While SBME fidelity or methods may become highly complicated in the AI era, the fact is that learners play a central role. As SBME and clinical education are complementary, mutual feedback and improvement are essential, especially in non-technical skill development. For the development of sustainable SBME in the clinical field in the AI era, continuous improvement is needed by academia, educators, and learners.

Using remotely monitored patient activity patterns after hospital discharge to predict 30 day hospital readmission: a randomized trial

Hospital readmission prediction models often perform poorly, but most only use information collected until the time of hospital discharge. In this clinical trial, we randomly assigned 500 patients discharged from hospital to home to use either a smartphone or wearable device to collect and transmit remote patient monitoring (RPM) data on activity patterns after hospital discharge. Analyses were conducted at the patient-day level using discrete-time survival analysis. Each arm was split into training and testing folds. The training set used fivefold cross-validation and then final model results are from predictions on the test set. A standard model comprised data collected up to the time of discharge including demographics, comorbidities, hospital length of stay, and vitals prior to discharge. An enhanced model consisted of the standard model plus RPM data. Traditional parametric regression models (logit and lasso) were compared to nonparametric machine learning approaches (random forest, gradient boosting, and ensemble). The main outcome was hospital readmission or death within 30 days of discharge. Prediction of 30-day hospital readmission significantly improved when including remotely-monitored patient data on activity patterns after hospital discharge and using nonparametric machine learning approaches. Wearables slightly outperformed smartphones but both had good prediction of 30-day hospital-readmission.

All-Cause Readmission or Potentially Avoidable Readmission: Which Is More Predictable Using Frailty, Comorbidities, and ADL?

Background and Objectives

Readmission-related health care reforms have shifted their focus from all-cause readmissions (ACR) to potentially avoidable readmissions (PAR). However, little is known about the utility of analytic tools from administrative data in predicting PAR. This study determined whether 30-day ACR or 30-day PAR is more predictable using tools that assess frailty, comorbidities, and activities of daily living (ADL) from administrative data.

Research Design and Methods

This retrospective cohort study was conducted at a large general acute care hospital in Tokyo, Japan. We analyzed patients aged ≥70 years who had been admitted to and discharged from the subject hospital between July 2016 and February 2021. Using administrative data, we assessed each patient's Hospital Frailty Risk Score, Charlson Comorbidity Index, and Barthel Index on admission. To determine the influence of each tool on readmission predictions, we constructed logistic regression models with different combinations of independent variables for predicting unplanned ACR and PAR within 30 days of discharge.

Results

Among 16 313 study patients, 4.1% experienced 30-day ACR and 1.8% experienced 30-day PAR. The full model (including sex, age, annual household income, frailty, comorbidities, and ADL as independent variables) for 30-day PAR showed better discrimination (C-statistic: 0.79, 95% confidence interval: 0.77-0.82) than the full model for 30-day ACR (0.73, 0.71-0.75). The other prediction models for 30-day PAR also had consistently better discrimination than their corresponding models for 30-day ACR.

Discussion and Implications

PAR is more predictable than ACR when using tools that assess frailty, comorbidities, and ADL from administrative data. Our PAR prediction model may contribute to the accurate identification of at-risk patients in clinical settings who would benefit from transitional care interventions.

Machine Learning Algorithms to Predict Breast Cancer Recurrence Using Structured and Unstructured Sources from Electronic Health Records

Recurrence is a critical aspect of breast cancer (BC) that is inexorably tied to mortality. Reuse of healthcare data through Machine Learning (ML) algorithms offers great opportunities to improve the stratification of patients at risk of cancer recurrence. We hypothesized that combining features from structured and unstructured sources would provide better prediction results for 5-year cancer recurrence than either source alone. We collected and preprocessed clinical data from a cohort of BC patients, resulting in 823 valid subjects for analysis. We derived three sets of features: structured information, features from free text, and a combination of both. We evaluated the performance of five ML algorithms to predict 5-year cancer recurrence and selected the best-performing to test our hypothesis. The XGB (eXtreme Gradient Boosting) model yielded the best performance among the five evaluated algorithms, with precision = 0.900, recall = 0.907, F1-score = 0.897, and area under the receiver operating characteristic AUROC = 0.807. The best prediction results were achieved with the structured dataset, followed by the unstructured dataset, while the combined dataset achieved the poorest performance. ML algorithms for BC recurrence prediction are valuable tools to improve patient risk stratification, help with post-cancer monitoring, and plan more effective follow-up. Structured data provides the best results when fed to ML algorithms. However, an approach based on natural language processing offers comparable results while potentially requiring less mapping effort.

Promoting patient safety using electronic medical records in nursing/midwifery undergraduate curricula: Discussion paper

This discussion paper highlights the importance of Australian nursing and midwifery students' lack of exposure to electronic medical records during their undergraduate programs. There is pressure on universities offering nursing and midwifery programs to provide students with opportunities to learn to use patient electronic medical records. This will provide authentic rehearsal with the digital technology prior to clinical placements and increase graduate work readiness. Informed by contemporary literature, we describe the benefits of implementing electronic medical records (eMR) in health organisations and identify the challenges and barriers to implementing and integrating the education of electronic records into undergraduate nursing and midwifery programs. Undergraduate students who had not experienced eMR as part of on-campus learning felt unprepared and lacked confidence when commencing clinical practice. Some international nursing and midwifery programs have found that student's skills improve in decision-making and documenting patient observations when eMR is integrated into their university education program. Successful integration of an eMR program should consider academic/teaching staff skills and confidence in technology use, initial and ongoing costs and technical support required to deliver the program. In conclusion, Australian universities need to embed eMR learning experiences into the nursing and midwifery undergraduate curricula to increase students work-readiness with a focus on patient safety.

Prediction of Unplanned Hospital Readmission using Clinical and Longitudinal Wearable Sensor Features

Predictive models have been suggested as potential tools for identifying highest risk patients for hospital readmissions, in order to improve care coordination and ultimately long-term patient outcomes. However, the accuracy of current predictive models for readmission prediction is still moderate and further data enrichment is needed to identify at risk patients. This paper describes models to predict 90-day readmission, focusing on testing the predictive performance of wearable sensor features generated using multiscale entropy techniques and clinical features. Our study explores ways to incorporate pre-discharge and post-discharge wearable sensor features to make robust patient predictions. Data were used from participants enrolled in the AllofUs Research program. We extracted the inpatient cohort of patients and integrated clinical data from the electronic health records (EHR) and Fitbit sensor measurements. Entropy features were calculated from the longitudinal wearable sensor data, such as heart rate and mobility-related measurements, in order to characterize time series variability and complexity. Our best performing model acheived an AUC of 83%, and at 80% sensitivity acheived 75% specificity and 57% positive predictive value. Our results indicate that it would be possible to improve the ability to predict unplanned hospital readmissions by considering pre-discharge and post-discharge wearable features.

Integration of patient experience factors improves readmission prediction

Many readmission prediction models have marginal accuracy and are based on clinical and demographic data that exclude patient response data. The objective of this study was to evaluate the accuracy of a 30-day hospital readmission prediction model that incorporates patient response data capturing the patient experience. This was a prospective cohort study of 30-day hospital readmissions. A logistic regression model to predict readmission risk was created using patient responses obtained during interviewer-administered questionnaires as well as demographic and clinical data. Participants (N = 846) were admitted to 2 inpatient adult medicine units at Massachusetts General Hospital from 2012 to 2016. The primary outcome was the accuracy (measured by receiver operating characteristic) of a 30-day readmission risk prediction model. Secondary analyses included a readmission-focused factor analysis of individual versus collective patient experience questions. Of 1754 eligible participants, 846 (48%) were enrolled and 201 (23.8%) had a 30-day readmission. Demographic factors had an accuracy of 0.56 (confidence interval [CI], 0.50-0.62), clinical disease factors had an accuracy of 0.59 (CI, 0.54-0.65), and the patient experience factors had an accuracy of 0.60 (CI, 0.56-0.64). Taken together, their combined accuracy of receiver operating characteristic = 0.78 (CI, 0.74-0.82) was significantly more accurate than these factors were individually. The individual accuracy of patient experience, demographic, and clinical data was relatively poor and consistent with other risk prediction models. The combination of the 3 types of data significantly improved the ability to predict 30-day readmissions. This study suggests that more accurate 30-day readmission risk prediction models can be generated by including information about the patient experience.

Development of a risk prediction score for acute postpartum care utilization

BACKGROUND

Acute postpartum care utilization and readmissions are increasing in the United States and contribute significantly to maternal morbidity, mortality, and healthcare costs. Currently, there are limited data on the prediction of patients who will require acute postpartum care utilization.

OBJECTIVE

To develop and validate a risk prediction model for acute postpartum care utilization.

STUDY DESIGN

A retrospective cohort study of delivery hospitalizations with a linked birth certificate and discharge records in California from 2011 to 2015 was divided into a training and testing set for analysis and validation. Predictive models for acute postpartum care utilization using demographic, comorbidity, obstetrical complication, and other factors were developed using a backward stepwise logistic regression on training data. A risk score for acute postpartum care utilization was developed using beta coefficients from the factors remaining in the final multivariable model. Risk scores were validated using the testing dataset.

RESULTS

The final sample included 2,045,988 delivery hospitalizations with an acute postpartum care utilization rate of 7.6% in both training and testing cohorts. Twenty-two risk factors were identified for the final multivariable model, including several that were associated with two or more increased odds of acute care utilization (public insurance, postpartum hemorrhage, extremes of maternal age). The mean risk score was 2.45, conferring a 15 times higher risk of acute postpartum care utilization compared to those with a risk score <1 (RR 15.4, 95% CI: 11.0, 21.7). Demographics and test performance characteristics were comparably similar in predictive capability in both models (0.67 in both the training and testing cohorts).

CONCLUSION

Risk factors that are identifiable before discharge can be used to create a cumulative risk score to stratify patients at the lowest and highest risk of acute postpartum care utilization with satisfactory accuracy. External validation and the addition of other granular clinical variables are necessary to validate the feasibility of use.

Nationwide hospital admission data statistics and disease-specific 30-day readmission prediction

Purpose

Hospital readmission prediction uses historical patient visit data to train machine learning models to predict risk of patients being readmitted after the discharge. Data used to train models, such as patient demographics, disease types, localized distributions etc., play significant roles in the model performance. To date, many methods exist for hospital readmission prediction, but answers to some important questions still remain open. For example, how will demographics, such as gender, age, geographic, impact on readmission prediction? Do patients suffering from different diseases vary significantly in their readmission rates? What are the nationwide hospital admission data characteristics? and how do hospital speciality, ownership, and locations impact on their readmission rates? In this study, we carry systematic investigations to answer the above questions, and propose a predictive modeling framework to predict disease-specific 30-day hospital readmission.

Methods

We first implement statistics analysis by using National Readmission Databases (NRD) with over 15 million hospital visits. After that, we create features and disease-specific readmission datasets. An ensemble learning framework is proposed to conduct hospital readmission prediction and Friedman test and Nemenyi post-hoc test is used to validate our proposed method.

Results

Using National Readmission Databases (NRD), with over 15 million hospital visits, as our testbed, we summarize nationwide patient admission data statistics, in related to demographic, disease types, and hospital factors. We use feature engineering to design 526 representative features to model each patient visit. Our studies found that readmission rates vary significantly from diseases to diseases. For six diseases studied in our research, their readmission rates vary from 1.832 (Pneumonia) to 8.761% (Diabetes). Using random sampling and voting approaches, our study shows that soft voting outperforms hard voting on majority results, especially for AUC and balanced accuracy which are the main measures for imbalanced data. Random under sampling using 1.1:1 for negative:positive ratio achieves the best performance for AUC, balanced accuracy, and F1-score.

Conclusion

This paper carries out systematic studies to understand US nationwide hospital readmission data statistics, and further designs a machine learning framework for disease-specific 30-day hospital readmission prediction. Our study shows that hospital readmission rates vary significantly with respect to different disease types, gender, age groups, any other factors. Gradient boosting achieves the best performance for disease specific hospital readmission prediction.

Effective hospital readmission prediction models using machine-learned features

BACKGROUND

Hospital readmissions are one of the costliest challenges facing healthcare systems, but conventional models fail to predict readmissions well. Many existing models use exclusively manually-engineered features, which are labor intensive and dataset-specific. Our objective was to develop and evaluate models to predict hospital readmissions using derived features that are automatically generated from longitudinal data using machine learning techniques.

METHODS

We studied patients discharged from acute care facilities in 2015 and 2016 in Alberta, Canada, excluding those who were hospitalized to give birth or for a psychiatric condition. We used population-level linked administrative hospital data from 2011 to 2017 to train prediction models using both manually derived features and features generated automatically from observational data. The target value of interest was 30-day all-cause hospital readmissions, with the success of prediction measured using the area under the curve (AUC) statistic.

RESULTS

Data from 428,669 patients (62% female, 38% male, 27% 65 years or older) were used for training and evaluating models: 24,974 (5.83%) were readmitted within 30 days of discharge for any reason. Patients were more likely to be readmitted if they utilized hospital care more, had more physician office visits, had more prescriptions, had a chronic condition, or were 65 years old or older. The LACE readmission prediction model had an AUC of 0.66 ± 0.0064 while the machine learning model's test set AUC was 0.83 ± 0.0045, based on learning a gradient boosting machine on a combination of machine-learned and manually-derived features.

CONCLUSION

Applying a machine learning model to the computer-generated and manual features improved prediction accuracy over the LACE model and a model that used only manually-derived features. Our model can be used to identify high-risk patients, for whom targeted interventions may potentially prevent readmissions.

Machine learning methods to predict 30-day hospital readmission outcome among US adults with pneumonia: analysis of the national readmission database

Background

Hospital readmissions for pneumonia are a growing concern in the US, with significant consequences for costs and quality of care. This study developed the rule-based model and other machine learning (ML) models to predict 30-day readmission risk in patients with pneumonia and compared model performance.

Methods

This population-based study involved patients aged ≥ 18 years hospitalized with pneumonia from January 1, 2016, through November 30, 2016, using the Healthcare Cost and Utilization Project-National Readmission Database (HCUP-NRD). Rule-based algorithms and other ML algorithms, specifically decision trees, random forest, extreme gradient descent boosting (XGBoost), and Least Absolute Shrinkage and Selection Operator (LASSO), were used to model all-cause readmissions 30 days post-discharge from index pneumonia hospitalization. A total of 61 clinically relevant variables were included for ML model development. Models were trained on randomly partitioned 50% of the data and evaluated using the remaining dataset. Model hyperparameters were tuned using the ten-fold cross-validation on the resampled training dataset. The area under the receiver operating curves (AUROC) and area under precision-recall curves (AUPRC) were calculated for the testing set to evaluate the model performance.

Results

Of the 372,293 patients with an index hospital hospitalization for pneumonia, 48,280 (12.97%) were readmitted within 30 days. Judged by AUROC in the testing data, rule-based model (0.6591) significantly outperformed decision tree (0.5783, p value < 0.001), random forest (0.6509, p value < 0.01) and LASSO (0.6087, p value < 0.001), but was less superior than XGBoost (0.6606, p value = 0.015). The AUPRC of the rule-based model in the testing data (0.2146) was higher than the decision tree (0.1560), random forest (0.2052), and LASSO (0.2042), but was similar to XGBoost (0.2147). The top risk-predictive rules captured by the rule-based algorithm were comorbidities, illness severity, disposition locations, payer type, age, and length of stay. These predictive risk factors were also identified by other ML models with high variable importance.

Conclusion

The performance of machine learning models for predicting readmission in pneumonia patients varied. The XGboost was better than the rule-based model based on the AUROC. However, important risk factors for predicting readmission remained consistent across ML models.

Real-world data: a brief review of the methods, applications, challenges and opportunities

Background

The increased adoption of the internet, social media, wearable devices, e-health services, and other technology-driven services in medicine and healthcare has led to the rapid generation of various types of digital data, providing a valuable data source beyond the confines of traditional clinical trials, epidemiological studies, and lab-based experiments.

Methods

We provide a brief overview on the type and sources of real-world data and the common models and approaches to utilize and analyze real-world data. We discuss the challenges and opportunities of using real-world data for evidence-based decision making This review does not aim to be comprehensive or cover all aspects of the intriguing topic on RWD (from both the research and practical perspectives) but serves as a primer and provides useful sources for readers who interested in this topic.

Results and Conclusions

Real-world hold great potential for generating real-world evidence for designing and conducting confirmatory trials and answering questions that may not be addressed otherwise. The voluminosity and complexity of real-world data also call for development of more appropriate, sophisticated, and innovative data processing and analysis techniques while maintaining scientific rigor in research findings, and attentions to data ethics to harness the power of real-world data.

Government plans in the 2016 and 2021 Peruvian presidential elections: A natural language processing analysis of the health chapters

Background: While clinical medicine has exploded, electronic health records for Natural Language Processing (NLP) analyses, public health, and health policy research have not yet adopted these algorithms. We aimed to dissect the health chapters of the government plans of the 2016 and 2021 Peruvian presidential elections, and to compare different NLP algorithms. Methods: From the government plans (18 in 2016; 19 in 2021) we extracted each sentence from the health chapters. We used five NLP algorithms to extract keywords and phrases from each plan: Term Frequency–Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), TextRank, Keywords Bidirectional Encoder Representations from Transformers (KeyBERT), and Rapid Automatic Keywords Extraction (Rake). Results: In 2016 we analysed 630 sentences, whereas in 2021 there were 1,685 sentences. The TF-IDF algorithm showed that in 2016, 26 terms appeared with a frequency of 0.08 or greater, while in 2021 27 terms met this criterion. The LDA algorithm defined two groups. The first included terms related to things the population would receive (e.g., ’insurance’), while the second included terms about the health system (e.g., ’capacity’). In 2021, most of the government plans belonged to the second group. The TextRank analysis provided keywords showing that ’universal health coverage’ appeared frequently in 2016, while in 2021 keywords about the COVID-19 pandemic were often found. The KeyBERT algorithm provided keywords based on the context of the text. These keywords identified some underlying characteristics of the political party (e.g., political spectrum such as left-wing). The Rake algorithm delivered phrases, in which we found ’universal health coverage’ in 2016 and 2021. Conclusion: The NLP analysis could be used to inform on the underlying priorities in each government plan. NLP analysis could also be included in research of health policies and politics during general elections and provide informative summaries for the general population.

Identifying patients at risk of unplanned re-hospitalisation using statewide electronic health records

Preventing unplanned hospitalisations, including readmissions and re-presentations to the emergency department, is an important strategy for addressing the growing demand for hospital care. Significant successes have been reported from interventions put in place by hospitals to reduce their incidence. However, there is limited use of data-driven algorithms in hospital services to identify patients for enrolment into these intervention programs. Here we present the results of a study aiming to develop algorithms deployable at scale as part of a state government’s initiative to address rehospitalizations and which fills several gaps identified in the state-of-the-art literature. To the best of our knowledge, our study involves the largest-ever sample size for developing risk models. Logistic regression, random forests and gradient boosted techniques were explored as model candidates and validated retrospectively on five years of data from 27 hospitals in Queensland, Australia. The models used a range of predictor variables sourced from state-wide Emergency Department(ED), inpatient, hospital-dispensed medications and hospital-requested pathology databases. The investigation leads to several findings: (i) the advantage of looking at a longer patient data history, (ii) ED and inpatient datasets alone can provide useful information for predicting hospitalisation risk and the addition of medications and pathology test results leads to trivial performance improvements, (iii) predicting readmissions to the hospital was slightly easier than predicting re-presentations to ED after an inpatient stay, which was slightly easier again than predicting re-presentations to ED after an EDstay, (iv) a gradient boosted approach (XGBoost) was systematically the most powerful modelling approach across various tests.

Predictive Modeling of Hospital Readmission: Challenges and Solutions

Hospital readmission prediction is a study to learn models from historical medical data to predict probability of a patient returning to hospital in a certain period, e.g. 30 or 90 days, after the discharge. The motivation is to help health providers deliver better treatment and post-discharge strategies, lower the hospital readmission rate, and eventually reduce the medical costs. Due to inherent complexity of diseases and healthcare ecosystems, modeling hospital readmission is facing many challenges. By now, a variety of methods have been developed, but existing literature fails to deliver a complete picture to answer some fundamental questions, such as what are the main challenges and solutions in modeling hospital readmission; what are typical features/models used for readmission prediction; how to achieve meaningful and transparent predictions for decision making; and what are possible conflicts when deploying predictive approaches for real-world usages. In this paper, we systematically review computational models for hospital readmission prediction, and propose a taxonomy of challenges featuring four main categories: (1) data variety and complexity; (2) data imbalance, locality and privacy; (3) model interpretability; and (4) model implementation. The review summarizes methods in each category, and highlights technical solutions proposed to address the challenges. In addition, a review of datasets and resources available for hospital readmission modeling also provides firsthand materials to support researchers and practitioners to design new approaches for effective and efficient hospital readmission prediction.

Identify Caregiver Availability Using Medical Notes: Rule-Based Natural Language Processing

Background

Identifying caregiver availability, particularly for patients with dementia or those with a disability, is critical to informing the appropriate care planning by the health systems, hospitals, and providers. This information is not readily available, and there is a paucity of pragmatic approaches to automatically identifying caregiver availability and type.

Objective

Our main objective was to use medical notes to assess caregiver availability and type for hospitalized patients with dementia. Our second objective was to identify whether the patient lived at home or resided at an institution.

Methods

In this retrospective cohort study, we used 2016-2019 telephone-encounter medical notes from a single institution to develop a rule-based natural language processing (NLP) algorithm to identify the patient’s caregiver availability and place of residence. Using note-level data, we compared the results of the NLP algorithm with human-conducted chart abstraction for both training (749/976, 77%) and test sets (227/976, 23%) for a total of 223 adults aged 65 years and older diagnosed with dementia. Our outcomes included determining whether the patients (1) reside at home or in an institution, (2) have a formal caregiver, and (3) have an informal caregiver.

Results

Test set results indicated that our NLP algorithm had high level of accuracy and reliability for identifying whether patients had an informal caregiver (F₁=0.94, accuracy=0.95, sensitivity=0.97, and specificity=0.93), but was relatively less able to identify whether the patient lived at an institution (F₁=0.64, accuracy=0.90, sensitivity=0.51, and specificity=0.98). The most common explanations for NLP misclassifications across all categories were (1) incomplete or misspelled facility names; (2) past, uncertain, or undecided status; (3) uncommon abbreviations; and (4) irregular use of templates.

Conclusions

This innovative work was the first to use medical notes to pragmatically determine caregiver availability. Our NLP algorithm identified whether hospitalized patients with dementia have a formal or informal caregiver and, to a lesser extent, whether they lived at home or in an institutional setting. There is merit in using NLP to identify caregivers. This study serves as a proof of concept. Future work can use other approaches and further identify caregivers and the extent of their availability.

A Machine Learning Approach for Predicting Readmission Charges Billed by Hospitals

Background

The Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.

Objective

We aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).

Methods

We defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.

Results

Our preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.

Conclusions

Models built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.

Tasks as needs: reframing the paradigm of clinical natural language processing research for real-world decision support

Electronic medical records are increasingly used to store patient information in hospitals and other clinical settings. There has been a corresponding proliferation of clinical natural language processing (cNLP) systems aimed at using text data in these records to improve clinical decision-making, in comparison to manual clinician search and clinical judgment alone. However, these systems have delivered marginal practical utility and are rarely deployed into healthcare settings, leading to proposals for technical and structural improvements. In this paper, we argue that this reflects a violation of Friedman's "Fundamental Theorem of Biomedical Informatics," and that a deeper epistemological change must occur in the cNLP field, as a parallel step alongside any technical or structural improvements. We propose that researchers shift away from designing cNLP systems independent of clinical needs, in which cNLP tasks are ends in themselves-"tasks as decisions"-and toward systems that are directly guided by the needs of clinicians in realistic decision-making contexts-"tasks as needs." A case study example illustrates the potential benefits of developing cNLP systems that are designed to more directly support clinical needs.

Introduction to Clinical Prediction Models

Clinical prediction models include a diagnostic prediction model to estimate the probability of an individual currently having a disease (e.g., pulmonary embolism) and a prognostic prediction model to estimate the probability of an individual developing a specific health outcome over a specific time period (e.g., myocardial infarction and stroke in 10 years). Clinical prediction models can be developed by applying traditional regression models (e.g., logistic and Cox regression models) or emerging machine learning models to real-world data, such as electronic health records and administrative claims data. For derivation, researchers select candidate variables based on a literature review and clinical knowledge, and predictor variables in the final model based on pre-defined criteria (e.g., thresholds for the size of relative risk and p-values) or strategies such as the stepwise regression and the least absolute shrinkage and selection operator (LASSO) regression. For validation, the clinical prediction model's performance is evaluated in terms of goodness of fit (e.g., R2), discrimination (e.g., area under the receiver operating characteristic curve or c-statistics), and calibration (e.g., calibration plot and Hosmer-Lemeshow test). Performance of a new variable added to an existing clinical prediction model is evaluated in terms of reclassification (e.g., net reclassification improvement and integrated discrimination improvement). The model should be validated using the original data to examine internal validity through methods such as resampling (e.g., cross-validation and bootstrapping) and using other participants' data to examine external validity. For successful implementation of a clinical prediction model in actual clinical practice, presentation methods such as paper-based (nomogram) or web-based calculator and an easy-to-use risk score should be considered.

Forecasting Hospital Readmissions with Machine Learning

Hospital readmissions are regarded as a compounding economic factor for healthcare systems. In fact, the readmission rate is used in many countries as an indicator of the quality of services provided by a health institution. The ability to forecast patients’ readmissions allows for timely intervention and better post-discharge strategies, preventing future life-threatening events, and reducing medical costs to either the patient or the healthcare system. In this paper, four machine learning models are used to forecast readmissions: support vector machines with a linear kernel, support vector machines with an RBF kernel, balanced random forests, and weighted random forests. The dataset consists of 11,172 actual records of hospitalizations obtained from the General Hospital of Komotini “Sismanogleio” with a total of 24 independent variables. Each record is composed of administrative, medical-clinical, and operational variables. The experimental results indicate that the balanced random forest model outperforms the competition, reaching a sensitivity of 0.70 and an AUC value of 0.78.

Government plans in the 2016 and 2021 Peruvian presidential elections: A natural language processing analysis of the health chapters

Background: While clinical medicine has exploded, electronic health records for Natural Language Processing (NLP) analyses, public health, and health policy research have not yet adopted these algorithms. We aimed to dissect the health chapters of the government plans of the 2016 and 2021 Peruvian presidential elections, and to compare different NLP algorithms. Methods: From the government plans (18 in 2016; 19 in 2021) we extracted each sentence from the health chapters. We used five NLP algorithms to extract keywords and phrases from each plan: Term Frequency–Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), TextRank, Keywords Bidirectional Encoder Representations from Transformers (KeyBERT), and Rapid Automatic Keywords Extraction (Rake). Results: In 2016 we analysed 630 sentences, whereas in 2021 there were 1,685 sentences. The TF-IDF algorithm showed that in 2016, 22 terms appeared with a frequency of 0.05 or greater, while in 2021 27 terms met this criterion. The LDA algorithm defined two groups. The first included terms related to things the population would receive (e.g., ’insurance’), while the second included terms about the health system (e.g., ’capacity’). In 2021, most of the government plans belonged to the second group. The TextRank analysis provided keywords showing that ’universal health coverage’ appeared frequently in 2016, while in 2021 keywords about the COVID-19 pandemic were often found. The KeyBERT algorithm provided keywords based on the context of the text. These keywords identified some underlying characteristics of the political party (e.g., political spectrum such as left-wing). The Rake algorithm delivered phrases, in which we found ’universal health coverage’ in 2016 and 2021. Conclusion: The NLP analysis could be used to inform on the underlying priorities in each government plan. NLP analysis could also be included in research of health policies and politics during general elections and provide informative summaries for the general population.

A review of literature on risk prediction tools for hospital readmissions in older adults

PURPOSE

The purpose of this paper is to identify and analyse the readmission risk prediction tools reported in the literature and their benefits when it comes to healthcare organisations and management.

DESIGN/METHODOLOGY/APPROACH

Readmission risk prediction is a growing topic of interest with the aim of identifying patients in particular those suffering from chronic diseases such as congestive heart failure, chronic obstructive pulmonary disease and diabetes, who are at risk of readmission. Several models have been developed with different levels of predictive ability. A structured and extensive literature search of several databases was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis strategy, and this yielded a total of 48,984 records.

FINDINGS

Forty-three articles were selected for full-text and extensive review after following the screening process and according to the eligibility criteria. About 34 unique readmission risk prediction models were identified, in which their predictive ability ranged from poor to good (c statistic 0.5-0.86). Readmission rates ranged between 3.1 and 74.1% depending on the risk category. This review shows that readmission risk prediction is a complex process and is still relatively new as a concept and poorly understood. It confirms that readmission prediction models hold significant accuracy at identifying patients at higher risk for such an event within specific context.

RESEARCH LIMITATIONS/IMPLICATIONS

Since most prediction models were developed for specific populations, conditions or hospital settings, the generalisability and transferability of the predictions across wider or other contexts may be difficult to achieve. Therefore, the value of prediction models remains limited to hospital management. Future research is indicated in this regard.

ORIGINALITY/VALUE

This review is the first to cover readmission risk prediction tools that have been published in the literature since 2011, thereby providing an assessment of the relevance of this crucial KPI to health organisations and managers.

OUP accepted manuscript

Objective

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias.

Materials and Methods

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

Results

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

Discussion

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

Conclusion

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Artificial intelligence unifies knowledge and actions in drug repositioning

Drug repositioning aims to reuse existing drugs, shelved drugs, or drug candidates that failed clinical trials for other medical indications. Its attraction is sprung from the reduction in risk associated with safety testing of new medications and the time to get a known drug into the clinics. Artificial Intelligence (AI) has been recently pursued to speed up drug repositioning and discovery. The essence of AI in drug repositioning is to unify the knowledge and actions, i.e. incorporating real-world and experimental data to map out the best way forward to identify effective therapeutics against a disease. In this review, we share positive expectations for the evolution of AI and drug repositioning and summarize the role of AI in several methods of drug repositioning.

Government plans in the 2016 and 2021 Peruvian presidential elections: A natural language processing analysis of the health chapters

Background: While clinical medicine has exploded, electronic health records for Natural Language Processing (NLP) analyses, public health, and health policy research have not yet adopted these algorithms. We aimed to dissect the health chapters of the government plans of the 2016 and 2021 Peruvian presidential elections, and to compare different NLP algorithms. Methods: From the government plans (18 in 2016; 19 in 2021) we extracted each sentence from the health chapters. We used five NLP algorithms to extract keywords and phrases from each plan: Term Frequency–Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), TextRank, Keywords Bidirectional Encoder Representations from Transformers (KeyBERT), and Rapid Automatic Keywords Extraction (Rake). Results: In 2016 we analysed 630 sentences, whereas in 2021 there were 1,685 sentences. The TF-IDF algorithm showed that in 2016, 22 terms appeared with a frequency of 0.05 or greater, while in 2021 27 terms met this criterion. The LDA algorithm defined two groups. The first included terms related to things the population would receive (e.g., ’insurance’), while the second included terms about the health system (e.g., ’capacity’). In 2021, most of the government plans belonged to the second group. The TextRank analysis provided keywords showing that ’universal health coverage’ appeared frequently in 2016, while in 2021 keywords about the COVID-19 pandemic were often found. The KeyBERT algorithm provided keywords based on the context of the text. These keywords identified some underlying characteristics of the political party (e.g., political spectrum such as left-wing). The Rake algorithm delivered phrases, in which we found ’universal health coverage’ in 2016 and 2021. Conclusion: The NLP analysis could be used to inform on the underlying priorities in each government plan. NLP analysis could also be included in research of health policies and politics during general elections and provide informative summaries for the general population.

Bayesian inference on the number of recurrent events: A joint model of recurrence and survival

The number of recurrent events before a terminating event is often of interest. For instance, death terminates an individual's process of rehospitalizations and the number of rehospitalizations is an important indicator of economic cost. We propose a model in which the number of recurrences before termination is a random variable of interest, enabling inference and prediction on it. Then, conditionally on this number, we specify a joint distribution for recurrence and survival. This novel conditional approach induces dependence between recurrence and survival, which is often present, for instance, due to frailty that affects both. Additional dependence between recurrence and survival is introduced by the specification of a joint distribution on their respective frailty terms. Moreover, through the introduction of an autoregressive model, our approach is able to capture the temporal dependence in the recurrent events trajectory. A non-parametric random effects distribution for the frailty terms accommodates population heterogeneity and allows for data-driven clustering of the subjects. A tailored Gibbs sampler involving reversible jump and slice sampling steps implements posterior inference. We illustrate our model on colorectal cancer data, compare its performance with existing approaches and provide appropriate inference on the number of recurrent events.

External validation of the PAR-Risk Score to assess potentially avoidable hospital readmission risk in internal medicine patients

BACKGROUND

Readmission prediction models have been developed and validated for targeted in-hospital preventive interventions. We aimed to externally validate the Potentially Avoidable Readmission-Risk Score (PAR-Risk Score), a 12-items prediction model for internal medicine patients with a convenient scoring system, for our local patient cohort.

METHODS

A cohort study using electronic health record data from the internal medicine ward of a Swiss tertiary teaching hospital was conducted. The individual PAR-Risk Score values were calculated for each patient. Univariable logistic regression was used to predict potentially avoidable readmissions (PARs), as identified by the SQLape algorithm. For additional analyses, patients were stratified into low, medium, and high risk according to tertiles based on the PAR-Risk Score. Statistical associations between predictor variables and PAR as outcome were assessed using both univariable and multivariable logistic regression.

RESULTS

The final dataset consisted of 5,985 patients. Of these, 340 patients (5.7%) experienced a PAR. The overall PAR-Risk Score showed rather poor discriminatory power (C statistic 0.605, 95%-CI 0.575-0.635). When using stratified groups (low, medium, high), patients in the high-risk group were at statistically significant higher odds (OR 2.63, 95%-CI 1.33-5.18) of being readmitted within 30 days compared to low risk patients. Multivariable logistic regression identified previous admission within six months, anaemia, heart failure, and opioids to be significantly associated with PAR in this patient cohort.

CONCLUSION

This external validation showed a limited overall performance of the PAR-Risk Score, although higher scores were associated with an increased risk for PAR and patients in the high-risk group were at significantly higher odds of being readmitted within 30 days. This study highlights the importance of externally validating prediction models.