Usability of Electronic Health records in Predicting Short-term falls: Machine learning Applications in Senior Care Facilities (Preprint)

Development and External Validation of a Machine Learning-based Fall Prediction Model for Nursing Home Residents: A Prospective Cohort Study

OBJECTIVES

To develop and externally validate a machine learning-based fall prediction model for ambulatory nursing home residents. The focus is on predicting fall occurrences within 6 months after baseline assessment through a binary classification task, aiming to provide staff with an effective and user-friendly fall-risk assessment tool.

DESIGN

Prospective cohort study.

SETTING AND PARTICIPANTS

A total of 864 older residents living in 4 nursing homes between May 2022 and March 2023 in China.

METHODS

Potential fall-risk predictors were collected through in-person interviews and assessments of anthropometric and physical function. Participants were followed for 6 months, with falls recorded by trained nurses. Seven machine learning algorithms, including Logistic Regression (LR), Gradient Boosting Machine (GBM), eXtreme Gradient Boosting (XGBoost), Random Forest (RF), Support Vector Machine (SVM), Neural Networks (NN), and Decision Tree (DT), were used to develop prediction models. Performance was evaluated using the area under the receiver operating characteristic curve (ROC-AUC) and Precision-Recall curve (PR-AUC), with calibration assessed via a calibration curve. Feature importance was visualized using SHapley Additive exPlanations (SHAP).

RESULTS

The 6 selected predictors were balance, grip strength, fatigue, fall history, age, and comorbidity. The ROC-AUC for the models ranged from 0.710 to 0.750, PR-AUC from 0.415 to 0.473, sensitivity from 0.704 to 0.914, and specificity from 0.511 to 0.687 in the validation cohort. The LR model was converted into a nomogram.

CONCLUSIONS AND IMPLICATIONS

The machine learning-based fall-prediction models effectively identified nursing home residents at high risk of falls. The developed nomogram can be integrated into clinical practice to enhance fall risk assessment protocols, ultimately improving patient safety and care in nursing homes.

Legal and Ethical Considerations of Artificial Intelligence for Residents in Post-Acute and Long-Term Care

This article proposes a framework for examining the ethical and legal concerns for using artificial intelligence (AI) in post-acute and long-term care (PA-LTC). It argues that established frameworks on health, AI, and the law should be adapted to specific care contexts. For residents in PA-LTC, their social, psychological, and mobility needs should act as a gauge for examining the benefits and risks of integrating AI into their care. Using those needs as a gauge, 4 areas of particular concern are identified. First, the threat that AI poses to the autonomy of residents can undermine their core needs. Second, how discrimination and bias in algorithmic decision-making can undermine Medicare coverage for PA-LTC, causing doctors' recommendations to be ignored and denying residents the care they are entitled to. Third, privacy rules concerning data use may undermine developers' ability to train accurate AI systems, limiting their usefulness in PA-LTC contexts. Fourth, the importance of obtaining consent before AI is used and discussions about how that care should continue if there are concerns about an ongoing decline in cognition. Together, these considerations elevate existing frameworks and adapt them to the context-specific case of PA-LTC. It is hoped that future research will examine the legal implications of these matters in each of these specific cases.

Machine Learning Approach with Harmonized Multinational Datasets for Enhanced Prediction of Hypothyroidism in Patients with Type 2 Diabetes

Type 2 diabetes (T2D) is a global health concern with increasing prevalence. Comorbid hypothyroidism (HT) exacerbates kidney, cardiac, neurological and other complications of T2D; these risks can be mitigated pharmacologically upon detecting HT. The current HT standard of care (SOC) screening in T2D is infrequent, delaying HT diagnosis and treatment. We present a first-to-date machine learning algorithm (MLA) clinical decision tool to classify patients as low vs. high risk for developing HT comorbid with T2D; the MLA was developed using readily available patient data from harmonized multinational datasets. The MLA was trained on data from NIH All of US (AoU) and UK Biobank (UKBB) (Combined dataset) and achieved a high negative predictive value (NPV) of 0.989 and an AUROC of 0.762 in the Combined dataset, exceeding AUROCs for the models trained on AoU or UKBB alone (0.666 and 0.622, respectively), indicating that increasing dataset diversity for MLA training improves performance. This high-NPV automated tool can supplement SOC screening and rule out T2D patients with low HT risk, allowing for the prioritization of lab-based testing for at-risk patients. Conversely, an MLA output that designates a patient to be at risk of developing HT allows for tailored clinical management and thereby promotes improved patient outcomes.

Validation and Improvement of the Saga Fall Risk Model: A Multicenter Retrospective Observational Study

Purpose

We conducted a pilot study in an acute care hospital and developed the Saga Fall Risk Model 2 (SFRM2), a fall prediction model comprising eight items: Bedriddenness rank, age, sex, emergency admission, admission to the neurosurgery department, history of falls, independence of eating, and use of hypnotics. The external validation results from the two hospitals showed that the area under the curve (AUC) of SFRM2 may be lower in other facilities. This study aimed to validate the accuracy of SFRM2 using data from eight hospitals, including chronic care hospitals, and adjust the coefficients to improve the accuracy of SFRM2 and validate it.

Patients and Methods

This study included all patients aged ≥20 years admitted to eight hospitals, including chronic care, acute care, and tertiary hospitals, from April 1, 2018, to March 31, 2021. In-hospital falls were used as the outcome, and the AUC and shrinkage coefficient of SFRM2 were calculated. Additionally, SFRM2.1, which was modified from the coefficients of SFRM2 using logistic regression with the eight items comprising SFRM2, was developed using two-thirds of the data randomly selected from the entire population, and its accuracy was validated using the remaining one-third portion of the data.

Results

Of the 124,521 inpatients analyzed, 2,986 (2.4%) experienced falls during hospitalization. The median age of all inpatients was 71 years, and 53.2% were men. The AUC of SFRM2 was 0.687 (95% confidence interval [CI]:0.678-0.697), and the shrinkage coefficient was 0.996. SFRM2.1 was created using 81,790 patients, and its accuracy was validated using the remaining 42,731 patients. The AUC of SFRM2.1 was 0.745 (95% CI: 0.731-0.758).

Conclusion

SFRM2 showed good accuracy in predicting falls even on validating in diverse populations with significantly different backgrounds. Furthermore, the accuracy can be improved by adjusting the coefficients while keeping the model's parameters fixed.

BERT-Based Neural Network for Inpatient Fall Detection From Electronic Medical Records: Retrospective Cohort Study

BACKGROUND

Inpatient falls are a substantial concern for health care providers and are associated with negative outcomes for patients. Automated detection of falls using machine learning (ML) algorithms may aid in improving patient safety and reducing the occurrence of falls.

OBJECTIVE

This study aims to develop and evaluate an ML algorithm for inpatient fall detection using multidisciplinary progress record notes and a pretrained Bidirectional Encoder Representation from Transformers (BERT) language model.

METHODS

A cohort of 4323 adult patients admitted to 3 acute care hospitals in Calgary, Alberta, Canada from 2016 to 2021 were randomly sampled. Trained reviewers determined falls from patient charts, which were linked to electronic medical records and administrative data. The BERT-based language model was pretrained on clinical notes, and a fall detection algorithm was developed based on a neural network binary classification architecture.

RESULTS

To address various use scenarios, we developed 3 different Alberta hospital notes-specific BERT models: a high sensitivity model (sensitivity 97.7, IQR 87.7-99.9), a high positive predictive value model (positive predictive value 85.7, IQR 57.2-98.2), and the high F1-score model (F1=64.4). Our proposed method outperformed 3 classical ML algorithms and an International Classification of Diseases code-based algorithm for fall detection, showing its potential for improved performance in diverse clinical settings.

CONCLUSIONS

The developed algorithm provides an automated and accurate method for inpatient fall detection using multidisciplinary progress record notes and a pretrained BERT language model. This method could be implemented in clinical practice to improve patient safety and reduce the occurrence of falls in hospitals.

Association of Prospective Falls in Older People With Ubiquitous Step-Based Fall Risk Parameters Calculated From Ambulatory Inertial Signals: Secondary Data Analysis

Background

In recent years, researchers have been advocating for the integration of ambulatory gait monitoring as a complementary approach to traditional fall risk assessments. However, current research relies on dedicated inertial sensors that are fixed on a specific body part. This limitation impacts the acceptance and adoption of such devices.

Objective

Our study objective is twofold: (1) to propose a set of step-based fall risk parameters that can be obtained independently of the sensor placement by using a ubiquitous step detection method and (2) to evaluate their association with prospective falls.

Methods

A reanalysis was conducted on the 1-week ambulatory inertial data from the StandingTall study, which was originally described by Delbaere et al. The data were from 301 community-dwelling older people and contained fall occurrences over a 12-month follow-up period. Using the ubiquitous and robust step detection method Smartstep, which is agnostic to sensor placement, a range of step-based fall risk parameters can be calculated based on walking bouts of 200 steps. These parameters are known to describe different dimensions of gait (ie, variability, complexity, intensity, and quantity). First, the correlation between parameters was studied. Then, the number of parameters was reduced through stepwise backward elimination. Finally, the association of parameters with prospective falls was assessed through a negative binomial regression model using the area under the curve metric.

Results

The built model had an area under the curve of 0.69, which is comparable to models exclusively built on fixed sensor placement. A higher fall risk was noted with higher gait variability (coefficient of variance of stride time), intensity (cadence), and quantity (number of steps) and lower gait complexity (sample entropy and fractal exponent).

Conclusions

These findings highlight the potential of our method for comprehensive and accurate fall risk assessments, independent of sensor placement. This approach has promising implications for ambulatory gait monitoring and fall risk monitoring using consumer-grade devices.

Identifying Predictors of Nursing Home Admission by Using Electronic Health Records and Administrative Data: Scoping Review

Background

Among older adults, nursing home admissions (NHAs) are considered a significant adverse outcome and have been extensively studied. Although the volume and significance of electronic data sources are expanding, it is unclear what predictors of NHA have been systematically identified in the literature via electronic health records (EHRs) and administrative data.

Objective

This study synthesizes findings of recent literature on identifying predictors of NHA that are collected from administrative data or EHRs.

Methods

The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines were used for study selection. The PubMed and CINAHL databases were used to retrieve the studies. Articles published between January 1, 2012, and March 31, 2023, were included.

Results

A total of 34 papers were selected for final inclusion in this review. In addition to NHA, all-cause mortality, hospitalization, and rehospitalization were frequently used as outcome measures. The most frequently used models for predicting NHAs were Cox proportional hazards models (studies: n=12, 35%), logistic regression models (studies: n=9, 26%), and a combination of both (studies: n=6, 18%). Several predictors were used in the NHA prediction models, which were further categorized into sociodemographic, caregiver support, health status, health use, and social service use factors. Only 5 (15%) studies used a validated frailty measure in their NHA prediction models.

Conclusions

NHA prediction tools based on EHRs or administrative data may assist clinicians, patients, and policy makers in making informed decisions and allocating public health resources. More research is needed to assess the value of various predictors and data sources in predicting NHAs and validating NHA prediction models externally.

A comprehensive review of machine learning algorithms and their application in geriatric medicine: present and future

The increasing access to health data worldwide is driving a resurgence in machine learning research, including data-hungry deep learning algorithms. More computationally efficient algorithms now offer unique opportunities to enhance diagnosis, risk stratification, and individualised approaches to patient management. Such opportunities are particularly relevant for the management of older patients, a group that is characterised by complex multimorbidity patterns and significant interindividual variability in homeostatic capacity, organ function, and response to treatment. Clinical tools that utilise machine learning algorithms to determine the optimal choice of treatment are slowly gaining the necessary approval from governing bodies and being implemented into healthcare, with significant implications for virtually all medical disciplines during the next phase of digital medicine. Beyond obtaining regulatory approval, a crucial element in implementing these tools is the trust and support of the people that use them. In this context, an increased understanding by clinicians of artificial intelligence and machine learning algorithms provides an appreciation of the possible benefits, risks, and uncertainties, and improves the chances for successful adoption. This review provides a broad taxonomy of machine learning algorithms, followed by a more detailed description of each algorithm class, their purpose and capabilities, and examples of their applications, particularly in geriatric medicine. Additional focus is given on the clinical implications and challenges involved in relying on devices with reduced interpretability and the progress made in counteracting the latter via the development of explainable machine learning.

In-hospital fall prediction using machine learning algorithms and the Morse fall scale in patients with acute stroke: a nested case-control study

BACKGROUND

Falls are one of the most common accidents in medical institutions, which can threaten the safety of inpatients and negatively affect their prognosis. Herein, we developed a machine learning (ML) model for fall prediction in patients with acute stroke and compared its accuracy with that of the existing fall risk prediction tool, the Morse Fall Scale (MFS).

METHODS

This is a retrospective nested case-control study. The initial sample size was 8462 admitted to a single cerebrovascular specialty hospital with acute stroke. A total of 156 fall events occurred, and each fall case was randomly matched with six control cases. Six ML algorithms were used, namely, regularized logistic regression, support vector machine, naïve Bayes (NB), k-nearest neighbors, random forest, and extreme-gradient boosting (XGB).

RESULTS

We included 156 in the fall group and 934 in the non-fall group. The mean ages of the fall and non-fall groups were 68.3 (± 12.2) and 65.3 (± 12.9) years old, respectively. The MFS total score was significantly higher in the fall group (54.3 ± 18.3) than in the non-fall group (37.7 ± 14.7). The area under the receiver operating curve (AUROC) of the MFS in predicting falls was 0.76 (0.73-0.79). XGB had the highest AUROC of 0.85 (0.78-0.92), and XGB and NB had the highest F1 score of 0.44.

CONCLUSIONS

The AUROC values of all of ML algorithms were similar to those of the MFS in predicting fall risk in patients with acute stroke, allowing for accurate and efficient fall screening.

Machine Learning Models to Predict Future Frailty in Community-Dwelling Middle-Aged and Older Adults: The ELSA Cohort Study

BACKGROUND

Machine learning (ML) models can be used to predict future frailty in the community setting. However, outcome variables for epidemiologic data sets such as frailty usually have an imbalance between categories, that is, there are far fewer individuals classified as frail than as nonfrail, adversely affecting the performance of ML models when predicting the syndrome.

METHODS

A retrospective cohort study with participants (50 years or older) from the English Longitudinal Study of Ageing who were nonfrail at baseline (2008-2009) and reassessed for the frailty phenotype at 4-year follow-up (2012-2013). Social, clinical, and psychosocial baseline predictors were selected to predict frailty at follow-up in ML models (Logistic Regression, Random Forest [RF], Support Vector Machine, Neural Network, K-nearest neighbor, and Naive Bayes classifier).

RESULTS

Of all the 4 378 nonfrail participants at baseline, 347 became frail at follow-up. The proposed combined oversampling and undersampling method to adjust imbalanced data improved the performance of the models, and RF had the best performance, with areas under the receiver-operating characteristic curve and the precision-recall curve of 0.92 and 0.97, respectively, specificity of 0.83, sensitivity of 0.88, and balanced accuracy of 85.5% for balanced data. Age, chair-rise test, household wealth, balance problems, and self-rated health were the most important frailty predictors in most of the models trained with balanced data.

CONCLUSIONS

ML proved useful in identifying individuals who became frail over time, and this result was made possible by balancing the data set. This study highlighted factors that may be useful in the early detection of frailty.

Machine Learning Differentiation of Autism Spectrum Sub-Classifications

PURPOSE

Disorders on the autism spectrum have characteristics that can manifest as difficulties with communication, executive functioning, daily living, and more. These challenges can be mitigated with early identification. However, diagnostic criteria has changed from DSM-IV to DSM-5, which can make diagnosing a disorder on the autism spectrum complex. We evaluated machine learning to classify individuals as having one of three disorders of the autism spectrum under DSM-IV, or as non-spectrum.

METHODS

We employed machine learning to analyze retrospective data from 38,560 individuals. Inputs encompassed clinical, demographic, and assessment data.

RESULTS

The algorithm achieved AUROCs ranging from 0.863 to 0.980. The model correctly classified 80.5% individuals; 12.6% of individuals from this dataset were misclassified with another disorder on the autism spectrum.

CONCLUSION

Machine learning can classify individuals as having a disorder on the autism spectrum or as non-spectrum using minimal data inputs.

Exploring outdoor activity limitation (OAL) factors among older adults using interpretable machine learning

BACKGROUND

The occurrence of outdoor activity limitation (OAL) among older adults is influenced by multidimensional and confounding factors associated with aging.

AIM

The aim of this study was to apply interpretable machine learning (ML) to develop models for multidimensional aging constraints on OAL and identify the most predictive constraints and dimensions across multidimensional aging data.

METHODS

This study involved 6794 community-dwelling participants older than 65 from the National Health and Aging Trends Study (NHATS). Predictors included related to six dimensions: sociodemographics, health condition, physical capacity, neurological manifestation, daily living habits and abilities, and environmental conditions. Multidimensional interpretable machine learning models were assembled for model construction and analysis.

RESULTS

The multidimensional model demonstrated the best predictive performance (AUC: 0.918) compared to the six sub-dimensional models. Among the six dimensions, physical capacity had the most remarkable prediction (AUC: physical capacity: 0.895, daily habits and abilities: 0.828, physical health: 0.826, neurological performance: 0.789, sociodemographic: 0.773, and environment condition: 0.623). The top-ranked predictors were SPPB score, lifting ability, leg strength, free kneeling, laundry mode, self-rated health, age, attitude toward outdoor recreation, standing time on one foot with eyes open, and fear of falling.

DISCUSSION

Reversible and variable factors, which are higher in the set of high-contribution constraints, should be prioritized as the main contributing group in terms of interventions.

CONCLUSION

The integration of potentially reversible factors, such as neurological performance in addition to physical function into ML models, yields a more accurate assessment of OAL risk, which provides insights for targeted, sequential interventions for older adults with OAL.