Diagnostic accuracy of the STRATIFY clinical prediction rule for falls: a systematic review and meta-analysis

Validation of the stratify scale for the prediction of falls among hospitalized adults in a tertiary hospital in Colombia: a retrospective cohort study

The STRATIFY scale has been implemented as a preventive strategy for predicting the risk of accidental falls among hospitalized adults. However, there is still uncertainty about its accuracy. This study aimed to perform an external validation of the STRATIFY fall prediction scale in hospitalized adults in one tertiary care hospital in Bogotá, Colombia. The study was a retrospective cohort of adult hospitalized patients in a high-level complexity care hospital. The sample selected included admitted patients (age ≥ 18), consecutively by the institution between 2018 and 2020, with an evaluation of the fall risk measured by the STRATIFY score given to each at the time of hospital admission. For assessing the scale's feasibility, its discriminative capability was obtained by calculating sensitivity, specificity, likelihood ratios, predictive values, and area under the ROC curve. The evaluation included 93,347 patient hospital records (mean 56.9 years, 50.2% women). The overall sensitivity score was 0.672 [IC 95% 0.612-0.723], the specificity score was 0.612 [IC 95% 0.605-0.615], and the positive likelihood ratio was 1.73 [IC 95% 1.589-1.891]. The area under the ROC curve was 0.69 [IC 95% 0.66-0.72]. Subgroups of age obtained similar results. Applying the STRATIFY scale at hospital admission resulted in a lower performance of the tool-predict falls in hospitalized patients. It is necessary to implement an individual evaluation of the risk factors for falls in order to structure appropriate care plans to prevent and improve hospital safety.

Falls in Senior Adults: Demographics, Cost, Risk Stratification, and Evaluation

Falls occur at staggering rates across the country, with 25% of Americans over 65 reporting annual falls. The fall rate in Missourian older adults is 27.3%. Eighty-six percent of fall-related deaths happen over the age of 65. There are many intrinsic and extrinsic factors that contribute to falls, with some factors that can be targeted and optimized by physicians. There are nuances to the history and physical that can help physicians identify these risk factors.

Testing of Reliability and Validity of the Peninsula Health Falls Risk Assessment Tool (PHFRAT) in Acute Care: A Cross-Sectional Study

Purpose

The purpose of this study was to evaluate the validity and reliability of the Peninsula Health Fall Risk Assessment Tool (PHFRAT) in acute care in various medical specialties. The assessment has not been previously studied in acute care.

Methods

The cross-sectional study was conducted in a large acute care hospital system. The retrospective medical data were used to explore the validity of the PHFRAT. The data consisted of all adult inpatients (≥ 18 age) evaluated by the PHFRAT during 2014-2016 (n = 22,700). The Poisson regression, logistic regression, sensitivity, specificity, and the area under the ROC curve were evaluated. The data for the reliability study were collected in 2016 in twelve units by evaluating the patients (n = 359) twice using the PHFRAT. The prospective data were analyzed using Fleiss' Kappa, and the content validity index was also counted.

Results

In the somatic data, the change in the risk level from low risk to high risk increases the probability of falls by a factor of 2.8 (p<0.01). When the cut-off point was 9, sensitivity and specificity were 72% and 59%, respectively, and the area under the ROC curve was 0.67 (p<0.01). Validity varied by medical specialties. In the validity analysis, it was not possible to calculate the statistical significance from the psychiatry data. The inter-rater reliability was 0.68 (p<0.01).

Conclusion

This study shows that the PHFRAT proved to be moderately suitable for detecting the risk of falling for adult patients admitted to somatic units in acute care. The reliability of the PHFRAT was moderate. The results indicate the need to study the PHFRAT more broadly in psychiatric care as well as some specialties in somatic care.

Tailoring a Comprehensive Bundled Intervention for ED Fall Prevention

INTRODUCTION

Falls in the emergency department pose an important challenge for patient safety. Multifactorial fall prevention bundles have been associated with a reduction in patient falls in the inpatient setting. The purpose of this project was to tailor and implement a comprehensive fall prevention bundle in our emergency department.

METHODS

Fall bundle components for this intervention were selected on the basis of a review of fall prevention research and included fall risk assessment, safe ambulation, safe toileting, staff communication, early warning, and patient education. The fall risk assessment was tailored to the emergency department through an appraisal of select inpatient fall risk assessments, literature search for ED-specific fall risk factors, and a site-specific chart review, after which pertinent fall risk factors were integrated into a modified screening. Fall prevention materials that were both practical and applicable to the emergency department and facilitated patient safety along each bundle domain were selected for implementation at our site.

RESULTS

The tailored fall prevention bundle was championed by the interdisciplinary ED Fall Prevention Team and implemented over the course of 5 months in 1 emergency department. Education on fall prevention equipment was delivered in a peer-to-peer format, and an online module was designed to guide staff through the new fall risk assessment. The fall prevention bundle was adopted into clinical practice after staff education was completed, and the fall risk screening was merged into the electronic medical record.

DISCUSSION

ED fall prevention requires a comprehensive bundled approach, which includes a fall risk screening and multifactorial interventions that are tailored to the ED setting. Successful implementation relies on the involvement of front-line staff from the design through the delivery of the bundled fall prevention measures. Continued inquiry and innovation in ED fall prevention will help provide a safer health care environment and improve patient outcomes.

The diagnostic and predictive accuracy of the PRISMA-7 screening tool for frailty in older adults: A systematic review protocol

Background: Older adults are at risk of adverse outcomes due to frailty. A number of frailty screening instruments have been developed to identify older adults at increased risk of frailty. This systematic review and meta-analysis will look to examine the diagnostic accuracy of the Program of Research to Integrate the Services for the Maintenance of Autonomy 7 (PRISMA-7). Methods and analysis: A systematic literature search will be conducted from 2008-February 2020 in PubMed, EMBASE, CINAHL, EBSCO and the Cochrane Library to identify validation studies of the PRISMA-7 tool.  A pre-specified PRISMA-7 score of ≥3 (maximum score 7 points) will be used to identify frailty in older adults. Prospective or retrospective cohort studies, cross-sectional studies and the control arm of randomised controlled trials will be included that attempt to validate the diagnostic accuracy of the PRISMA-7 screening tool in older adults across all healthcare settings when compared to a reference standard. The predictive accuracy of the PRISMA-7 tool will also be explored. Study quality will be assessed by the QUADAS-2 tool. A bivariate random effects model will be used to generate pooled estimates of sensitivity and specificity. Statistical heterogeneity will be explored using validated methods. Ethics and dissemination: Formal ethical approval is not required as primary data will not be collected. The results will be disseminated through a peer-reviewed publication, conference presentation and the popular press. Protocol registration: Awaiting registration with the International Prospective Register for Systematic Reviews (PROSPERO).

The Trail Making test: a study of its ability to predict falls in the acute neurological in-patient population

Objective:

To determine whether tests of cognitive function and patient-reported outcome measures of motor function can be used to create a machine learning-based predictive tool for falls.

Design:

Prospective cohort study.

Setting:

Tertiary neurological and neurosurgical center.

Subjects:

In all, 337 in-patients receiving neurosurgical, neurological, or neurorehabilitation-based care.

Main Measures:

Binary (Y/N) for falling during the in-patient episode, the Trail Making Test (a measure of attention and executive function) and the Walk-12 (a patient-reported measure of physical function).

Results:

The principal outcome was a fall during the in-patient stay ( n = 54). The Trail test was identified as the best predictor of falls. Moreover, addition of other variables, did not improve the prediction (Wilcoxon signed-rank P < 0.001). Classical linear statistical modeling methods were then compared with more recent machine learning based strategies, for example, random forests, neural networks, support vector machines. The random forest was the best modeling strategy when utilizing just the Trail Making Test data (Wilcoxon signed-rank P < 0.001) with 68% (± 7.7) sensitivity, and 90% (± 2.3) specificity.

Conclusion:

This study identifies a simple yet powerful machine learning (Random Forest) based predictive model for an in-patient neurological population, utilizing a single neuropsychological test of cognitive function, the Trail Making test.

Reducing waste in evaluation studies on fall risk assessment tools for older people

OBJECTIVES

To critically appraise the recognition of methodological challenges in evaluation studies on assessment tools and nurses' clinical judgment on fall risk in older people and suggest how to reduce respective research waste.

STUDY DESIGN AND SETTING

Opinion article and narrative review covering systematic reviews on studies assessing diagnostic accuracy and impact of assessment tools and/or nurses' clinical judgment.

RESULTS

Eighteen reviews published in the last 15 years were analyzed. Only one reflects potentially important factors threatening the accuracy of assessments using delayed verification with fall events as reference after a certain period of time, that is, natural course, preventive measures, and treatment paradox where accurate assessment leads to prevention of falls, that is, influencing the reference standard and falsely indicating low diagnostic accuracy. In addition, only one review mentions randomized controlled trials as appropriate study design for the investigation of the impact of fall risk assessment tools on patient-important outcomes. Until now, only one randomized controlled trial dealing with this question has been performed showing no effect on falls and injuries. Instead of investigating the diagnostic accuracy of fall assessment tools, the focus of future research should be on the effectiveness of the implementation of fall assessment tools at reducing falls and injuries.

A Probabilistic Model to Investigate the Properties of Prognostic Tools for Falls

Background: Falls are a prevalent and burdensome problem in the elderly. Tools for the assessment of fall risk are fundamental for fall prevention. Clinical studies for the development and evaluation of prognostic tools for falls show high heterogeneity in the settings and in the reported results. Newly developed tools are susceptible to over- optimism.

Objectives: This study proposes a probabilistic model to address critical issues about fall prediction through the analysis of the properties of an ideal prognostic tool for falls.

Methods: The model assumes that falls occur within a population according to the Greenwood and Yule scheme for accident-proneness. Parameters for the fall rate distribution are estimated from counts of falls of four different epidemiological studies.

Results: We obtained analytic formulas and quantitative estimates for the predictive and discriminative properties of the ideal prognostic tool. The area under the receiver operating characteristic curve (AUC) ranges between about 0.80 and 0.89 when prediction on any fall is made within a follow-up of one year. Predicting on multiple falls results in higher AUC.

Conclusions: The discriminative ability of current validated prognostic tools for falls is sensibly lower than what the proposed ideal perfect tool achieves. A sensitivity analysis of the predictive and discriminative properties of the tool with respect to study settings and fall rate distribution identifies major factors that can account for the high heterogeneity of results observed in the literature.

Fall prevention: is the STRATIFY tool the right instrument in Italian Hospital inpatient? A retrospective observational study

BACKGROUND

Although several risk assessment tools are in use, uncertainties on their accuracy in detecting fall risk already exist. Choosing the most accurate tool for hospital inpatient is still a challenge for the organizations. We aimed to retrospectively assess the appropriateness of a fall risk prevention program with the STRATIFY assessment tool in detecting acute-care inpatient fall risk.

METHODS

Number of falls and near falls, occurred from January 2014 to March 2015, was collected through the incident reporting web-system implemented in the hospital's intranet. We reported whether the fall risk was assessed with the STRATIFY assessment tool and, if so, which was the judgement. Primary outcome was the proportion of inpatients identified as high risk of fall among inpatients who fell (True Positive Rate), and the proportion of inpatients identified as low-risk that experienced a fall howsoever (False Negative Rate). Characteristics of population and fall events were described among subgroups of low risk and high risk inpatients.

RESULTS

We collected 365 incident reports from 40 hospital units, 349 (95.6%) were real falls and 16 (4.4%) were near falls. The fall risk assessment score at patient's admission had been reported in 289 (79%) of the overall incident reports. Thus, 74 (20.3%) fallers were actually not assessed with the STRATIFY, even though the majority of them presented risk recommended to be assessed. The True Positive Rate was 35.6% (n = 101, 95% CI 30% - 41.1%). The False Negative Rate was 64.4% (n = 183, 95% CI 58.9%-70%) of fallers, nevertheless they incurred in a fall. The STRATIFY mean score was 1.3 ± 1.4; the median was 1 (IQQ 0-2).

CONCLUSIONS

The prevention program using only the STRATIFY tool was found to be not adequate to screen our inpatients population. The incorrect identification of patients' needs leads to allocate resources to erroneous priorities and to untargeted interventions, decreasing healthcare performance and quality.

Functional Measures for Fall Risk in the Acute Care Setting: A Review

This review explores the evidence pertaining to the use of functional ability measures for fall risk in the acute care setting. We included studies from six bibliographic databases that investigated fall risk functional ability measures in hospitalized older adults (≥55 years). We utilized the following search terms: acute care, subacute care, critical care, inpatient, fall, and fall prevention. Nineteen articles met the inclusion criteria. Timed "Up and Go" (TUG) was identified as a feasible fall risk functional ability measure for clinicians; it demonstrated clinical performance of fair sensitivity (56%-68%) and good specificity (74%-80%). Clinical performance of other measures (Berg Balance Scale and Functional Reach test) was not as favorable as the TUG. Functional ability measures are underutilized in the acute care setting, potentially due to limited knowledge and training on administration. Combining functional measures with subjective screening tools may optimize performance and accuracy of identifying fall risk identification.

Adverse outcomes in older adults attending emergency departments: a systematic review and meta-analysis of the Identification of Seniors At Risk (ISAR) screening tool

Background

older adults are frequent users of emergency services and demonstrate high rates of adverse outcomes following emergency care.

Objective

to perform a systematic review and meta-analysis of the Identification of Seniors At Risk (ISAR) screening tool, to determine its predictive value in identifying adults ≥65 years at risk of functional decline, unplanned emergency department (ED) readmission, emergency hospitalisation or death within 180 days after index ED visit/hospitalisation.

Methods

a systematic literature search was conducted in PubMed, EMBASE, CINAHL, EBSCO and the Cochrane Library to identify validation and impact analysis studies of the ISAR tool. A pre-specified ISAR score of ≥2 (maximum score 6 points) was used to identify patients at high risk of adverse outcomes. A bivariate random effects model generated pooled estimates of sensitivity and specificity. Statistical heterogeneity was explored and methodological quality was assessed using validated criteria.

Results

thirty-two validation studies (n = 12,939) are included. At ≥2, the pooled sensitivity of the ISAR for predicting ED return, emergency hospitalisation and mortality at 6 months is 0.80 (95% confidence interval (CI) 0.70-0.87), 0.82 (95% CI 0.74-0.88) and 0.87 (95% CI 0.75-0.94), respectively, with a pooled specificity of 0.31 (95% CI 0.24-0.38), 0.32 (95% CI 0.24-0.41) and 0.35 (95% CI 0.26-0.44). Similar values are demonstrated at 30 and 90 days. Three heterogeneous impact analysis studies examined the clinical implementation of the ISAR and reported mixed findings across patient and process outcomes.

Conclusion

the ISAR has modest predictive accuracy and may serve as a decision-making adjunct when determining which older adults can be safely discharged.

Speed, resistance, and unexpected accelerations modulate feed forward and feedback control during a novel weight bearing task

We developed a method to investigate feed-forward and feedback movement control during a weight bearing visuomotor knee tracking task. We hypothesized that a systematic increase in speed and resistance would show a linear decrease in movement accuracy, while unexpected perturbations would induce a velocity-dependent decrease in movement accuracy. We determined the effects of manipulating the speed, resistance, and unexpected events on error during a functional weight bearing task. Our long term objective is to benchmark neuromuscular control performance across various groups based on age, injury, disease, rehabilitation status, and/or training. Twenty-six healthy adults between the ages of 19-45 participated in this study. The study involved a single session using a custom designed apparatus to perform a single limb weight bearing task under nine testing conditions: three movement speeds (0.2, 0.4, and 0.6Hz) in combination with three levels of brake resistance (5%, 10%, and 15% of individual's body weight). Individuals were to perform the task according to a target with a fixed trajectory across all speeds, corresponding to a∼0 (extension) to 30° (flexion) of knee motion. An increase in error occurred with speed (p<0.0001, effect size (eta²): η²=0.50) and resistance (p<0.0001, η²=0.01). Likewise, during unexpected perturbations, the ratio of perturbed/non-perturbed error increased with each increment in velocity (p<0.0014, η²=0.08), and resistance (p<0.0001, η²=0.11). The hierarchical framework of these measurements offers a standardized functional weight bearing strategy to assess impaired neuro-muscular control and/or test the efficacy of therapeutic rehabilitation interventions designed to influence neuromuscular control of the knee.

Predictive Performance of a Fall Risk Assessment Tool for Community-Dwelling Older People (FRAT-up) in 4 European Cohorts

BACKGROUND AND OBJECTIVE

The fall risk assessment tool (FRAT-up) is a tool for predicting falls in community-dwelling older people based on a meta-analysis of fall risk factors. Based on the fall risk factor profile, this tool calculates the individual risk of falling over the next year. The objective of this study is to evaluate the performance of FRAT-up in predicting future falls in multiple cohorts.

METHODS

Information about fall risk factors in 4 European cohorts of older people [Activity and Function in the Elderly (ActiFE), Germany; English Longitudinal Study of Aging (ELSA), England; Invecchiare nel Chianti (InCHIANTI), Italy; Irish Longitudinal Study on Aging (TILDA), Ireland] was used to calculate the FRAT-up risk score in individual participants. Information about falls that occurred after the assessment of the risk factors was collected from subsequent longitudinal follow-ups. We compared the performance of FRAT-up against those of other prediction models specifically fitted in each cohort by calculation of the area under the receiver operating characteristic curve (AUC).

RESULTS

The AUC attained by FRAT-up is 0.562 [95% confidence interval (CI) 0.530-0.594] for ActiFE, 0.699 (95% CI 0.680-0.718) for ELSA, 0.636 (95% CI 0.594-0.681) for InCHIANTI, and 0.685 (95% CI 0.660-0.709) for TILDA. Mean FRAT-up AUC as estimated from meta-analysis is 0.646 (95% CI 0.584-0.708), with substantial heterogeneity between studies. In each cohort, FRAT-up discriminant ability is surpassed, at most, by the cohort-specific risk model fitted on that same cohort.

CONCLUSIONS

We conclude that FRAT-up is a valid approach to estimate risk of falls in populations of community-dwelling older people. However, further studies should be performed to better understand the reasons for the observed heterogeneity across studies and to refine a tool that performs homogeneously with higher accuracy measures across different populations.

The Diagnostic Accuracy of the Berg Balance Scale in Predicting Falls

This study aimed to evaluate the predictive validity of the Berg Balance Scale (BBS) as a screening tool for fall risks among those with varied levels of balance. A total of 21 studies reporting predictive validity of the BBS of fall risk were meta-analyzed. With regard to the overall predictive validity of the BBS, the pooled sensitivity and specificity were 0.72 and 0.73, respectively; the accuracy curve area was 0.84. The findings showed statistical heterogeneity among studies. Among the sub-groups, the age group of those younger than 65 years, those with neuromuscular disease, those with 2+ falls, and those with a cutoff point of 45 to 49 showed better sensitivity with statistically less heterogeneity. The empirical evidence indicates that the BBS is a suitable tool to screen for the risk of falls and shows good predictability when used with the appropriate criteria and applied to those with neuromuscular disease.

A systematic review and meta-analysis of the diagnostic accuracy of MRI for suspected ACL and meniscal tears of the knee

PURPOSE

To determine the diagnostic accuracy of magnetic resonance imaging (MRI) and ultrasound (US) in the diagnosis of anterior cruciate ligament (ACL), medial meniscus and lateral meniscus tears in people with suspected ACL and/or meniscal tears.

METHODS

MEDLINE, Web of Science and the Cochrane library were searched from inception to March 2014. All prospective studies of the diagnostic accuracy of MRI or US against arthroscopy as the reference standard were included in the systematic review. Studies with a retrospective design and those with evidence of verification bias were excluded. Methodological quality of included studies was assessed using the QUADAS-2 tool. A meta-analysis of studies evaluating MRI to calculate the pooled sensitivity and specificity for each target condition was performed using a bivariate model with random effects. Sub-group and sensitivity analysis were used to examine the effect of methodological and other study variables.

RESULTS

There were 14 studies included in the meta-analysis of the accuracy of MRI for ACL tears, 19 studies included for medial meniscal tears and 19 studies for lateral meniscal tears. The summary estimates of sensitivity and specificity of MRI were 87 % (95 % CI 77-94 %) and 93 % (95 % CI 91-96 %), respectively, for ACL tears; 89 % (95 % CI 83-94 %) and 88 % (95 % CI 82-93 %), respectively, for medial meniscal tears; and 78 % (95 % CI 66-87 %) and 95 % (95 % CI 91-97 %), respectively, for lateral meniscal tears. Magnetic field strength had no significant effect on accuracy. Most studies had a high or unclear risk of bias. There were an insufficient number of studies that evaluated US to perform a meta-analysis.

CONCLUSION

This study provides a systematic review and meta-analysis of diagnostic accuracy studies of MRI and applies strict exclusion criteria in relation to the risk of verification bias. The risk of bias in most studies is high or unclear in relation to the reference standard. Concerns regarding the applicability of patient selection are also present in most studies.

LEVEL OF EVIDENCE

III.

Falls and Fractures: A systematic approach to screening and prevention

Falls are one of the major causes of mortality and morbidity in older adults. Every year, an estimated 30-40% of patients over the age of 65 will fall at least once. Falls lead to moderate to severe injuries, fear of falling, loss of independence and death in a third of those patients. Falls account for 87 % of all fractures in the elderly. These fractures are almost always due to low impact injuries in osteoporotic bones. Several organizations have recommended screening older patients to identify those with a high risk of falling and, or fractures. The present review provides a brief summary and update of the relevant literature, summarizing screening tools and interventions to prevent falls and fractures. The major risk factors identified are impaired balance and gait, polypharmacy, and history of previous falls. Other risk factors include advancing age, female gender, visual impairments, cognitive decline especially attention and executive dysfunction, and environmental factors. Recommendations for the clinician to screen and prevent falls in older patients are also summarized.

Is the Timed Up and Go test a useful predictor of risk of falls in community dwelling older adults: a systematic review and meta-analysis

Background

The Timed Up and Go test (TUG) is a commonly used screening tool to assist clinicians to identify patients at risk of falling. The purpose of this systematic review and meta-analysis is to determine the overall predictive value of the TUG in community-dwelling older adults.

Methods

A literature search was performed to identify all studies that validated the TUG test. The methodological quality of the selected studies was assessed using the QUADAS-2 tool, a validated tool for the quality assessment of diagnostic accuracy studies. A TUG score of ≥13.5 seconds was used to identify individuals at higher risk of falling. All included studies were combined using a bivariate random effects model to generate pooled estimates of sensitivity and specificity at ≥13.5 seconds. Heterogeneity was assessed using the variance of logit transformed sensitivity and specificity.

Results

Twenty-five studies were included in the systematic review and 10 studies were included in meta-analysis. The TUG test was found to be more useful at ruling in rather than ruling out falls in individuals classified as high risk (>13.5 sec), with a higher pooled specificity (0.74, 95% CI 0.52-0.88) than sensitivity (0.31, 95% CI 0.13-0.57). Logistic regression analysis indicated that the TUG score is not a significant predictor of falls (OR = 1.01, 95% CI 1.00-1.02, p = 0.05).

Conclusion

The Timed Up and Go test has limited ability to predict falls in community dwelling elderly and should not be used in isolation to identify individuals at high risk of falls in this setting.

Uncertainty in clinical prediction rules: the value of credible intervals

Decision making in physical therapy is increasingly informed by evidence in the form of probabilities. Prior beliefs concerning diagnoses, prognoses, and treatment effects are quantitatively revised by the integration of new information derived from the history, physical examination, and other investigations in a well-recognized application of Bayes' theorem. Clinical prediction rule development studies commonly employ such methodology to produce quantified estimates of the likelihood of patients having certain diagnoses or achieving given outcomes. To date, the physical therapy literature has been limited to the discussion and calculation of the point estimate of such probabilities. The degree of precision associated with the construction of posterior probabilities, which requires consideration of both uncertainty associated with pretest probability and uncertainty associated with test accuracy, remains largely unrecognized and unreported. This paper provides an introduction to the calculation of the uncertainty interval, known as a credible interval, around posterior probability estimates. The method for calculating the credible interval is detailed and illustrated with example data from 2 clinical prediction rule development studies. Two relatively quick and simple methods for approximating the credible interval are also outlined. It is anticipated that knowledge of the credible interval will have practical implications for the incorporation of probabilistic evidence in clinical practice. Consistent with reporting standards for interventional and diagnostic studies, it is equally appropriate that studies reporting posterior probabilities calculate and report the level of precision associated with these point estimates.