Recurrent events counted in evaluations of predictive accuracy

The use and predictive performance of the Peninsula Health Falls Risk Assessment Tool (PH-FRAT) in 25 residential aged care facilities: a retrospective cohort study using routinely collected data

BACKGROUND

The Peninsula Health Falls Risk Assessment Tool (PH-FRAT) is a validated and widely applied tool in residential aged care facilities (RACFs) in Australia. However, research regarding its use and predictive performance is limited. This study aimed to determine the use and performance of PH-FRAT in predicting falls in RACF residents.

METHODS

A retrospective cohort study using routinely-collected data from 25 RACFs in metropolitan Sydney, Australia from Jul 2014-Dec 2019. A total of 5888 residents aged ≥65 years who were assessed at least once using the PH-FRAT were included in the study. The PH-FRAT risk score ranges from 5 to 20 with a score > 14 indicating fallers and ≤ 14 non-fallers. The predictive performance of PH-FRAT was determined using metrics including area under receiver operating characteristics curve (AUROC), sensitivity, specificity, sensitivity_{Event Rate(ER)} and specificity_ER.

RESULTS

A total of 27,696 falls were reported over 3,689,561 resident days (a crude incident rate of 7.5 falls /1000 resident days). A total of 38,931 PH-FRAT assessments were conducted with a median of 4 assessments per resident, a median of 43.8 days between assessments, and an overall median fall risk score of 14. Residents with multiple assessments had increased risk scores over time. The baseline PH-FRAT demonstrated a low AUROC of 0.57, sensitivity of 26.0% (sensitivity_ER 33.6%) and specificity of 88.8% (specificity_ER 82.0%). The follow-up PH-FRAT assessments increased sensitivity_ER values although the specificity_ER decreased. The performance of PH-FRAT improved using a lower risk score cut-off of 10 with AUROC of 0.61, sensitivity of 67.5% (sensitivity_ER 74.4%) and specificity of 55.2% (specificity_ER 45.6%).

CONCLUSIONS

Although PH-FRAT is frequently used in RACFs, it demonstrated poor predictive performance raising concerns about its value. Introducing a lower PH-FRAT cut-off score of 10 marginally enhanced its predictive performance. Future research should focus on understanding the feasibility and accuracy of dynamic fall risk predictive tools, which may serve to better identify residents at risk of falls.

Fall risk assessment tools for use among older adults in long-term care settings: A systematic review of the literature

OBJECTIVE

To conduct a systematic review of published fall risk assessment tools (FRATs) tested for predictive validity among older adults in long-term care (LTC).

METHODS

A systematic search was conducted using five databases. Only studies reporting on sensitivity and specificity values, conducted in LTC on populations primarily aged over 60 years, were considered.

RESULTS

Fifteen papers were included and three different categories of FRATs emerged: multifactorial assessment tools, functional mobility assessments and algorithms. Several FRATs showed moderate-to-good predictive validity and reliability, with the Modified Fall Assessment Tool and the Peninsula Health Falls Risk Assessment Tool (PHFRAT) also demonstrating good feasibility.

CONCLUSION

Evidence for the best choice of FRAT for use in LTC remains limited. Further research is warranted for the PHFRAT, recommended for use in LTC by best practice guidelines, before its establishment as the tool of choice for these clinical settings.

Adding value to the STRATIFY falls risk assessment in acute hospitals

AIM

This paper is a report of a study conducted to compare the predictive accuracy for fallers of The Northern Hospital Modified St Thomas's Risk Assessment Tool and St Thomas's Risk Assessment Tool, and to determine the inter-rater agreement of each tool.

BACKGROUND

Falls risk assessment is a key component of fall prevention. Investigation of clinimetric properties of a tool should be completed before it are used in clinical practice.

METHODS

Local falls data were used to inform modification of the St Thomas's Risk Assessment Tool to improve faller prediction. Clinimetric properties of the St Thomas's Risk Assessment Tool and The Northern Hospital Modified St Thomas's Risk Assessment Tool were examined in a prospective cross-sectional study with acute hospital patients. Phase I involved assessment of predictive accuracy using sensitivity, specificity and the Youden Index (J) with 263 patients. Phase II of the evaluation involved assessment of inter-rater agreement using the Kappa statistic (κ) with 52 patients. Data were collected in 2008.

RESULTS

Impaired balance, age 80 years and older and drug and alcohol problems were identified as additional falls risk factors in The Northern Hospital population and added to the St Thomas's Risk Assessment Tool. The Northern Hospital Modified St Thomas's Risk Assessment Tool had higher sensitivity (0·65 vs. 0·35, P = 0·016). The St Thomas's Risk Assessment Tool had higher specificity (0·93 vs. 0·79, P = 0·000). The Northern Hospital Modified St Thomas's Risk Assessment Tool had the greater overall accuracy (J) (0·44 vs. 0·28, P = 0·006). Inter-rater agreement of The Northern Hospital Modified St Thomas's Risk Assessment Tool was fair (κ = 0·34) and low for the St Thomas's Risk Assessment Tool (κ = 0·19).

CONCLUSION

The Northern Hospital Modified St Thomas's Risk Assessment Tool and St Thomas's Risk Assessment Tool accurately identified patients at risk of falling. The Northern Hospital Modified St Thomas's Risk Assessment Tool was more accurate. Tools which have unknown validity and reliability should not be used. Future research is needed to provide evidence that use of falls risk assessments has a positive impact on reducing patient falls.

World Health Organization fracture risk assessment tool in the assessment of fractures after falls in hospital

Background

Falls are very common accidents in a hospital. Various risk factors and risk assessment tools are used to predict falls. However, outcomes of falls such as bone fractures have not been considered in these risk assessment tools, and the performance of risk assessment tools in a Japanese hospital setting is not clear.

Methods

This was a retrospective single-institution study of 20,320 inpatients aged from 40 to 90 years who were admitted to a tertiary-care university hospital during the period from April 2006 to March 2009. Possible risk factors for falls and fractures including STRATIFY score and FRAX™ score and information on falls and their outcome were obtained from the hospital information system. The datasets were divided randomly into a development dataset and a test dataset. The chi-square test, logistic regression analysis and survival analysis were used to identify risk factors for falls and fractures after falls.

Results

Fallers accounted for 3.1% of the patients in the development dataset and 3.5% of the patients in the test dataset, and 2.6% and 2.9% of the fallers in those datasets suffered peripheral fractures. Sensitivity and specificity of the STRATIFY score to predict falls were not optimal. Most of the known risk factors for falls had no power to predict fractures after falls. Multiple logistic analysis and multivariate Cox's regression analysis with time-dependent covariates revealed that FRAX™ score was significantly associated with fractures after falls.

Conclusions

Risk assessment tools for falls are not appropriate for predicting fractures after falls. FRAX™ might be a useful tool for that purpose. The performance of STRATIFY to predict falls in a Japanese hospital setting was similar to that in previous studies.

Four-item fall risk screening tool for subacute and residential aged care: The first step in fall prevention

AIM

To report the reliability, accuracy and compliance of a brief fall risk screening tool in subacute and residential aged care.

METHOD

A 9-item tool, developed by expert and literature review, was administered to 291 persons admitted to subacute and residential aged care at Peninsula Health (PH) Victoria, Australia. Items were analysed for their ability to predict falls and the four strongest incorporated into a screening tool. Reliability was assessed on six nurses.

RESULTS

Most predictive items were recent falls (0.82), psychological status (0.55), medications (0.46) and cognition (0.41) chi(2) (4, n= 291) = 89.89, P < 0.0001. The final 4-item tool (PH-FRAT) provides 80% accuracy (sensitivity(ER) 70.2%, specificity(ER) 68.8%) and high reliability (ICC = 0.79). The PH-FRAT is now used in 50 local subacute and residential facilities.

CONCLUSION

The 4-item PH-FRAT is a popular, moderately predictive, reliable and brief method of screening fall risk in subacute and residential aged care.

Predicting accidental falls in people with multiple sclerosis -- a longitudinal study

OBJECTIVE

To investigate accidental falls and near fall incidents in people with multiple sclerosis with respect to clinical variables and the predictive values of four tests.

DESIGN

A longitudinal, multi-centred cohort study with prospectively collected falls.

PROCEDURES

Self-reported incidents during the three months following a standardized test procedure.

SUBJECTS

Seventy-six people with multiple sclerosis and an Expanded Disability Status Scale score between 3.5 and 6.0.

MAIN OUTCOME MEASURES

Berg Balance Scale, Timed Up and Go cognitive, Four Square Step Test (FSST) and 12-item Multiple Sclerosis Walking Scale.

RESULTS

Forty-eight people (63%) registered 270 falls. Most falls occurred indoors during activities of daily life. We found a correlation of r(s)=0.57 between near falls and falls, and of r(s) = 0.82 between registered and retrospectively recalled falls. Fallers and non-fallers differed significantly regarding Expanded Disability Status Score (odds ratio (OR) 1.99, 95% confidence interval (CI) 1.22; 3.40), spasticity (OR 1.14, CI 1.02; 1.31), proprioception (OR 2.50, CI 1.36; 5.12) and use of walking aids (OR 2.27, CI 1.23; 4.37). Reported use of walking aids both indoors and outdoors increased the odds of falling fivefold while disturbed proprioception increased the odds 2.5-15.6 times depending on severity. The odds of falling were doubled for each degree of increased Expanded Disability Status Score and more than doubled for each degree of increased spasticity. The Berg Balance Scale, use of walking aids and Timed Up and Go cognitive best identified fallers (73-94%) and proprioception, Expanded Disability Status Score, 12-item Multiple Sclerosis Walking Scale and Four Square Step Test best identified non-fallers (75-93%).

CONCLUSIONS

In clinical practice, looking at the use of walking aids, investigating proprioception and spasticity, rating Expanded Disability Status Score and using Berg Balance Scale or Timed Up and Go cognitive all contribute when identifying fallers.

Cost-effectiveness analysis of screening for risk of in-hospital falls using physiotherapist clinical judgement

INTRODUCTION

Screening hospital patients for falls risk is now a contentious component of geriatric care despite its widespread clinical use. The economic implications of using a falls risk screening approach to deliver an effective falls prevention intervention have not previously been examined.

METHODS

This was a multicenter prospective longitudinal cohort and incremental cost-effectiveness analysis.

PARTICIPANTS/SETTING

One thousand one hundred twenty-three geriatric inpatients from 17 rehabilitation units across Australia.

MEASURES

Physiotherapist accuracy in predicting patient who will fall was captured with the question "Will this patient experience one or more falls during their rehabilitation period?" Falls were measured using hospital incident reporting systems.

PROCEDURE

The multicenter longitudinal cohort was undertaken to establish the predictive accuracy of physiotherapist clinical judgement. This data was used in the incremental cost-effectiveness analysis where estimates of the cost of falls and effectiveness of an intervention program were taken from previous research.

RESULTS

The accuracy of physiotherapist clinical judgement in predicting falls was high relative to previous research (sensitivity = 0.61, specificity = 0.82, Youden index = 0.43). Selectively providing patient falls-prevention education using physiotherapist clinical judgement would reduce falls [2.2 (SD: 0.19) fallers per 100 inpatients reduction] and reduce resources spent on trying to prevent and treat injuries from in-hospital falls [$2704 AUD (SD: $432) per 100 inpatients reduction] compared with doing nothing. However, there was greater uncertainty as to whether the patient education intervention modeled should be provided selectively or universally.

CONCLUSIONS

Preventing in-hospital falls using a targeted falls prevention intervention approach utilizing physiotherapist clinical judgement was more cost-effective than a "no intervention" approach.

Balance impairment not predictive of falls in geriatric rehabilitation wards

BACKGROUND

Falls are common among hospital inpatients, particularly in rehabilitation wards. Standing balance impairment is widely held to be a contributing factor to falls, is a component of several falls risk screening tools, and has motivated the development of balance retraining programs for the reduction of in-hospital falls. Little rigorous investigation of the link between standing balance impairment and in-hospital falls has been undertaken.

METHODS

We identified optimal cut-off points of four commonly used balance measures (functional reach, Timed Up and Go, step test, and timed static stance) in a prospective multicenter cohort study. Admission data (n = 1373) were clustered and matched by center then randomly allocated to development and validation data sets.

RESULTS

Optimal cut-off points for each test were identified from the development data set. The predictive accuracy of all four balance tests was poor when the optimal cut-off was applied to the validation data set (Youden Index scores ranged between 0.02 and 0.15).

CONCLUSIONS

These findings do not support an association between admission standing balance and falls in a geriatric rehabilitation setting. This result has implications for content of falls risk screening tools and interventions to prevent falls in a geriatric rehabilitation population.

Design-Related Bias in Hospital Fall Risk Screening Tool Predictive Accuracy Evaluations: Systematic Review and Meta-Analysis

INTRODUCTION

Fall risk screening tools are frequently used as a part of falls prevention programs in hospitals. Design-related bias in evaluations of tool predictive accuracy could lead to overoptimistic results, which would then contribute to program failure in practice.

METHODS

A systematic review was undertaken. Two blind reviewers assessed the methodology of relevant publications into a four-point classification system adapted from multiple sources. The association between study design classification and reported results was examined using linear regression with clustering based on screening tool and robust variance estimates with point estimates of Youden Index (= sensitivity + specificity - 1) as the dependent variable. Meta-analysis was then performed pooling data from prospective studies.

RESULTS

Thirty-five publications met inclusion criteria, containing 51 evaluations of fall risk screening tools. Twenty evaluations were classified as retrospective validation evaluations, 11 as prospective (temporal) validation evaluations, and 20 as prospective (external) validation evaluations. Retrospective evaluations had significantly higher Youden Indices (point estimate [95% confidence interval]: 0.22 [0.11, 0.33]). Pooled Youden Indices from prospective evaluations demonstrated the STRATIFY, Morse Falls Scale, and nursing staff clinical judgment to have comparable accuracy.

DISCUSSION

Practitioners should exercise caution in comparing validity of fall risk assessment tools where the evaluation has been limited to retrospective classifications of methodology. Heterogeneity between studies indicates that the Morse Falls Scale and STRATIFY may still be useful in particular settings, but that widespread adoption of either is unlikely to generate benefits significantly greater than that of nursing staff clinical judgment.