Predictive risk stratification model: a randomised stepped-wedge trial in primary care (PRISMATIC)

A Predefined Rule-Based Multi-Factor Risk Stratification Is Associated With Improved Outcomes at a Rural Primary Care Practice

This study built a predefined rule-based risk stratification paradigm using 19 factors in a primary care setting that works with rural communities. The factors include medical and nonmedical variables. The nonmedical variables represent 3 demographic attributes and one other factor represents transportation availability. Medical variables represent major clinical variables such as blood pressure and BMI. Many risk stratification models are found in the literature but few integrate medical and nonmedical variables, and to our knowledge, no such model is designed specifically for rural communities. The data used in this study contain the associated variables of all medical visits in 2021. Data from 2022 were used to evaluate the model. After our risk stratification model and several interventions were adopted in 2022, the percentage of patients with high or medium risk of deteriorating health outcomes dropped from 34.9% to 24.4%, which is a reduction of 30%. The medium-complex patient population size, which had been 29% of all patients, decreased by about 4% to 5.7%. According to the analysis, the total risk score showed a strong correlation with 3 risk factors: dual diagnoses, the number of seen providers, and PHQ9 (0.63, 0.54, and 0.45 correlation coefficients, respectively).

Promising algorithms to perilous applications: a systematic review of risk stratification tools for predicting healthcare utilisation

OBJECTIVES

Risk stratification tools that predict healthcare utilisation are extensively integrated into primary care systems worldwide, forming a key component of anticipatory care pathways, where high-risk individuals are targeted by preventative interventions. Existing work broadly focuses on comparing model performance in retrospective cohorts with little attention paid to efficacy in reducing morbidity when deployed in different global contexts. We review the evidence supporting the use of such tools in real-world settings, from retrospective dataset performance to pathway evaluation.

METHODS

A systematic search was undertaken to identify studies reporting the development, validation and deployment of models that predict healthcare utilisation in unselected primary care cohorts, comparable to their current real-world application.

RESULTS

Among 3897 articles screened, 51 studies were identified evaluating 28 risk prediction models. Half underwent external validation yet only two were validated internationally. No association between validation context and model discrimination was observed. The majority of real-world evaluation studies reported no change, or indeed significant increases, in healthcare utilisation within targeted groups, with only one-third of reports demonstrating some benefit.

DISCUSSION

While model discrimination appears satisfactorily robust to application context there is little evidence to suggest that accurate identification of high-risk individuals can be reliably translated to improvements in service delivery or morbidity.

CONCLUSIONS

The evidence does not support further integration of care pathways with costly population-level interventions based on risk prediction in unselected primary care cohorts. There is an urgent need to independently appraise the safety, efficacy and cost-effectiveness of risk prediction systems that are already widely deployed within primary care.

Statistical analysis of publicly funded cluster randomised controlled trials: a review of the National Institute for Health Research Journals Library

BACKGROUND

In cluster randomised controlled trials (cRCTs), groups of individuals (rather than individuals) are randomised to minimise the risk of contamination and/or efficiently use limited resources or solve logistic and administrative problems. A major concern in the primary analysis of cRCT is the use of appropriate statistical methods to account for correlation among outcomes from a particular group/cluster. This review aimed to investigate the statistical methods used in practice for analysing the primary outcomes in publicly funded cluster randomised controlled trials, adherence to the CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines for cRCTs and the recruitment abilities of the cluster trials design.

METHODS

We manually searched the United Kingdom's National Institute for Health Research (NIHR) online Journals Library, from 1 January 1997 to 15 July 2021 chronologically for reports of cRCTs. Information on the statistical methods used in the primary analyses was extracted. One reviewer conducted the search and extraction while the two other independent reviewers supervised and validated 25% of the total trials reviewed.

RESULTS

A total of 1942 reports, published online in the NIHR Journals Library were screened for eligibility, 118 reports of cRCTs met the initial inclusion criteria, of these 79 reports containing the results of 86 trials with 100 primary outcomes analysed were finally included. Two primary outcomes were analysed at the cluster-level using a generalized linear model. At the individual-level, the generalized linear mixed model was the most used statistical method (80%, 80/100), followed by regression with robust standard errors (7%) then generalized estimating equations (6%). Ninety-five percent (95/100) of the primary outcomes in the trials were analysed with appropriate statistical methods that accounted for clustering while 5% were not. The mean observed intracluster correlation coefficient (ICC) was 0.06 (SD, 0.12; range, - 0.02 to 0.63), and the median value was 0.02 (IQR, 0.001-0.060), although 42% of the observed ICCs for the analysed primary outcomes were not reported.

CONCLUSIONS

In practice, most of the publicly funded cluster trials adjusted for clustering using appropriate statistical method(s), with most of the primary analyses done at the individual level using generalized linear mixed models. However, the inadequate analysis and poor reporting of cluster trials published in the UK is still happening in recent times, despite the availability of the CONSORT reporting guidelines for cluster trials published over a decade ago.

Emergency admission risk stratification tools in UK primary care: a cross-sectional survey of availability and use

Background

Stratifying patient populations by risk of adverse events was believed to support preventive care for those identified, but recent evidence does not support this. Emergency admission risk stratification (EARS) tools have been widely promoted in UK policy and GP contracts.

Aim

To describe availability and use of EARS tools across the UK, and identify factors perceived to influence implementation.

Design and setting

Cross-sectional survey in UK.

Method

Online survey of 235 organisations responsible for UK primary care: 209 clinical commissioning groups (CCGs) in England; 14 health boards in Scotland; seven health boards in Wales; and five local commissioning groups (LCGs) in Northern Ireland. Analysis results are presented using descriptive statistics for closed questions and by theme for open questions.

Results

Responses were analysed from 171 (72.8%) organisations, of which 148 (86.5%) reported that risk tools were available in their areas. Organisations identified 39 different EARS tools in use. Promotion by NHS commissioners, involvement of clinical leaders, and engagement of practice managers were identified as the most important factors in encouraging use of tools by general practices. High staff workloads and information governance were identified as important barriers. Tools were most frequently used to identify individual patients, but also for service planning. Nearly 40% of areas using EARS tools reported introducing or realigning services as a result, but relatively few reported use for service evaluation.

Conclusion

EARS tools are widely available across the UK, although there is variation by region. There remains a need to align policy and practice with research evidence.

Hospital admissions after vertical integration of general practices with an acute hospital: a retrospective synthetic matched controlled database study

Background

New healthcare models are being explored to enhance care coordination, efficiency, and outcomes. Evidence is scarce regarding the impact of vertical integration of primary and secondary care on emergency department (ED) attendances, unplanned hospital admissions, and readmissions.

Aim

To examine the impact of vertical integration of an NHS provider hospital and 10 general practices on unplanned hospital care

Design and setting

A retrospective database study using synthetic controls of an NHS hospital in Wolverhampton integrated with 10 general practices, providing primary medical services for 67 402 registered patients.

Method

For each vertical integration GP practice, a synthetic counterpart was constructed. The difference in rate of ED attendances, unplanned hospital admissions, and unplanned hospital readmissions was compared, and pooled across vertical integration practices versus synthetic control practices pre-intervention versus post-intervention.

Results

Across the 10 practices, pooled rates of ED attendances did not change significantly after vertical integration. However, there were statistically significant reductions in the rates of unplanned hospital admissions (−0.11, 95% CI = −0.18 to −0.045, P = 0.0012) and unplanned hospital readmissions (−0.021, 95% CI = −0.037 to −0.0049, P = 0.012), per 100 patients per month. These effect sizes represent 888 avoided unplanned hospital admissions and 168 readmissions for a population of 67 402 patients per annum. Utilising NHS reference costs, the estimated savings from the reductions in unplanned care are ∼£1.7 million.

Conclusion

Vertical integration was associated with a reduction in the rate of unplanned hospital admissions and readmissions in this study. Further work is required to understand the mechanisms involved in this complex intervention, to assess the generalisability of these findings, and to determine the impact on patient satisfaction, health outcomes, and GP workload.

Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial

OBJECTIVE

To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments.

STUDY DESIGN AND SETTING

A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness.

RESULTS

Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs.

CONCLUSION

Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.

The impact of a combinatorial digital and organisational intervention on the management of long-term conditions in UK primary care: a non-randomised evaluation

BACKGROUND

Better management of long-term conditions remains a policy priority, with a focus on improving outcomes and reducing use of expensive hospital services. A number of interventions have been tested, but many have failed to show benefit in rigorous comparative research. In 2016, the NHS Test Beds scheme was launched to implement and test interventions combining digital technologies and pathway redesign in routine health care settings, with each intervention comprising multiple innovations to better realise benefit from their 'combinatorial' effect. We present the evaluation of one of the NHS Test Beds, which combined risk stratification algorithms, practice-based quality improvement and health monitoring and coaching to improve management of long-term conditions in a single health economy in the north-west of England.

METHODS

The NHS Test Bed was implemented in one clinical commissioning group in the north-west of England (patient population 235,800 served by 36 general practices). Routine administrative data on hospital use (the primary outcome) and a selection of secondary outcomes (data from both hospital and primary care) were collected in the intervention site, and from a comparator area in the same region. We used difference-in-differences analysis to compare outcomes in the NHS Test Bed area and the comparator after initiation of the combinatorial intervention.

RESULTS

Tests confirmed the existence of parallel trends in the intervention and comparator sites for hospital outcomes for the period April 2016 to March 2017, and for some of the planned primary care outcomes. Based on 10 months of post-intervention secondary care data and 13 months post-intervention primary care data, we found no significant impact on primary outcomes between the intervention and comparator site, and a significant impact on only one secondary outcome.

CONCLUSION

A combinatorial digital and organisational intervention to improve the management of long-term conditions was implemented across a whole health economy, but we found no evidence of a positive impact on health care utilisation outcomes in hospital and primary care.