Clinical Prediction Rules: Challenges, Barriers, and Promise

Systematic review for the prevention and management of falls and fear of falling in patients with Parkinson's disease

OBJECTIVE

To synthesize recent empirical evidence for the prevention and management of falls and fear of falling in patients with Parkinson's disease (PD).

DATA SOURCE

Database from PubMed, Cochrane Library, and EMBASE.

STUDY DESIGN

Systematic review.

DATA COLLECTION

We searched the PubMed, Cochrane Library, and EMBASE databases for studies published from inception to February 27, 2021. Inclusion criteria were nonreview articles on prevention and management measures related to falls and fall prevention in Parkinson's disease patients.

PRINCIPAL FINDINGS

We selected 45 articles and conducted in-depth research and discussion. According to the causes of falls in PD patients, they were divided into five directions, namely physical status, pre-existing conditions, environment, medical care, and cognition. In the cognitive domain, we focused on the fear of falling. On the above basis, we constructed a fall prevention model, which is a tertiary prevention health care network, based on The Johns Hopkins Fall Risk Assessment Tool to provide ideas for the prevention and management of falling and fear of falling in PD patients in clinical practice CONCLUSIONS: Falls and fear of falls in patients with Parkinson's disease can be reduced by effective clinical prevention and management. Future studies are needed to explore the efficacy of treatment and prevention of falls and fear of falls.

An Electronic Health Record-Compatible Model to Predict Personalized Treatment Effects From the Diabetes Prevention Program: A Cross-Evidence Synthesis Approach Using Clinical Trial and Real-World Data

OBJECTIVE

To develop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.

PATIENTS AND METHODS

A risk prediction model was developed and validated in a large observational database of patients with an index visit date between January 1, 2012, and December 31, 2016, with treatment effect estimates from risk-based reanalysis of clinical trial data. The risk model development cohort included 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); the validation cohort included a distinct sample of 1.1 million patients in OLDW. The randomly assigned clinical trial cohort included 3081 people from the Diabetes Prevention Program (DPP) study.

RESULTS

Eleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (C statistic = 0.76; observed 3-year diabetes rate was 1.8% (95% confidence interval [CI], 1.7 to 1.9) in the lowest-risk quarter and 19.6% (19.4 to 19.8) in the highest-risk quarter). In the DPP, the hazard ratio (HR) for lifestyle modification was constant across all levels of risk (HR, 0.43; 95% CI, 0.35 to 0.53), whereas the HR for metformin was highly risk dependent (HR, 1.1; 95% CI, 0.61 to 2.0 in the lowest-risk quarter vs HR, 0.45; 95% CI, 0.35 to 0.59 in the highest-risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification and 73% of the benefits of population-wide metformin therapy can be obtained by targeting the highest-risk quarter of patients.

CONCLUSION

The Tufts-Predictive Analytics and Comparative Effectiveness DPP Risk model is an EHR-compatible tool that might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.

Understanding GPs' views and experiences of using clinical prediction rules in the management of respiratory infections: a qualitative study

Background

Respiratory tract infections (RTIs) account for 60% of antibiotic prescribing in primary care. Several clinical prediction rules (CPRs) have been developed to help reduce unnecessary prescribing for RTIs, but there is a lack of studies exploring whether or how these CPRs are being used in UK general practice.

Aim

To explore UK GPs’ views and experiences with regards to RTI CPRs, and to identify barriers and facilitators to their use in practice.

Design & setting

A qualitative analysis of interviews with in-hours GPs working in the South and South West of England.

Method

Semi-structured qualitative telephone interviews were conducted, digitally recorded, transcribed verbatim, and analysed using an inductive thematic approach. Patient and public involvement representatives contributed to study design and interpretation of findings.

Results

Thirty-two GPs were interviewed. Some CPRs were more commonly used than others. Participants used CPRs to facilitate patient—clinician discussion, confirm and support their decision, and document the consultation. GPs also highlighted concerns including lack of time, inability of CPRs to incorporate patient complexity, a shift in focus from the patient during consultations, and limited use in remote consultation (during the COVID-19 pandemic).

Conclusion

This study highlights the need for user-friendly CPRs that are readily integrated into computer systems, and easily embedded into routine practice to complement clinical decision-making. Existing CPRs need to be validated for other populations where demographics and clinical characteristics may differ, as well different settings including remote consultations and self-assessment.

Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity

OBJECTIVE

To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.

STUDY DESIGN AND SETTING

Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).

RESULTS

Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.

CONCLUSIONS

Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.

TRIAL REGISTRATION NUMBER

PROSPERO id: CRD42018088129.

Using a clinical prediction rule to prioritize diagnostic testing leads to reduced transmission and hospital burden: A modeling example of early SARS-CoV-2

BACKGROUND

Prompt identification of infections is critical for slowing the spread of infectious diseases. However, diagnostic testing shortages are common in emerging diseases, low resource settings, and during outbreaks. This forces difficult decisions regarding who receives a test, often without knowing the implications of those decisions on population-level transmission dynamics. Clinical prediction rules (CPRs) are commonly used tools to guide clinical decisions.

METHODS

Using early SARS-CoV-2 as an example, we used data from electronic health records to develop a parsimonious 5-variable CPR to identify those who are most likely to test positive. To consider the implications of gains in daily case detection at the population level, we incorporated testing using the CPR into a compartmentalized model of SARS-CoV-2.

RESULTS

We found that applying this CPR (AUC: 0.69 (95% CI: 0.68 - 0.70)) to prioritize testing increased the proportion of those testing positive in settings of limited testing capacity. We found that prioritized testing led to a delayed and lowered infection peak (i.e., "flattens the curve"), with the greatest impact at lower values of the effective reproductive number (such as with concurrent community mitigation efforts), and when higher proportions of infectious persons seek testing. Additionally, prioritized testing resulted in reductions in overall infections as well as hospital and intensive care unit (ICU) burden.

CONCLUSION

We highlight the population-level benefits of evidence-based allocation of limited diagnostic capacity.

Validation of a Community-Acquired Pneumonia Score To Improve Empiric Antibiotic Selection at an Academic Medical Center

The 2019 American Thoracic Society and the Infectious Diseases Society of America community-acquired pneumonia (CAP) guidelines recommend that drug-resistant pathogens (DRP) be empirically covered if locally validated risk factors are present. This retrospective case-control validation study evaluated the performance of the drug resistance in pneumonia (DRIP) clinical prediction score. Two hundred seventeen adult patients with ICD-10 (https://www.who.int/classifications/classification-of-diseases) pneumonia diagnosis, positive confirmed microbiologic data, and clinical signs and symptoms were included. A DRIP score of ≥4 was used to assess model performance. Logistic regression was used to select for significant predictors and create a modified DRIP score, which was evaluated to define clinical application. The DRIP score predicted pneumonia due to a DRP with a sensitivity of 67% and specificity of 73%. The area under the receiver operating characteristic (AUROC) curve was 0.76 (95% confidence interval [CI], 0.69 to 0.82). From regression analysis, prior infection with a DRP and antibiotics in the last 60 days, yielding scores of 2 points and 1 point, respectively, remained local risk factors in predicting drug-resistant pneumonia. Sensitivity (47%) and specificity (94%) were maximized at a threshold of ≥2 in the modified DRIP model. Therefore, prior infection with a DRP remained the only clinically relevant predictor for drug-resistant pneumonia. The original DRIP score demonstrated a decreased performance in our patient population and behaved similarly to other clinical prediction models. Empiric CAP therapy without anti-methicillin-resistant Staphylococcus aureus and antipseudomonal coverage should be considered for noncritically ill patients without a drug resistant pathogen infection in the past year. Our data support the necessity of local validation to authenticate clinical risk predictors for drug-resistant pneumonia.

Modeling reductions in SARS-CoV-2 transmission and hospital burden achieved by prioritizing testing using a clinical prediction rule

Prompt identification of cases is critical for slowing the spread of COVID-19. However, many areas have faced diagnostic testing shortages, requiring difficult decisions to be made regarding who receives a test, without knowing the implications of those decisions on population-level transmission dynamics. Clinical prediction rules (CPRs) are commonly used tools to guide clinical decisions. We used data from electronic health records to develop a parsimonious 5-variable CPR to identify those who are most likely to test positive, and found that its application to prioritize testing increases the proportion of those testing positive in settings of limited testing capacity. To consider the implications of these gains in daily case detection on the population level, we incorporated testing using the CPR into a compartmentalized disease transmission model. We found that prioritized testing led to a delayed and lowered infection peak (i.e. 'flattens the curve'), with the greatest impact at lower values of the effective reproductive number (such as with concurrent social distancing measures), and when higher proportions of infectious persons seek testing. Additionally, prioritized testing resulted in reductions in overall infections as well as hospital and intensive care unit (ICU) burden. In conclusion, we present a novel approach to evidence-based allocation of limited diagnostic capacity, to achieve public health goals for COVID-19.