Predicting death in home care users: derivation and validation of the Risk Evaluation for Support: Predictions for Elder-Life in the Community Tool (RESPECT)

BACKGROUND:

Prognostication tools that report personalized mortality risk and survival could improve discussions about end-of-life and advance care planning. We sought to develop and validate a mortality risk model for older adults with diverse care needs in home care using self-reportable information — the Risk Evaluation for Support: Predictions for Elder-Life in the Community Tool (RESPECT).

METHODS:

Using a derivation cohort that comprised adults living in Ontario, Canada, aged 50 years and older with at least 1 Resident Assessment Instrument for Home Care (RAI-HC) record between Jan. 1, 2007, and Dec. 31, 2012, we developed a mortality risk model. The primary outcome was mortality 6 months after a RAI-HC assessment. We used proportional hazards regression with robust standard errors to account for clustering by the individual. We validated this algorithm for a second cohort of users of home care who were assessed between Jan. 1 and Dec. 31, 2013. We used Kaplan–Meier survival curves to estimate the observed risk of death at 6 months for assessment of calibration and median survival. We constructed 61 risk groups based on incremental increases in the estimated median survival of about 3 weeks among adults at high risk and 3 months among adults at lower risk.

RESULTS:

The derivation and validation cohorts included 435 009 and 139 388 adults, respectively. We identified a total of 122 823 deaths within 6 months of a RAI-HC assessment in the derivation cohort. The mean predicted 6-month mortality risk was 10.8% (95% confidence interval [CI] 10.7%–10.8%) and ranged from 1.54% (95% CI 1.53%–1.54%) in the lowest to 98.1% (95% CI 98.1%–98.2%) in the highest risk group. Estimated median survival spanned from 28 days (11 to 84 d at the 25th and 75th percentiles) in the highest risk group to over 8 years (1925 to 3420 d) in the lowest risk group. The algorithm had a c-statistic of 0.753 (95% CI 0.750–0.756) in our validation cohort.

INTERPRETATION:

The RESPECT mortality risk prediction tool that makes use of readily available information can improve the identification of palliative and end-of-life care needs in a diverse older adult population receiving home care.

Development and validation of a predictive algorithm for risk of dementia in the community setting

Background

Most dementia algorithms are unsuitable for population-level assessment and planning as they are designed for use in the clinical setting. A predictive risk algorithm to estimate 5-year dementia risk in the community setting was developed.

Methods

The Dementia Population Risk Tool (DemPoRT) was derived using Ontario respondents to the Canadian Community Health Survey (survey years 2001 to 2012). Five-year incidence of physician-diagnosed dementia was ascertained by individual linkage to administrative healthcare databases and using a validated case ascertainment definition with follow-up to March 2017. Sex-specific proportional hazards regression models considering competing risk of death were developed using self-reported risk factors including information on socio-demographic characteristics, general and chronic health conditions, health behaviours and physical function.

Results

Among 75 460 respondents included in the combined derivation and validation cohorts, there were 8448 cases of incident dementia in 348 677 person-years of follow-up (5-year cumulative incidence, men: 0.044, 95% CI: 0.042 to 0.047; women: 0.057, 95% CI: 0.055 to 0.060). The final full models each include 90 df (65 main effects and 25 interactions) and 28 predictors (8 continuous). The DemPoRT algorithm is discriminating (C-statistic in validation data: men 0.83 (95% CI: 0.81 to 0.85); women 0.83 (95% CI: 0.81 to 0.85)) and well-calibrated in a wide range of subgroups including behavioural risk exposure categories, socio-demographic groups and by diabetes and hypertension status.

Conclusions

This algorithm will support the development and evaluation of population-level dementia prevention strategies, support decision-making for population health and can be used by individuals or their clinicians for individual risk assessment.

cchsflow: an open science approach to transform and combine population health surveys

Setting

The Canadian Community Health Survey (CCHS) is one of the world’s largest ongoing cross-sectional population health surveys, with over 130,000 respondents every two years or over 1.1 million respondents since its inception in 2001. While the survey remains relatively consistent over the years, there are differences between cycles that pose a challenge to analyze the survey over time.

Intervention

A program package called cchsflow was developed to transform and harmonize CCHS variables to consistent formats across multiple survey cycles. An open science approach was used to maintain transparency, reproducibility and collaboration.

Outcomes

The cchsflow R package uses CCHS survey data between 2001 and 2014. Worksheets were created that identify variables, their names in previous cycles, their category structure, and their final variable names. These worksheets were then used to recode variables in each CCHS cycle into consistently named and labelled variables. Following, survey cycles can be combined. The package was then added as a GitHub repository to encourage collaboration with other researchers.

Implication

The cchsflow package has been added to the Comprehensive R Archive Network (CRAN) and contains support for over 160 CCHS variables, generating a combined data set of over 1 million respondents. By implementing open science practices, cchsflow aims to minimize the amount of time needed to clean and prepare data for the many CCHS users across Canada.

A Data Science Approach to Predictive Analytic Research and Knowledge Translation

IntroductionCurrent approaches to the development and application of predictive studies is inefficient and difficult to reproduce. Thousands of predictive health algorithms have been developed; however, less than 2\% have been assessed outside their original setting and even fewer have been applied and evaluated in practice. Objectives and ApproachObjective: To develop a standardized workflow for algorithm development, dissemination and implementation. Existing predictive analytics workflow and open standards were adapted and expanded for health research and health care settings. The approach was designed to work within multidisciplinary teams and to improve research transparency, reproducibility, quality, efficiency and application. Key components include standardized algorithm description files, documentation and code libraries. All libraries and programming packages, which were created for/with open-source software, can be used for a wide range of statistical and machine learning models. Publicly-available repositories contain the algorithms, validation data, R code and other supporting infrastructure. ResultsAlgorithm development involves variable pre-specification and documentation of model variables, followed by creation of data preprocessing code to generate model variables from the study dataset. Preprocessing uses algorithm specification documentation and a function library, building upon and integrating with existing algorithms when possible to preventing code duplication. Models are output as a Predictive Modelling Markup Language (PMML) file, a portable industry standard for describing and scoring predictive models. A separate scoring "engine" is used to implement PMML-described algorithms in a range of settings, including algorithm validation at other research institutions. Algorithm applications currently include the Project Big Life (www.projectbiglife.ca) online calculators, population, health services and public health planning uses and an algorithm visualization tool. An API permits use of the calculator engine by other organizations. Conclusion/ImplicationsBarriers to the implementation of predictive analytics in real-world settings—such as within electronic medical records or decision aid applications—can be mitigated with well described algorithms that are easy to replicate and implement, especially as access to big health data increases and algorithms become increasingly complex.

Using Large Data to Present Uncertainty for Risk Prediction in the Era of Precision Medicine: The RESPECT Algorithm for Predicted Death at End-of-Life

IntroductionIn Ontario, only 52% of people received palliative care in their last year of life, with only 20\% of those receiving it at home, which can improve the dying experience. Existing algorithms identifying people at end-of-life can potentially improve access to palliative care but are difficult for patients to understand. Objectives and ApproachTo predict and communicate risk of death for community dwelling older adults using a pre-specified and published approach (Trial registration NCT02779309). All assessments from community-dwelling Ontarians (N = 488,636) who received at least one home care assessment from the residential assessment instrument – home care (RAI HC) from 2007 to 2013 (N=1,331,273) were included. The algorithm used a two-step approach by rank ordering participants into 61 groups based on six-month probability of death (from Cox-proportional hazard models) and generated Kaplan-Meier five-year survival curves for each group. Median Survival time is reported with uncertainties expressed with 25th to 75th percentiles. ResultsThe median predicted probability of death within six-months was 0.1095 (0.1093-0.1097, 95% CI). Risk varied among the 61 groups from 0.0158 (0.0158-0.0159) to 0.9820 (0.9810-0.9830). Median observed survival time varied from 27 days (10 to 81 days, 25th and 75th percentile) in the highest risk group to 10 years (3655 days (2111 to >3655 days)) in the lowest risk group. Discrimination and calibration were satisfactory between the derivation (2007-2012 assessments) and validation (2013 assessments) cohorts, with a C statistics of 0.77 and discrimination plot intercept 0.094, slope 0.914. The Kaplan-Meier five-year survival curves for each of the 61 groups will be visually represented in six different ways displaying the risk and uncertainty, and can be altered to yield information of interest specific to each patient/caregiver. Conclusion/ImplicationsRESPECT is adaptive and personalized, with instantaneous feedback as the user provides a response to each question. We will present RESPECT’s development and implementation processes and set up an interactive presentation of the calculator, demonstrating RESPECT’s ability to deliver patient-comprehensible end-of-life prognoses with uncertainty to patients and their caregivers.

Acceptability of a Mobile Clinical Decision Tool Among Emergency Department Clinicians: Development and Evaluation of The Ottawa Rules App

Background

The Ottawa Ankle Rules, Ottawa Knee Rule, and Canadian C-Spine Rule—together known as The Ottawa Rules—are a set of internationally validated clinical decision rules developed to decrease unnecessary diagnostic imaging in the emergency department. In this study, we sought to develop and evaluate the use of a mobile app version of The Ottawa Rules.

Objective

The primary objective of this study was to determine acceptability of The Ottawa Rules app among emergency department clinicians. The secondary objective was to evaluate the effect of publicity efforts on uptake of The Ottawa Rules app.

Methods

The Ottawa Rules app was developed and publicly released for free on iOS and Android operating systems in April 2016. Local and national news and academic media coverage coincided with app release. This study was conducted at a large tertiary trauma care center in Ottawa, Canada. The study was advertised through posters and electronically by email. Emergency department clinicians were approached in person to enroll via in-app consent for a 1-month study during which time they were encouraged to use the app when evaluating patients with suspected knee, foot, or neck injuries. A 23-question survey was administered at the end of the study period via email to determine self-reported frequency, perceived ease of use of the app, and participant Technology Readiness Index scores.

Results

A total of 108 emergency department clinicians completed the study including 42 nurses, 33 residents, 20 attending physicians, and 13 medical students completing emergency department rotations. The median Technology Readiness Index for this group was 3.56, indicating a moderate degree of openness for technological adoption. The majority of survey respondents indicated favorable receptivity to the app including finding it helpful to applying the rules (73/108, 67.6%), that they would recommend the app to colleagues (81/108, 75.0%), and that they would continue using the app (73/108, 67.6%). Feedback from study participants highlighted a desire for access to more clinical decision rules and a higher degree of interactivity of the app. Between April 21, 2016, and June 1, 2017, The Ottawa Rules app was downloaded approximately 4000 times across 89 countries.

Conclusions

We have found The Ottawa Rules app to be an effective means to disseminate the Ottawa Ankle Rules, Ottawa Knee Rule, and Canadian C-Spine Rule among all levels of emergency department clinicians. We have been successful in monitoring uptake and access of the rules in the app as a result of our publicity efforts. Mobile technology can be leveraged to improve the accessibility of clinical decision tools to health professionals.