Interpreting research on clinical prediction rules for physiotherapy treatments

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

Clinical prediction rules for prognosis and treatment prescription in neck pain: A systematic review

Clinical prediction rules (CPRs) developed to identify sub-groups of people with neck pain for different prognoses (i.e. prognostic) or response to treatments (i.e. prescriptive) have been recommended as a research priority to improve health outcomes for these conditions. A systematic review was undertaken to identify prognostic and prescriptive CPRs relevant to the conservative management of adults with neck pain and to appraise stage of development, quality and readiness for clinical application. Six databases were systematically searched from inception until 4th July 2016. Two independent reviewers assessed eligibility, risk of bias (PEDro and QUIPS), methodological quality and stage of development. 9840 records were retrieved and screened for eligibility. Thirty-two studies reporting on 26 CPRs were included in this review. Methodological quality of included studies varied considerably. Most prognostic CPR development studies employed appropriate designs. However, many prescriptive CPR studies (n = 12/13) used single group designs and/or analysed controlled trials using methods that were inadequate for identifying treatment effect moderators. Most prognostic (n = 11/15) and all prescriptive (n = 11) CPRs have not progressed beyond the derivation stage of development. Four prognostic CPRs relating to acute whiplash (n = 3) or non-traumatic neck pain (n = 1) have undergone preliminary validation. No CPRs have undergone impact analysis. Most prognostic and prescriptive CPRs for neck pain are at the initial stage of development and therefore routine clinical use is not yet supported. Further validation and impact analyses of all CPRs are required before confident conclusions can be made regarding clinical utility.

Clinical Decision Support Tools for Selecting Interventions for Patients with Disabling Musculoskeletal Disorders: A Scoping Review

Purpose We aimed to identify and inventory clinical decision support (CDS) tools for helping front-line staff select interventions for patients with musculoskeletal (MSK) disorders. Methods We used Arksey and O'Malley's scoping review framework which progresses through five stages: (1) identifying the research question; (2) identifying relevant studies; (3) selecting studies for analysis; (4) charting the data; and (5) collating, summarizing and reporting results. We considered computer-based, and other available tools, such as algorithms, care pathways, rules and models. Since this research crosses multiple disciplines, we searched health care, computing science and business databases. Results Our search resulted in 4605 manuscripts. Titles and abstracts were screened for relevance. The reliability of the screening process was high with an average percentage of agreement of 92.3 %. Of the located articles, 123 were considered relevant. Within this literature, there were 43 CDS tools located. These were classified into 3 main areas: computer-based tools/questionnaires (n = 8, 19 %), treatment algorithms/models (n = 14, 33 %), and clinical prediction rules/classification systems (n = 21, 49 %). Each of these areas and the associated evidence are described. The state of evidentiary support for CDS tools is still preliminary and lacks external validation, head-to-head comparisons, or evidence of generalizability across different populations and settings. Conclusions CDS tools, especially those employing rapidly advancing computer technologies, are under development and of potential interest to health care providers, case management organizations and funders of care. Based on the results of this scoping review, we conclude that these tools, models and systems should be subjected to further validation before they can be recommended for large-scale implementation for managing patients with MSK disorders.

Uncertainty in clinical prediction rules: the value of credible intervals

Decision making in physical therapy is increasingly informed by evidence in the form of probabilities. Prior beliefs concerning diagnoses, prognoses, and treatment effects are quantitatively revised by the integration of new information derived from the history, physical examination, and other investigations in a well-recognized application of Bayes' theorem. Clinical prediction rule development studies commonly employ such methodology to produce quantified estimates of the likelihood of patients having certain diagnoses or achieving given outcomes. To date, the physical therapy literature has been limited to the discussion and calculation of the point estimate of such probabilities. The degree of precision associated with the construction of posterior probabilities, which requires consideration of both uncertainty associated with pretest probability and uncertainty associated with test accuracy, remains largely unrecognized and unreported. This paper provides an introduction to the calculation of the uncertainty interval, known as a credible interval, around posterior probability estimates. The method for calculating the credible interval is detailed and illustrated with example data from 2 clinical prediction rule development studies. Two relatively quick and simple methods for approximating the credible interval are also outlined. It is anticipated that knowledge of the credible interval will have practical implications for the incorporation of probabilistic evidence in clinical practice. Consistent with reporting standards for interventional and diagnostic studies, it is equally appropriate that studies reporting posterior probabilities calculate and report the level of precision associated with these point estimates.

Which prognostic factors for low back pain are generic predictors of outcome across a range of recovery domains?

BACKGROUND

Recovery from low back pain (LBP) is multidimensional and requires the use of multiple-response (outcome) measures to fully reflect these many dimensions. Predictive prognostic variables that are present or stable in all or most predictive models that use different outcome measures could be considered "universal" prognostic variables.

OBJECTIVE

The aim of this study was to explore the potential of universal prognostic variables in predictive models for 4 different outcome measures in patients with mechanical LBP.

DESIGN

Predictive modeling was performed using data extracted from a randomized controlled trial. Four prognostic models were created using backward stepwise deletion logistic, Poisson, and linear regression.

METHODS

Data were collected from 16 outpatient physical therapy facilities in 10 states. All 149 patients with LBP were treated with manual therapy and spine strengthening exercises until discharge. Four different measures of response were used: Oswestry Disability Index and Numeric Pain Rating Scale change scores, total visits, and report of rate of recovery.

RESULTS

The set of statistically significant predictors was dependent on the definition of response. All regression models were significant. Within both forms of the 4 models, meeting the clinical prediction rule for manipulation at baseline was present in all 4 models, whereas no irritability at baseline and diagnosis of sprains and strains were present in 2 of 4 of the predictive models.

LIMITATIONS

The primary limitation is that this study evaluated only 4 of the multiple outcome measures that are pertinent for patients with LBP.

CONCLUSIONS

Meeting the clinical prediction rule was prognostic for all outcome measures and should be considered a universal prognostic predictor. Other predictive variables were dependent on the outcomes measure used in the predictive model.

Development of a clinical prediction rule to identify patients with neck pain likely to benefit from thrust joint manipulation to the cervical spine

STUDY DESIGN

Prospective cohort/predictive validity study.

OBJECTIVE

To determine the predictive validity of selected clinical examination items and to develop a clinical prediction rule to determine which patients with neck pain may benefit from cervical thrust joint manipulation (TJM) and exercise.

BACKGROUND

TJM to the cervical spine has been shown to be effective in patients presenting with a primary report of neck pain. It would be useful for clinicians to have a decision-making tool, such as a clinical prediction rule, that could accurately identify which subgroup of patients would respond positively to cervical TJM.

METHODS

Consecutive patients who presented to physical therapy with a primary complaint of neck pain completed a series of self-report measures, then received a detailed standardized history and physical examination. After the clinical examination, all patients received a standardized treatment regimen consisting of cervical TJM and range-of-motion exercise. Depending on response to treatment, patients were treated for 1 or 2 sessions over approximately 1 week. At the end of their participation in the study, patients were classified as having experienced a successful outcome based on a score of +5 ("quite a bit better") or higher on the global rating of change scale. Sensitivity, specificity, and positive and negative likelihood ratios were calculated for all potential predictor variables. Univariate techniques and stepwise logistic regression were used to determine the most parsimonious set of variables for prediction of treatment success. Variables retained in the regression model were used to develop a multivariate clinical prediction rule.

RESULTS

Eighty-two patients were included in data analysis, of whom 32 (39%) achieved a successful outcome. A clinical prediction rule with 4 attributes (symptom duration less than 38 days, positive expectation that manipulation will help, side-to-side difference in cervical rotation range of motion of 10° or greater, and pain with posteroanterior spring testing of the middle cervical spine) was identified. If 3 or more of the 4 attributes (positive likelihood ratio of 13.5) were present, the probability of experiencing a successful outcome improved from 39% to 90%.

CONCLUSION

The clinical prediction rule may improve decision making by providing the ability to a priori identify patients with neck pain who are likely to benefit from cervical TJM and range-of-motion exercise. However, this is only the first step in the process of developing and testing a clinical prediction rule, as future studies are necessary to validate the results and should include long-term follow-up and a comparison group.

LEVEL OF EVIDENCE

Prognosis, level 2b.