Prognosis research: why is Dr. Lydgate still waiting?

Clinical prediction model for prognosis in kidney transplant recipients (KIDMO): study protocol

BACKGROUND

Many potential prognostic factors for predicting kidney transplantation outcomes have been identified. However, in Switzerland, no widely accepted prognostic model or risk score for transplantation outcomes is being routinely used in clinical practice yet. We aim to develop three prediction models for the prognosis of graft survival, quality of life, and graft function following transplantation in Switzerland.

METHODS

The clinical kidney prediction models (KIDMO) are developed with data from a national multi-center cohort study (Swiss Transplant Cohort Study; STCS) and the Swiss Organ Allocation System (SOAS). The primary outcome is the kidney graft survival (with death of recipient as competing risk); the secondary outcomes are the quality of life (patient-reported health status) at 12 months and estimated glomerular filtration rate (eGFR) slope. Organ donor, transplantation, and recipient-related clinical information will be used as predictors at the time of organ allocation. We will use a Fine & Gray subdistribution model and linear mixed-effects models for the primary and the two secondary outcomes, respectively. Model optimism, calibration, discrimination, and heterogeneity between transplant centres will be assessed using bootstrapping, internal-external cross-validation, and methods from meta-analysis.

DISCUSSION

Thorough evaluation of the existing risk scores for the kidney graft survival or patient-reported outcomes has been lacking in the Swiss transplant setting. In order to be useful in clinical practice, a prognostic score needs to be valid, reliable, clinically relevant, and preferably integrated into the decision-making process to improve long-term patient outcomes and support informed decisions for clinicians and their patients. The state-of-the-art methodology by taking into account competing risks and variable selection using expert knowledge is applied to data from a nationwide prospective multi-center cohort study. Ideally, healthcare providers together with patients can predetermine the risk they are willing to accept from a deceased-donor kidney, with graft survival, quality of life, and graft function estimates available for their consideration.

STUDY REGISTRATION

Open Science Framework ID: z6mvj.

Prediction models for the prediction of unplanned hospital admissions in community-dwelling older adults: A systematic review

Background

Identification of community-dwelling older adults at risk of unplanned hospitalizations is of importance to facilitate preventive interventions. Our objective was to review and appraise the methodological quality and predictive performance of prediction models for predicting unplanned hospitalizations in community-dwelling older adults

Methods and findings

We searched MEDLINE, EMBASE and CINAHL from August 2013 to January 2021. Additionally, we checked references of the identified articles for the inclusion of relevant publications and added studies from two previous reviews that fulfilled the eligibility criteria. We included prospective and retrospective studies with any follow-up period that recruited adults aged 65 and over and developed a prediction model predicting unplanned hospitalizations. We included models with at least one (internal or external) validation cohort. The models had to be intended to be used in a primary care setting. Two authors independently assessed studies for inclusion and undertook data extraction following recommendations of the CHARMS checklist, while quality assessment was performed using the PROBAST tool. A total of 19 studies met the inclusion criteria. Prediction horizon ranged from 4.5 months to 4 years. Most frequently included variables were specific medical diagnoses (n = 11), previous hospital admission (n = 11), age (n = 11), and sex or gender (n = 8). Predictive performance in terms of area under the curve ranged from 0.61 to 0.78. Models developed to predict potentially preventable hospitalizations tended to have better predictive performance than models predicting hospitalizations in general. Overall, risk of bias was high, predominantly in the analysis domain.

Conclusions

Models developed to predict preventable hospitalizations tended to have better predictive performance than models to predict all-cause hospitalizations. There is however substantial room for improvement on the reporting and analysis of studies. We recommend better adherence to the TRIPOD guidelines.

QUAPAS: An Adaptation of the QUADAS-2 Tool to Assess Prognostic Accuracy Studies

Whereas diagnostic tests help detect the cause of signs and symptoms, prognostic tests assist in evaluating the probable course of the disease and future outcome. Studies to evaluate prognostic tests are longitudinal, which introduces sources of bias different from those for diagnostic accuracy studies. At present, systematic reviews of prognostic tests often use the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to assess risk of bias and applicability of included studies because no equivalent instrument exists for prognostic accuracy studies. QUAPAS (Quality Assessment of Prognostic Accuracy Studies) is an adaptation of QUADAS-2 for prognostic accuracy studies. Questions likely to identify bias were evaluated in parallel and collated from QUIPS (Quality in Prognosis Studies) and PROBAST (Prediction Model Risk of Bias Assessment Tool) and paired to the corresponding question (or domain) in QUADAS-2. A steering group conducted and reviewed 3 rounds of modifications before arriving at the final set of domains and signaling questions. QUAPAS follows the same steps as QUADAS-2: Specify the review question, tailor the tool, draw a flow diagram, judge risk of bias, and identify applicability concerns. Risk of bias is judged across the following 5 domains: participants, index test, outcome, flow and timing, and analysis. Signaling questions assist the final judgment for each domain. Applicability concerns are assessed for the first 4 domains. The authors used QUAPAS in parallel with QUADAS-2 and QUIPS in a systematic review of prognostic accuracy studies. QUAPAS improved the assessment of the flow and timing domain and flagged a study at risk of bias in the new analysis domain. Judgment of risk of bias in the analysis domain was challenging because of sparse reporting of statistical methods.

Prognostication in neurology

Prognosticating is central to primary palliative care in neurology. Many neurologic diseases carry a high burden of troubling symptoms, and many individuals consider health states due to neurologic disease worse than death. Many patients and families report high levels of need for information at all disease stages, including information about prognosis. There are many barriers to communicating prognosis including prognostic uncertainty, lack of training and experience, fear of destroying hope, and not enough time. Developing the right mindset, tools, and skills can improve one's ability to formulate and communicate prognosis. Prognosticating is subject to many biases which can dramatically affect the quality of patient care; it is important for providers to recognize and reduce them. Patients and surrogates often do not hear what they are told, and even when they hear correctly, they form their own opinions. With practice and self-reflection, one can improve their prognostic skills, help patients and families create honest roadmaps of the future, and deliver high-quality person-centered care.

Benchmarking Machine Learning Models to Assist in the Prognosis of Tuberculosis

Tuberculosis (TB) is an airborne infectious disease caused by organisms in the Mycobacterium tuberculosis (Mtb) complex. In many low and middle-income countries, TB remains a major cause of morbidity and mortality. Once a patient has been diagnosed with TB, it is critical that healthcare workers make the most appropriate treatment decision given the individual conditions of the patient and the likely course of the disease based on medical experience. Depending on the prognosis, delayed or inappropriate treatment can result in unsatisfactory results including the exacerbation of clinical symptoms, poor quality of life, and increased risk of death. This work benchmarks machine learning models to aid TB prognosis using a Brazilian health database of confirmed cases and deaths related to TB in the State of Amazonas. The goal is to predict the probability of death by TB thus aiding the prognosis of TB and associated treatment decision making process. In its original form, the data set comprised 36,228 records and 130 fields but suffered from missing, incomplete, or incorrect data. Following data cleaning and preprocessing, a revised data set was generated comprising 24,015 records and 38 fields, including 22,876 reported cured TB patients and 1139 deaths by TB. To explore how the data imbalance impacts model performance, two controlled experiments were designed using (1) imbalanced and (2) balanced data sets. The best result is achieved by the Gradient Boosting (GB) model using the balanced data set to predict TB-mortality, and the ensemble model composed by the Random Forest (RF), GB and Multi-Layer Perceptron (MLP) models is the best model to predict the cure class.

PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Prognosticating autoimmune encephalitis: A systematic review

OBJECTIVE

To perform a systematic review of the current scientific literature in order to identify variables associated with patient prognosis in autoimmune encephalitis.

METHODS

We performed a systematic literature search using MEDLINE, Embase, PubMed and PsychInfo databases. We selected studies that explored the correlation between early clinical and paraclinical findings, and patient outcomes. Data was extracted, analyzed and recorded in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

Forty four publications detailing 2823 subjects matched our inclusion criteria. There was considerable heterogeneity in methodology, patient profile, investigation results and clinical outcome measures. Findings were often discrepant for cases of anti-NMDAR encephalitis when compared with other causes of autoimmune encephalitis. Delay in immunotherapy contributed to a variety of worse outcomes for patients with different subsets of autoimmune encephalitis. Altered consciousness, ICU admission and no use of immunotherapy were variables associated with poor prognosis in anti-NMDAR encephalitis. Older age, sex, the presence of status epilepticus, CSF abnormalities and MRI changes were unlikely to have significant prognostic value. The influence of antibody titers, autonomic dysfunction and underlying malignancy was unclear.

CONCLUSIONS

A number of variables were identified to have potential predictive value for outcomes in autoimmune encephalitis. Heterogeneous study design, size and quality were major limiting factors in this review.

How to conduct a high-quality original study on a prognostic research topic

This is the third article of our educational series, which focuses on how to conduct a high-quality original study on a prognostic research topic from a methodological perspective. We introduce four aspects: clarifying the objectives; generating an appropriate research question; planning the study; and reporting and analyzing data. This paper has several highlights. (1) There are four types of prognostic studies: Type I-fundamental prognostic research, Type II-prognostic factors research, Type III-prediction model research, and Type IV-stratified medicine research. (2) We present the defining characteristics for each type of prognostic study. (3) For Types I-III, we suggest that "PFOT″ components (target Population, prognostic or predictive Factor[s] or a predictive model with a combination of multiple Factors, Outcome, and follow-up Time) should be included in the research questions; for Type IV, "PIFOT″ components (Intervention was added to PFOT) should be included in the research questions. (4) As with other study designs, prognostic studies should be registered to help mitigate duplication of effort across study teams and to accelerate the pace of scientific evolution. (5) Sample size calculations are an important step for prognostic studies. (6) Confounders and missing data issues should be considered carefully during study planning, reporting, and analyzing data. (7) For Type III studies, at least an internal validation should be performed, and univariable analysis to select significant variables (e.g., p-value < 0.05) for a multivariable model is not recommended. (8) A test for interaction is a necessary step for Type IV prognostic studies. A high-quality prognostic study would benefit from clinicians, methodologists, and statisticians working together.

Prognosis as an Intervention

This article elaborates on how neonatologists and perinatologists might conceive of prognosis as an intervention with outcomes relevant to patients, families, and society at large and highlights aspects of this important area of practice requiring further study.

Discussing prognosis with patients with osteoarthritis: a cross-sectional survey in general practice

Osteoarthritis is a leading cause of chronic pain and disability and one of the most common conditions diagnosed and managed in primary care. Despite the evidence that patients would value discussions about the course of osteoarthritis to help them make informed treatment decisions and plan for the future, little is known of GPs’ practice of, or views regarding, discussing prognosis with these patients. A cross-sectional postal survey asked 2500 randomly selected UK GPs their views on discussing prognosis with patients with osteoarthritis and potential barriers or facilitators to such discussions. They were also asked if prognostic discussions were part of their current practice and what indicators they considered important in assessing the prognosis associated with osteoarthritis. Of 768 respondents (response rate 30.7 %), the majority felt it necessary to discuss prognosis with osteoarthritis patients (n = 738, 96.1 %), but only two thirds reported that it was part of their routine practice (n = 498, 64.8 %). Most respondents found predicting the course of osteoarthritis (n = 703, 91.8 %) and determining the prognosis of patients difficult (n = 589, 76.7 %). Obesity, level of physical disability and pain severity were considered the most important prognostic indicators in osteoarthritis. Although GPs consider prognostic discussions necessary for patients with osteoarthritis, few prioritise these discussions. Lack of time and perceived difficulties in predicting the disease course and determining prognosis for patients with osteoarthritis may be barriers to engaging in prognostic discussions. Further research is required to identify ways to assist GPs making prognostic predictions for patients with osteoarthritis and facilitate engagement in these discussions.

Development and validation of an epidemiologic case definition of epilepsy for use with routinely collected Australian health data

OBJECTIVES

We report the diagnostic validity of a selection algorithm for identifying epilepsy cases.

STUDY DESIGN AND SETTING

Retrospective validation study of International Classification of Diseases 10th Revision Australian Modification (ICD-10AM)-coded hospital records and pharmaceutical data sampled from 300 consecutive potential epilepsy-coded cases and 300 randomly chosen cases without epilepsy from 3/7/2012 to 10/7/2013. Two epilepsy specialists independently validated the diagnosis of epilepsy. A multivariable logistic regression model was fitted to identify the optimum coding algorithm for epilepsy and was internally validated.

RESULTS

One hundred fifty-eight out of three hundred (52.6%) epilepsy-coded records and 0/300 (0%) nonepilepsy records were confirmed to have epilepsy. The kappa for interrater agreement was 0.89 (95% CI=0.81-0.97). The model utilizing epilepsy (G40), status epilepticus (G41) and ≥1 antiepileptic drug (AED) conferred the highest positive predictive value of 81.4% (95% CI=73.1-87.9) and a specificity of 99.9% (95% CI=99.9-100.0). The area under the receiver operating curve was 0.90 (95% CI=0.88-0.93).

CONCLUSION

When combined with pharmaceutical data, the precision of case identification for epilepsy data linkage design was considerably improved and could provide considerable potential for efficient and reasonably accurate case ascertainment in epidemiological studies.

Moderate efficiency of clinicians' predictions decreased for blurred clinical conditions and benefits from the use of BRASS index. A longitudinal study on geriatric patients' outcomes

OBJECTIVES

Accurate prognosis is an essential aspect of good clinical practice and efficient health services, particularly for chronic and disabling diseases, as in geriatric populations. This study aims to examine the accuracy of clinical prognostic predictions and to devise prediction models combining clinical variables and clinicians' prognosis for a geriatric patient sample.

STUDY DESIGN AND SETTING

In a sample of 329 consecutive older patients admitted to 10 geriatric units, we evaluated the accuracy of clinicians' prognosis regarding three outcomes at discharge: global functioning, length of stay (LoS) in hospital, and destination at discharge (DD). A comprehensive set of sociodemographic, clinical, and treatment-related information were also collected.

RESULTS

Moderate predictive performance was found for all three outcomes: area under receiver operating characteristic curve of 0.79 and 0.78 for functioning and LoS, respectively, and moderate concordance, Cohen's K = 0.45, between predicted and observed DD. Predictive models found the Blaylock Risk Assessment Screening Score together with clinicians' judgment relevant to improve predictions for all outcomes (absolute improvement in adjusted and pseudo-R(2) up to 19%).

CONCLUSION

Although the clinicians' estimates were important factors in predicting global functioning, LoS, and DD, more research is needed regarding both methodological aspects and clinical measurements, to improve prognostic clinical indices.

Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist

Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary.

Risk factors for adverse prognosis and death in American visceral leishmaniasis: a meta-analysis

BACKGROUND

In the current context of high fatality rates associated with American visceral leishmaniasis (VL), the appropriate use of prognostic factors to identify patients at higher risk of unfavorable outcomes represents a potential tool for clinical practice. This systematic review brings together information reported in studies conducted in Latin America, on the potential predictors of adverse prognosis (continued evolution of the initial clinical conditions of the patient despite the implementation of treatment, independent of the occurrence of death) and death from VL. The limitations of the existing knowledge, the advances achieved and the approaches to be used in future research are presented.

METHODS/PRINCIPAL FINDINGS

The full texts of 14 studies conforming to the inclusion criteria were analyzed and their methodological quality examined by means of a tool developed in the light of current research tools. Information regarding prognostic variables was synthesized using meta-analysis. Variables were grouped according to the strength of evidence considering summary measures, patterns and heterogeneity of effect-sizes, and the results of multivariate analyses. The strongest predictors identified in this review were jaundice, thrombocytopenia, hemorrhage, HIV coinfection, diarrhea, age <5 and age >40-50 years, severe neutropenia, dyspnoea and bacterial infections. Edema and low hemoglobin concentration were also associated with unfavorable outcomes. The main limitation identified was the absence of validation procedures for the few prognostic models developed so far.

CONCLUSIONS/SIGNIFICANCE

Integration of the results from different investigations conducted over the last 10 years enabled the identification of consistent prognostic variables that could be useful in recognizing and handling VL patients at higher risk of unfavorable outcomes. The development of externally validated prognostic models must be prioritized in future investigations.

A novel clinical risk prediction model for sudden cardiac death in hypertrophic cardiomyopathy (HCM risk-SCD)

AIMS

Hypertrophic cardiomyopathy (HCM) is a leading cause of sudden cardiac death (SCD) in young adults. Current risk algorithms provide only a crude estimate of risk and fail to account for the different effect size of individual risk factors. The aim of this study was to develop and validate a new SCD risk prediction model that provides individualized risk estimates.

METHODS AND RESULTS

The prognostic model was derived from a retrospective, multi-centre longitudinal cohort study. The model was developed from the entire data set using the Cox proportional hazards model and internally validated using bootstrapping. The cohort consisted of 3675 consecutive patients from six centres. During a follow-up period of 24 313 patient-years (median 5.7 years), 198 patients (5%) died suddenly or had an appropriate implantable cardioverter defibrillator (ICD) shock. Of eight pre-specified predictors, age, maximal left ventricular wall thickness, left atrial diameter, left ventricular outflow tract gradient, family history of SCD, non-sustained ventricular tachycardia, and unexplained syncope were associated with SCD/appropriate ICD shock at the 15% significance level. These predictors were included in the final model to estimate individual probabilities of SCD at 5 years. The calibration slope was 0.91 (95% CI: 0.74, 1.08), C-index was 0.70 (95% CI: 0.68, 0.72), and D-statistic was 1.07 (95% CI: 0.81, 1.32). For every 16 ICDs implanted in patients with ≥4% 5-year SCD risk, potentially 1 patient will be saved from SCD at 5 years. A second model with the data set split into independent development and validation cohorts had very similar estimates of coefficients and performance when externally validated.

CONCLUSION

This is the first validated SCD risk prediction model for patients with HCM and provides accurate individualized estimates for the probability of SCD using readily collected clinical parameters.

Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework

BACKGROUND

Prognosis research aims to identify factors associated with the course of health conditions. It is often challenging to judge the overall quality of research evidence in systematic reviews about prognosis due to the nature of the primary studies. Standards aimed at improving the quality of primary studies on the prognosis of health conditions have been created, but these standards are often not adequately followed causing confusion about how to judge the evidence.

METHODS

This article presents a proposed adaptation of Grading of Recommendations Assessment, Development and Evaluation (GRADE), which was developed to rate the quality of evidence in intervention research, to judge the quality of prognostic evidence.

RESULTS

We propose modifications to the GRADE framework for use in prognosis research along with illustrative examples from an ongoing systematic review in the pediatric pain literature. We propose six factors that can decrease the quality of evidence (phase of investigation, study limitations, inconsistency, indirectness, imprecision, publication bias) and two factors that can increase it (moderate or large effect size, exposure-response gradient).

CONCLUSIONS

We describe criteria for evaluating the potential impact of each of these factors on the quality of evidence when conducting a review including a narrative synthesis or a meta-analysis. These recommendations require further investigation and testing.

Survival of patients with non-small cell lung cancer without treatment: a systematic review and meta-analysis

BACKGROUND

Lung cancer is considered a terminal illness with a five-year survival rate of about 16%. Informed decision-making related to the management of a disease requires accurate prognosis of the disease with or without treatment. Despite the significance of disease prognosis in clinical decision-making, systematic assessment of prognosis in patients with lung cancer without treatment has not been performed. We conducted a systematic review and meta-analysis of the natural history of patients with confirmed diagnosis of lung cancer without active treatment, to provide evidence-based recommendations for practitioners on management decisions related to the disease. Specifically, we estimated overall survival when no anticancer therapy is provided.

METHODS

Relevant studies were identified by search of electronic databases and abstract proceedings, review of bibliographies of included articles, and contacting experts in the field. All prospective or retrospective studies assessing prognosis of lung cancer patients without treatment were eligible for inclusion. Data on mortality was extracted from all included studies. Pooled proportion of mortality was calculated as a back-transform of the weighted mean of the transformed proportions using the random-effects model. To perform meta-analysis of median survival, published methods were used to pool the estimates as mean and standard error under the random-effects model. Methodological quality of the studies was examined.

RESULTS

Seven cohort studies (4,418 patients) and 15 randomized controlled trials (1,031 patients) were included in the meta-analysis. All studies assessed mortality without treatment in patients with non-small cell lung cancer (NSCLC). The pooled proportion of mortality without treatment in cohort studies was 0.97 (95% CI: 0.96 to 0.99) and 0.96 in randomized controlled trials (95% CI: 0.94 to 0.98) over median study periods of eight and three years, respectively. When data from cohort and randomized controlled trials were combined, the pooled proportion of mortality was 0.97 (95% CI: 0.96 to 0.98). Test of interaction showed a statistically non-significant difference between subgroups of cohort and randomized controlled trials. The pooled mean survival for patients without anticancer treatment in cohort studies was 11.94 months (95% CI: 10.07 to 13.8) and 5.03 months (95% CI: 4.17 to 5.89) in RCTs. For the combined data (cohort studies and RCTs), the pooled mean survival was 7.15 months (95% CI: 5.87 to 8.42), with a statistically significant difference between the two designs. Overall, the studies were of moderate methodological quality.

CONCLUSION

Systematic evaluation of evidence on prognosis of NSCLC without treatment shows that mortality is very high. Untreated lung cancer patients live on average for 7.15 months. Although limited by study design, these findings provide the basis for future trials to determine optimal expected improvement in mortality with innovative treatments.

The risky business of studying prognosis

Prognosis studies provide important healthcare information. Clinicians use prognostic factors to predict disease progress, thus allowing individualization of disease management. Prognosis is the issue in many translational studies that aim to identify biomarkers to predict outcomes. In a clinical trial, researchers may use prognostic factors to sort patients into risk groups, to clarify the effects of a new therapeutic agent. Prognosis studies can have significant effects on clinical practice.

The prognosis of idiopathic generalized epilepsy

Prognosis describes the trajectory and long-term outcome of a condition. Most studies indicate a better prognosis in idiopathic generalized epilepsy (IGE) in comparison with other epilepsy syndromes. Studies looking at the long-term outcome of different IGE syndromes are relatively scant. Childhood absence epilepsy appears to have a higher rate of remission compared to juvenile absence epilepsy. In absence epilepsies, development of myoclonus and generalized tonic-clonic seizures predicts lower likelihood of remission. Although most patients with juvenile myoclonic epilepsy (JME) achieve remission on antiepileptic drug therapy, <20% appear to remain in remission without treatment. Data on the prognosis of other IGE syndromes are scarce. There are contradictory findings reported on the value of electroencephalography as a predictor of prognosis. Comparisons are made difficult by study heterogeneity, particularly in methodology and diagnostic criteria.

Prognostic research and its potential role in modern gynaecology: a call for more prognostic research in urogynaecology

Over the last 30 years, many researchers have focussed on therapeutic and aetiological studies. Randomised controlled trials (RCT) are considered the 'gold standard' in research circles ( Ward et al. 2004 ). Prognostic research has been neglected probably due to a combination of a lack of perception of its importance and also a failure to produce good quality trials. The word 'prognosis' means the ability to foresee or predict an outcome or an event. Prognostic research in medicine is the ability to predict the likelihood of outcomes from a number of clinical variables. There are two main forms of prognostic research. One form identifies the prognostic value of a single risk factor (e.g. a tumour marker). The second one focuses on the development of a model based on multiple variables and is called multivariable prognostic modelling. The planning and powering of prognostic studies is managed differently from traditional randomised controlled trials.

Developing and evaluating prediction models in rehabilitation populations

This article presents a 3-part framework for developing and evaluating prediction models in rehabilitation populations. First, a process for developing and refining prognostic research questions and the scientific approach to prediction models is presented. Primary components of the scientific approach include the study design and sampling of patients, outcome measurement, selecting predictor variable(s), minimizing methodologic sources of bias, assuring a sufficient sample size for statistical power, and selecting an appropriate statistical model. Examples focus on prediction modeling using samples of rehabilitation patients. Second, a brief overview for statistically building and validating multivariable prediction models is provided, which includes the following 7 steps: data inspection, coding of predictors, model specification, model estimation, model performance, model validation, and model presentation. Third, we propose a set of primary considerations for evaluating prediction model studies using specific quality indicators as criteria to help stakeholders evaluate the quality of a prediction model study. Lastly, we offer perspectives on the future development and use of rehabilitation prediction models.

Defining and evaluating success in paediatric cochlear implantation--an exploratory study

OBJECTIVES

This work is a preliminary study that sought to investigate and develop a method for defining and evaluating "success" in paediatric cochlear implantation (PCI) and to apply a process by which a clinical team could optimally achieve this aim.

METHODS

A pilot group of 25 profoundly deaf children who received a unilateral cochlear implant from 1995 to 2008 was used to develop the process. The cases displayed features that are commonly encountered in PCI. Individual case records were examined retrospectively for adverse factors that might impact on the implantation outcome with particular reference to the probability and severity of impact of each factor. Case prognosis was then rated on a 1-4 basis (1: excellent, 2: good, 3: fair, 4: poor). The subsequent outcomes were assessed using standardised speech (GFW, DEAP), language (PLS-4; CELF) and vocabulary (PPVT; EVT) assessments. Auditory performance outcomes were assessed using a new Categories of Auditory Performance Index (CAPI) that incorporated criteria, testing and scoring aspects. Family issues were also evaluated. Case outcomes were rated 1-4 as above and the prognoses and outcomes were then compared.

RESULTS

Accurate prognostication was seen in 14 cases, 5 had better outcomes than expected and 6 obtained poorer results. "Success", where the outcome equalled or exceeded the prognosis, occurred in 19 (76%) of cases. The successful group contained some "limited gains" cases where the results were nonetheless in line with expectations and parental satisfaction. The detrimental effect of delayed implantation was evident; Connexin 26 (GJB2) mutation had little influence. Poor general medical condition and adverse family situations commonly produced poorer outcomes.

CONCLUSIONS

Success in PCI is achieved when the outcome matches or exceeds the pre-operative expectations of the well-counselled family, without adverse side effects. The assessments achieved a good success rate, but further research is required to clearly identify potential problems and a skilled team is needed to evaluate their risk to the PCI outcome. Unforseen events may also intervene. Currently, differing outcome evaluation techniques impede comparison of studies, particularly in the speech and hearing domains. Rationalisation of these is recommended to facilitate future research.

Survival prediction for terminally ill cancer patients: revision of the palliative prognostic score with incorporation of delirium

PURPOSE

An existing and validated palliative prognostic (PaP) score predicts survival in terminally ill cancer patients based on dyspnea, anorexia, Karnofsky performance status score, clinical prediction of survival, total WBC, and lymphocyte percentage. The PaP score assigns patients to three different risk groups according to a 30-day survival probability--group A, >70%; group B, 30%-70%; group C, <30%. The impact of delirium is known but was not incorporated into the PaP score.

MATERIALS AND METHODS

Our aim was to incorporate information on delirium into the PaP score based on a retrospective series of 361 terminally ill cancer patients. We followed the approach of "validation by calibration," proposed by van Houwelingen and later adapted by Miceli for achieving score revision with inclusion of a new variable. The discriminating performance of the scores was estimated using the K statistic.

RESULTS

The prognostic contribution of delirium was confirmed as statistically significant (p < .001) and the variable was accordingly incorporated into the PaP score (D-PaP score). Following this revision, 30-day survival estimates in groups A, B, and C were 83%, 50%, and 9% for the D-PaP score and 87%, 51%, and 16% for the PaP score, respectively. The overall performance of the D-PaP score was better than that of the PaP score.

CONCLUSION

The revision of the PaP score was carried out by modifying the cutoff values used for prognostic grouping without, however, affecting the partial scores of the original tool. The performance of the D-PaP score was better than that of the PaP score and its key feature of simplicity was maintained.

Reporting performance of prognostic models in cancer: a review

BACKGROUND

Appropriate choice and use of prognostic models in clinical practice require the use of good methods for both model development, and for developing prognostic indices and risk groups from the models. In order to assess reliability and generalizability for use, models need to have been validated and measures of model performance reported. We reviewed published articles to assess the methods and reporting used to develop and evaluate performance of prognostic indices and risk groups from prognostic models.

METHODS

We developed a systematic search string and identified articles from PubMed. Forty-seven articles were included that satisfied the following inclusion criteria: published in 2005; aiming to predict patient outcome; presenting new prognostic models in cancer with outcome time to an event and including a combination of at least two separate variables; and analysing data using multivariable analysis suitable for time to event data.

RESULTS

In 47 studies, Cox models were used in 94% (44), but the coefficients or hazard ratios for the variables in the final model were reported in only 72% (34). The reproducibility of the derived model was assessed in only 11% (5) of the articles. A prognostic index was developed from the model in 81% (38) of the articles, but researchers derived the prognostic index from the final prognostic model in only 34% (13) of the studies; different coefficients or variables from those in the final model were used in 50% (19) of models and the methods used were unclear in 16% (6) of the articles. Methods used to derive prognostic groups were also poor, with researchers not reporting the methods used in 39% (14 of 36) of the studies and data derived methods likely to bias estimates of differences between risk groups being used in 28% (10) of the studies. Validation of their models was reported in only 34% (16) of the studies. In 15 studies validation used data from the same population and in five studies from a different population. Including reports of validation with external data from publications up to four years following model development, external validation was attempted for only 21% (10) of models. Insufficient information was provided on the performance of models in terms of discrimination and calibration.

CONCLUSIONS

Many published prognostic models have been developed using poor methods and many with poor reporting, both of which compromise the reliability and clinical relevance of models, prognostic indices and risk groups derived from them.

Assessing the cost effectiveness of using prognostic biomarkers with decision models: case study in prioritising patients waiting for coronary artery surgery

OBJECTIVE

To determine the effectiveness and cost effectiveness of using information from circulating biomarkers to inform the prioritisation process of patients with stable angina awaiting coronary artery bypass graft surgery.

DESIGN

Decision analytical model comparing four prioritisation strategies without biomarkers (no formal prioritisation, two urgency scores, and a risk score) and three strategies based on a risk score using biomarkers: a routinely assessed biomarker (estimated glomerular filtration rate), a novel biomarker (C reactive protein), or both. The order in which to perform coronary artery bypass grafting in a cohort of patients was determined by each prioritisation strategy, and mean lifetime costs and quality adjusted life years (QALYs) were compared.

DATA SOURCES

Swedish Coronary Angiography and Angioplasty Registry (9935 patients with stable angina awaiting coronary artery bypass grafting and then followed up for cardiovascular events after the procedure for 3.8 years), and meta-analyses of prognostic effects (relative risks) of biomarkers.

RESULTS

The observed risk of cardiovascular events while on the waiting list for coronary artery bypass grafting was 3 per 10,000 patients per day within the first 90 days (184 events in 9935 patients). Using a cost effectiveness threshold of pound20,000- pound30,000 (euro22,000-euro33,000; $32,000-$48,000) per additional QALY, a prioritisation strategy using a risk score with estimated glomerular filtration rate was the most cost effective strategy (cost per additional QALY was < pound410 compared with the Ontario urgency score). The impact on population health of implementing this strategy was 800 QALYs per 100,000 patients at an additional cost of pound 245,000 to the National Health Service. The prioritisation strategy using a risk score with C reactive protein was associated with lower QALYs and higher costs compared with a risk score using estimated glomerular filtration rate.

CONCLUSION

Evaluating the cost effectiveness of prognostic biomarkers is important even when effects at an individual level are small. Formal prioritisation of patients awaiting coronary artery bypass grafting using a routinely assessed biomarker (estimated glomerular filtration rate) along with simple, routinely collected clinical information was cost effective. Prioritisation strategies based on the prognostic information conferred by C reactive protein, which is not currently measured in this context, or a combination of C reactive protein and estimated glomerular filtration rate, is unlikely to be cost effective. The widespread practice of using only implicit or informal means of clinically ordering the waiting list may be harmful and should be replaced with formal prioritisation approaches.

Discussing prognosis with older people with musculoskeletal pain: a cross-sectional study in general practice

Background

Prognosis has been described as an important but neglected branch of clinical science. While patients' views have been sought in the context of life-threatening illness, similar research is lacking for patients presenting with common, non-life-threatening musculoskeletal complaints. The aim of this study was to gauge whether and why older patients with musculoskeletal pain think prognostic information is important, and how often they felt prognosis was discussed in the general practice consultation.

Methods

A cross-sectional survey of consecutive patients aged 50 years of over presenting with non-inflammatory musculoskeletal pain to 5 Central Cheshire general practices. The frequency of responses to the prognostic questions were described and the association with sociodemographic, presenting pain complaint, and psychosocial variables explored using logistic regression.

Results

502 participants (77%) responded to the postal questionnaire. 165 (33%) participants reported discussing prognosis in the consultation with their GP. Discussions about prognosis were more often reported by male patients (OR 1.72, 95% CI 1.09, 2.71) and those for whom this was their first consultation (OR 1.81, 95% CI 1.16, 2.80). 402 (82%) participants thought that prognostic information was important. This was highest among those currently in paid employment (OR 2.95, 95% CI 1.33, 6.57). The reasons patients gave for believing prognostic information was important included 'knowing for the sake of knowing' and planning future activity. Reasons for not believing prognostic information to be important included the belief that progression of pain was inevitable and that nothing could be done to help.

Conclusion

Prognostic information is thought to be important amongst older people with musculoskeletal pain yet discussions occur infrequently in primary care. Barriers to effective prognostic communication and the exact information needs of patients are still unknown and warrant further research.

Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond

In oncology, prognostic markers are clinical measures used to help elicit an individual patient's risk of a future outcome, such as recurrence of disease after primary treatment. They thus facilitate individual treatment choice and aid in patient counselling. Evidence-based results regarding prognostic markers are therefore very important to both clinicians and their patients. However, there is increasing awareness that prognostic marker studies have been neglected in the drive to improve medical research. Large protocol-driven, prospective studies are the ideal, with appropriate statistical analysis and clear, unbiased reporting of the methods used and the results obtained. Unfortunately, published prognostic studies rarely meet such standards, and systematic reviews and meta-analyses are often only able to draw attention to the paucity of good-quality evidence. We discuss how better-quality prognostic marker evidence can evolve over time from initial exploratory studies, to large protocol-driven primary studies, and then to meta-analysis or even beyond, to large prospectively planned pooled analyses and to the initiation of tumour banks. We highlight articles that facilitate each stage of this process, and that promote current guidelines aimed at improving the design, analysis, and reporting of prognostic marker research. We also outline why collaborative, multi-centre, and multi-disciplinary teams should be an essential part of future studies.

Systematic reviews of low back pain prognosis had variable methods and results: guidance for future prognosis reviews

OBJECTIVE

Systematic reviews of prognostic factors for low back pain vary substantially in design and conduct. The objective of this study was to identify, describe, and synthesize systematic reviews of low back pain prognosis, and explore the potential impact of review methods on the conclusions.

STUDY DESIGN AND SETTING

We identified 17 low back pain prognosis reviews published between 2000 and 2006. One reviewer extracted and a second checked review characteristics and results. Two reviewers independently assessed review quality.

RESULTS

Review questions and selection criteria varied; there were both focused and broad reviews of prognostic factors. A quarter of reviews did not clearly define search strategies. The number of potential citations identified ranged from 15 to 4,458 and the number of included prognosis studies ranged from 3 to 32 (of 162 distinct citations included across reviews). Seventy percent of reviews assessed quality of included studies, but assessed only a median of four of six potential biases. All reviews reported associations based on statistical significance; they used various strategies for syntheses. Only a small number of important prognostic factors were consistently reported: older age, poor general health, increased psychological or psychosocial stress, poor relations with colleagues, physically heavy work, worse baseline functional disability, sciatica, and the presence of compensation. We found discrepancies across reviews: differences in some selection criteria influenced studies included, and various approaches to data interpretation influenced review conclusions about evidence for specific prognostic factors.

CONCLUSION

There is an immediate need for methodological work in the area of prognosis systematic reviews. Because of methodological shortcomings in the primary and review literature, there remains uncertainty about reliability of conclusions regarding prognostic factors for low back pain.

Sickness absence as a prognostic marker for common chronic conditions: analysis of mortality in the GAZEL study

OBJECTIVES

To determine whether sickness absence is a prognostic marker in terms of mortality among people with common chronic conditions.

METHODS

Prospective occupational cohort study of 13,077 men and 4871 women aged 37-51 from the National Gas and Electricity Company, France. Records of physician-certified sickness absences over a 3-year period were obtained from employers' registers. Chronic conditions were assessed in annual surveys over the same period. The main outcome measure was all-cause mortality (803 deaths, mean follow-up after assessment of sickness absence: 13.9 years).

RESULTS

In Cox proportional hazard models adjusted for age, sex, socioeconomic position and co-morbidity, >28 annual sickness-absence days versus no absence days was associated with an excess mortality risk among those with cancer (hazard ratio 5.4, 95% CI 2.2 to 13.1), depression (1.7, 1.1 to 2.8), chronic bronchitis or asthma (2.7, 1.6 to 4.6) and hypertension (1.6, 1.0 to 2.6). The corresponding hazard ratios for more than five long (>14 days) sickness-absence episodes per 10 person-years versus no such episodes were 5.4 (2.2 to 13.1), 1.8 (1.3 to 2.7), 2.0 (1.3 to 3.2) and 1.8 (1.2 to 2.7), respectively. Areas under receiver operating characteristics curves for these absence measures varied between 0.56 and 0.73, indicating the potential of these measures to distinguish groups at high risk of mortality. The findings were consistent across sex, age and socioeconomic groups and in those with and without co-morbid conditions.

CONCLUSION

Data on sickness absence may provide useful prognostic information for common chronic conditions at the population level.

Comorbidity and survival of Danish cirrhosis patients: a nationwide population-based cohort study

Patients with liver cirrhosis have a high mortality, not just from cirrhosis-related causes, but also from other causes. This observation indicates that many patients with cirrhosis have other chronic diseases, yet the prognostic impact of comorbidities has not been examined. Using data from a nationwide Danish population-based hospital registry, we identified patients who were diagnosed with cirrhosis between 1995 and 2006 and computed their burden of comorbidity using the Charlson comorbidity index. We compared survival between comorbidity groups, adjusting for alcoholism, sex, age, and calendar period. We also examined the risks of cirrhosis-related and non-cirrhosis-related death using data from death certificates and identified a matched comparison cohort without cirrhosis from the Danish population. We included 14,976 cirrhosis patients, 38% of whom had one or more comorbidities. The overall 1-year survival probability was 65.5%; the 10-year survival probability was 21.5%. Compared with patients with a Charlson comorbidity index of 0, the mortality rate was increased 1.17-fold in patients with an index of 1 [95% confidence interval (CI), 1.11-1.23], 1.51-fold in patients with an index of 2 (95% CI, 1.42-1.62), and two-fold in patients with an index of 3 or higher (95% CI, 1.85-2.15). In the first year of follow-up, but not later, comorbidity increased the risk of cirrhosis-related death, and this was consistent with an apparent synergy between the cirrhosis and comorbidity effects on mortality in the same period.

CONCLUSION

Our findings demonstrate that comorbidity is an important prognostic factor for patients with cirrhosis. Successful treatment of comorbid diseases in the first year after diagnosis may substantially reduce the mortality rate.

Prognosis of stable angina pectoris: why we need larger population studies with higher endpoint resolution

The prognosis of angina was described as "unhappy" by the Framingham investigators and as little different from that of 1-year survivors of acute myocardial infarction. Yet recent clinical trials now report that angina has a good prognosis with adverse outcomes reduced to "normal levels". These disparate prognostic assessments may not be incompatible, applying as they do to population cohorts (Framingham) and selected participants in clinical trials. Comparisons between studies are further complicated by the absence of agreed case definitions for stable angina (contrast this with acute coronary syndromes). Our recent data show that for patients with recent onset symptoms attending chest pain clinics, angina remains a high-risk diagnosis and although many patients receive symptomatic benefit from revascularisation, prognosis is usually unaffected. This leaves little room for complacency and, with angina the commonest initial manifestation of coronary artery disease, there is the opportunity for early detection, risk stratification and treatment to modify outcomes. Meanwhile, larger population-based studies are needed to define the patient journey from earliest presentation through the various syndrome transitions to coronary or noncardiac death in order to increase understanding of the aetiological and prognostic differences between the different coronary disease phenotypes.

The assessment of the prognosis of musculoskeletal conditions in older adults presenting to general practice: a research protocol

BACKGROUND

Musculoskeletal conditions represent a common reason for consulting general practice yet with the exception of low back pain, relatively little is known about the prognosis of these disorders. Recent evidence suggests that common 'generic' factors may be of value when assessing prognosis, irrespective of the location of the pain. This study will test a generic assessment tool used as part of the general practice consultation to determine prognosis of musculoskeletal complaints.

METHODS/DESIGN

Older adults (aged 50 years and over) presenting to six general practices with musculoskeletal complaints will be assessed as part of the routine consultation using a generic assessment of prognosis. Participants will receive a self-completion questionnaire at baseline, three, six and 12 months post consultation to gather further data on pain, disability and psychological status. The primary outcome measure is participant's global rating of change.

DISCUSSION

Prognosis is considered to be a fundamental component of scientific medicine yet prognostic research in primary care settings is currently neglected and prognostic enquiry is disappearing from general medical textbooks. This study aims to address this issue by examining the use of generic prognostic factors in a general practice setting.