Understanding articles describing clinical prediction tools. Evidence Based Medicine in Critical Care Group

ABCD2, ABCD2-I, and OTTAWA scores for stroke risk assessment: a direct retrospective comparison

Transient ischemic attack (TIA) is a neurologic emergency characterized by cerebral ischemia eliciting a temporary focal neurological deficit. Many clinical prediction scores have been proposed to assess the risk of stroke after TIA; however, studies on their clinical validity and comparisons among them are scarce. The objective is to compare the accuracy of ABCD², ABCD²-I, and OTTAWA scores in the prediction of a stroke at 7, 90 days, and 1 year in patients presenting with TIA. Single-centre, retrospective study including patients with TIA admitted to the Emergency Department of our third-level, University Hospital, between 2018 and 2019. Five hundred three patients were included. Thirty-nine (7.7%) had a stroke within 1 year from the TIA: 9 (1.7%) and 24 (4.7%) within 7 and 90 days, respectively. ABCD², ABCD²-I, and OTTAWA scores were significantly higher in patients who developed a stroke. AUROCs ranged from 0.66 to 0.75, without statistically significant differences at each time-point. Considering the best cut-off of each score, only ABCD² > 3 showed a sensitivity of 100% only in the prediction of stroke within 7 days. Among clinical items of each score, duration of symptoms, previous TIA, hemiparesis, speech disturbance, gait disturbance, previous cerebral ischemic lesions, and known carotid artery disease were independent predictors of stroke. Clinical scores have moderate prognostic accuracy for stroke after TIA. Considering the independent predictors for stroke, our study indicates the need to continue research and prompts the development of new tools on predictive scores for TIA.

Ethical dilemmas due to the Covid-19 pandemic

The devastating pandemic that has stricken the worldwide population induced an unprecedented influx of patients in ICUs, raising ethical concerns not only surrounding triage and withdrawal of life support decisions, but also regarding family visits and quality of end-of-life support. These ingredients are liable to shake up our ethical principles, sharpen our ethical dilemmas, and lead to situations of major caregiver sufferings. Proposals have been made to rationalize triage policies in conjunction with ethical justifications. However, whatever the angle of approach, imbalance between utilitarian and individual ethics leads to unsolvable discomforts that caregivers will need to overcome. With this in mind, we aimed to point out some critical ethical choices with which ICU caregivers have been confronted during the Covid-19 pandemic and to underline their limits. The formalized strategies integrating the relevant tools of ethical reflection were disseminated without deviating from usual practices, leaving to intensivists the ultimate choice of decision.

Treatment based classification systems for patients with non-specific neck pain. A systematic review

OBJECTIVE

We aimed to identify published classification systems with a targeted treatment approach (treatment-based classification systems (TBCSs)) for patients with non-specific neck pain, and assess their quality and effectiveness.

DESIGN

Systematic review.

DATA SOURCES

MEDLINE, CINAHL, EMBASE, PEDro and the grey literature were systematically searched from inception to December 2019.

STUDY APPRAISAL AND SYNTHESIS

The main selection criterium was a TBCS for patients with non-specific neck pain with physiotherapeutic interventions. For data extraction of descriptive data and quality assessment we used the framework developed by Buchbinder et al. We considered as score of ≤3 as low quality, a score between 3 and 5 as moderate quality and a score ≥5 as good quality. To assess the risk of bias of studies concerning the effectiveness of TBCSs (only randomized clinical trials (RCTs) were included) we used the PEDro scale. We considered a score of ≥ six points on this scale as low risk of bias.

RESULTS

Out of 7664 initial references we included 13 studies. The overall quality of the TBCSs ranged from low to moderate. We found two RCTs, both with low risk of bias, evaluating the effectiveness of two TBCSs compared to alternative treatments. The results showed that both TBCSs were not superior to alternative treatments.

CONCLUSION

Existing TBCSs are, at best, of moderate quality. In addition, TBCSs were not shown to be more effective than alternatives. Therefore using these TBCSs in daily practice is not recommended.

The Landing Error Scoring System Real-Time test as a predictive tool for knee injuries: A historical cohort study

PURPOSE

To assess the value of the Landing error score system - real time test as a predictive tool for knee injuries among combat soldiers in the Israeli defense forces.

METHODS

All 2474 Israeli defense forces' combat soldiers enrolled at the Israeli defense forces Injury Prevention and Rehabilitation Center were included. A retrospective cohort study was conducted. The predictive variable assessed was the landing error score system - real time score. The three main outcome variables were the incidence of overuse knee injuries, the meniscal injury, and the anterior cruciate ligament injury. Receiver operator characteristic analysis was performed to evaluate the test's potential as a predictive tool and in order to establish optimal cutoff scores.

RESULTS

The area under the curve of the receiver operation curves demonstrated no predictive value of the landing error score system - real time test for all three outcome variables (knee injuries: area under the curve 0.526, 95% confidence interval 0.498, 0.554, anterior cruciate ligament injuries: area under the curve 0.496, 95% confidence interval 0.337, 0.656, meniscus injuries: area under the curve 0.515, 95% confidence interval 0.454, 0.576).

INTERPRETATION

Based on the results of this study, the landing error score system - real time test has no predictive value for knee overuse injuries, meniscal injuries, and anterior cruciate ligament injuries. However, due to the small number of cases of anterior cruciate ligament injuries, the predictive value for anterior cruciate ligament injuries of this test should be further investigated.

Comparison of the pediatric risk of mortality, pediatric index of mortality, and pediatric index of mortality 2 models in a pediatric intensive care unit in China: A validation study

This study was designed with the aim of comparing the performances of the pediatric risk of mortality (PRISM), pediatric index of mortality (PIM), and revised version pediatric index of mortality 2 (PIM2) models in a pediatric intensive care unit (PICU) in China.A total of 852 critically ill pediatric patients were recruited in the study between January 1 and December 31, 2014. The variables required to calculate PRISM, PIM, and PIM2 were collected. Mode l performance was evaluated by assessing the calibration and discrimination. Discrimination between death and survival was assessed by calculating the area under the receiver-operating characteristic curve (AUC). Calibration across deciles of risk was evaluated using the Hosmer-Lemeshow goodness-of-fit χ test.Of the 852 patients enrolled in this study, 745 patients survived until the end of the PICU stay (107 patients died, 12.56%). The AUCs (95% confidence intervals, CI) were 0.729 (0.670-0.788) for PRISM, 0.721 (0.667-0.776) for PIM, and 0.726 (0.671-0.781) for PIM2. The Hosmer-Lemeshow test revealed a chi-square of 7.26 (P = 0.51, v = 10) for PRISM, 26.28 (P = 0.0009, v = 10) for PIM, and 10.28 (P = 0.21, v = 10) for PIM2. The standardized mortality rate was 1.14 (95%CI: 0.93-1.36) for PRISM, 1.89 (95%CI: 1.55-2.27) for PIM, and 2.13 (95%CI: 1.75-2.55) for PIM2.The PRISM, PIM, and PIM2 scores demonstrated an acceptable discriminatory performance. With the exception of PIM, the PRISM and PIM2 models had good calibrations.

Symptomatic spinal metastasis: A systematic literature review of the preoperative prognostic factors for survival, neurological, functional and quality of life in surgically treated patients and methodological recommendations for prognostic studies

Purpose

While several clinical prediction rules (CPRs) of survival exist for patients with symptomatic spinal metastasis (SSM), these have variable prognostic ability and there is no recognized CPR for health related quality of life (HRQoL). We undertook a critical appraisal of the literature to identify key preoperative prognostic factors of clinical outcomes in patients with SSM who were treated surgically. The results of this study could be used to modify existing or develop new CPRs.

Methods

Seven electronic databases were searched (1990–2015), without language restriction, to identify studies that performed multivariate analysis of preoperative predictors of survival, neurological, functional and HRQoL outcomes in surgical patients with SSM. Individual studies were assessed for class of evidence. The strength of the overall body of evidence was evaluated using GRADE for each predictor.

Results

Among 4,818 unique citations, 17 were included; all were in English, rated Class III and focused on survival, revealing a total of 46 predictors. The strength of the overall body of evidence was very low for 39 and low for 7 predictors. Due to considerable heterogeneity in patient samples and prognostic factors investigated as well as several methodological issues, our results had a moderately high risk of bias and were difficult to interpret.

Conclusions

The quality of evidence for predictors of survival was, at best, low. We failed to identify studies that evaluated preoperative prognostic factors for neurological, functional, or HRQoL outcomes in surgical patients with SSM. We formulated methodological recommendations for prognostic studies to promote acquiring high-quality evidence to better estimate predictor effect sizes to improve patient education, surgical decision-making and development of CPRs.

Prediction Models and Their External Validation Studies for Mortality of Patients with Acute Kidney Injury: A Systematic Review

OBJECTIVES

To systematically review AKI outcome prediction models and their external validation studies, to describe the discrepancy of reported accuracy between the results of internal and external validations, and to identify variables frequently included in the prediction models.

METHODS

We searched the MEDLINE and Web of Science electronic databases (until January 2016). Studies were eligible if they derived a model to predict mortality of AKI patients or externally validated at least one of the prediction models, and presented area under the receiver-operator characteristic curves (AUROC) to assess model discrimination. Studies were excluded if they described only results of logistic regression without reporting a scoring system, or if a prediction model was generated from a specific cohort.

RESULTS

A total of 2204 potentially relevant articles were found and screened, of which 12 articles reporting original prediction models for hospital mortality in AKI patients and nine articles assessing external validation were selected. Among the 21 studies for AKI prediction models and their external validation, 12 were single-center (57%), and only three included more than 1,000 patients (14%). The definition of AKI was not uniform and none used recently published consensus criteria for AKI. Although good performance was reported in their internal validation, most of the prediction models had poor discrimination with an AUROC below 0.7 in the external validation studies. There were 10 common non-renal variables that were reported in more than three prediction models: mechanical ventilation, age, gender, hypotension, liver failure, oliguria, sepsis/septic shock, low albumin, consciousness and low platelet count.

CONCLUSIONS

Information in this systematic review should be useful for future prediction model derivation by providing potential candidate predictors, and for future external validation by listing up the published prediction models.

Screening and detection of elder abuse: Research opportunities and lessons learned from emergency geriatric care, intimate partner violence, and child abuse

This article provides an overview of elder abuse screening and detection methods for community-dwelling and institutionalized older adults, including general issues and challenges for the field. Then, discussions of applications in emergency geriatric care, intimate partner violence (IPV), and child abuse are presented to inform research opportunities in elder abuse screening. The article provides descriptions of emerging screening and detection methods and technologies from the emergency geriatric care and IPV fields. We also discuss the variety of potential barriers to effective screening and detection from the viewpoint of the older adult, caregivers, providers, and the health care system, and we highlight the potential harms and unintended negative consequences of increased screening and mandatory reporting. We argue that research should continue on the development of valid screening methods and tools, but that studies of perceived barriers and potential harms of elder abuse screening among key stakeholders should also be conducted.

Predicting success of endoscopic third ventriculostomy: validation of the ETV Success Score in a mixed population of adult and pediatric patients

OBJECT

Endoscopic third ventriculostomy (ETV) has become the first line of treatment in obstructive hydrocephalus. The Toronto group (Kulkarni et al.) developed the ETV Success Score (ETVSS) to predict the clinical response following ETV based on age, previous shunt, and cause of hydrocephalus in a pediatric population. However, the use of the ETVSS has not been validated for a population comprising adults. The objective of this study was to validate the ETVSS in a "closed-skull" population, including patients 2 years of age and older.

METHODS

In this retrospective observational study, medical charts of all consecutive cases of ETV performed in two university hospitals were reviewed. The primary outcome, the success of ETV, was defined as the absence of reoperation or death attributable to hydrocephalus at 6 months. The ETVSS was calculated for all patients. Discriminative properties along with calibration of the ETVSS were established for the study population. The secondary outcome is the reoperation-free survival.

RESULTS

This study included 168 primary ETVs. The mean age was 40 years (range 3-85 years). ETV was successful at 6 months in 126 patients (75%) compared with a mean ETVSS of 82.4%. The area under the receiver operating characteristic curve was 0.61, revealing insufficient discrimination from the ETVSS in this population. In contrast, calibration of the ETVSS was excellent (calibration slope = 1.01), although the expected low numbers were obtained for scores < 70. Decision curve analyses demonstrate that ETVSS is marginally beneficial in clinical decision-making, a reduction of 4 and 2 avoidable ETVs per 100 cases if the threshold used on the ETVSS is set at 70 and 60, respectively. However, the use of the ETVSS showed inferior net benefit when compared with the strategy of not recommending ETV at all as a surgical option for thresholds set at 80 and 90. In this cohort, neither age nor previous shunt were significantly associated with unsuccessful ETV. However, better outcomes were achieved in patients with aqueductal stenosis, tectal compressions, and other tumor-associated hydrocephalus than in cases secondary to myelomeningocele, infection, or hemorrhage (p = 0.03).

CONCLUSIONS

The ETVSS did not show adequate discrimination but demonstrated excellent calibration in this population of patients 2 years and older. According to decision-curve analyses, the ETVSS is marginally useful in clinical scenarios in which 60% or 70% success rates are the thresholds for preferring ETV to CSF shunt. Previous history of CSF shunt and age were not associated with worse outcomes, whereas posthemorrhagic and postinfectious causes of the hydrocephalus were significantly associated with reduced success rates following ETV.

Validation and impact analysis of prognostic clinical prediction rules for low back pain is needed: a systematic review

OBJECTIVES

To identify prognostic forms of clinical prediction rules (CPRs) related to the nonsurgical management of adults with low back pain (LBP) and to evaluate their current stage of development.

STUDY DESIGN AND SETTING

Systematic review using a sensitive search strategy across seven databases with hand searching and citation tracking.

RESULTS

A total of 10,005 records were screened for eligibility with 35 studies included in the review. The included studies report on the development of 30 prognostic LBP CPRs. Most of the identified CPRs are in their initial phase of development. Three CPRs were found to have undergone validation--the Cassandra rule for predicting long-term significant functional limitations and the five-item and two-item Flynn manipulation CPRs for predicting a favorable functional prognosis in patients being treated with lumbopelvic manipulation. No studies were identified that investigated whether the implementation of a CPR resulted in beneficial patient outcomes or improved resource efficiencies.

CONCLUSION

Most of the identified prognostic CPRs for LBP are in the initial phase of development and are consequently not recommended for direct application in clinical practice at this time. The body of evidence provides emergent confidence in the limited predictive performance of the Cassandra rule and the five-item Flynn manipulation CPR in comparable clinical settings and patient populations.

How to derive and validate clinical prediction models for use in intensive care medicine

BACKGROUND

Clinical prediction models are formal combinations of historical, physical examination and laboratory or radiographic test data elements designed to accurately estimate the probability that a specific illness is present (diagnostic model), will respond to a form of treatment (therapeutic model) or will have a well-defined outcome (prognostic model) in an individual patient. They are derived and validated using empirical data and used to assist physicians in their clinical decision-making that requires a quantitative assessment of diagnostic, therapeutic or prognostic probabilities at the bedside.

PURPOSE

To provide intensivists with a comprehensive overview of the empirical development and testing phases that a clinical prediction model must satisfy before its implementation into clinical practice.

RESULTS

The development of a clinical prediction model encompasses three consecutive phases, namely derivation, (external) validation and impact analysis. The derivation phase consists of building a multivariable model, estimating its apparent predictive performance in terms of both calibration and discrimination, and assessing the potential for statistical over-fitting using internal validation techniques (i.e. split-sampling, cross-validation or bootstrapping). External validation consists of testing the predictive performance of a model by assessing its calibration and discrimination in different but plausibly related patients. Impact analysis involves comparative research [i.e. (cluster) randomized trials] to determine whether clinical use of a prediction model affects physician practices, patient outcomes or the cost of healthcare delivery.

CONCLUSIONS

This narrative review introduces a checklist of 19 items designed to help intensivists develop and transparently report valid clinical prediction models.

Baseline characteristics of patients with nerve-related neck and arm pain predict the likely response to neural tissue management

STUDY DESIGN

Planned secondary analysis of a randomized controlled trial comparing neural tissue management (NTM) to advice to remain active.

OBJECTIVE

To develop a model that predicts the likelihood of patient-reported improvement following NTM.

BACKGROUND

Matching patients to an intervention they are likely to benefit from potentially improves outcomes. However, baseline characteristics that predict patients' responses to NTM are unknown.

METHODS

Data came from 60 consecutive adults who had nontraumatic, nerve-related neck and unilateral arm pain for at least 4 weeks. Participants were assigned to a group that received NTM (n = 40), which involved brief education, manual therapy, and nerve gliding exercises for 4 treatments over 2 weeks, or to a group that was given advice to remain active (n = 20), which involved instruction to continue their usual activities. The participants' global rating of change at a 3- to 4-week follow-up defined improvement. Penalized regression of NTM data identified the best prediction model. A medical nomogram was created for prediction model scoring. Post hoc analysis determined whether the model predicted a specific response to NTM.

RESULTS

Absence of neuropathic pain qualities, older age, and smaller deficits in median nerve neurodynamic test range of motion predicted improvement. Prediction model cutoffs increased the likelihood of improvement from 53% to 90% (95% confidence interval: 56%, 98%) or decreased the likelihood of improvement to 9% (95% confidence interval: 1%, 42%). The model did not predict the outcomes of the advice to remain active group.

CONCLUSION

Baseline characteristics of patients with nerve-related neck and arm pain predicted the likelihood of improvement with NTM. Model performance needs to be validated in a new sample using different comparison interventions and longer follow-up. Australian New Zealand Clinical Trials Registry (ACTRN 12610000446066).

LEVEL OF EVIDENCE

Prognosis, level 2b-.

Predictive models for the diagnostic of human visceral leishmaniasis in Brazil

BACKGROUND AND OBJECTIVES

In Brazil, as in many other affected countries, a large proportion of visceral leishmaniasis (VL) occurs in remote locations and treatment is often performed on basis of clinical suspicion. This study aimed at developing predictive models to help with the clinical management of VL in patients with suggestive clinical of disease.

METHODS

Cases of VL (n = 213) had the diagnosis confirmed by parasitological method, non-cases (n = 119) presented suggestive clinical presentation of VL but a negative parasitological diagnosis and a firm diagnosis of another disease. The original data set was divided into two samples for generation and validation of the prediction models. Prediction models based on clinical signs and symptoms, results of laboratory exams and results of five different serological tests, were developed by means of logistic regression and classification and regression trees (CART). From these models, clinical-laboratory and diagnostic prediction scores were generated. The area under the receiver operator characteristic curve, sensitivity, specificity, and positive predictive value were used to evaluate the models' performance.

RESULTS

Based on the variables splenomegaly, presence of cough and leukopenia and on the results of five serological tests it was possible to generate six predictive models using logistic regression, showing sensitivity ranging from 90.1 to 99.0% and specificity ranging from 53.0 to 97.2%. Based on the variables splenomegaly, leukopenia, cough, age and weight loss and on the results of five serological tests six predictive models were generated using CART with sensitivity ranging from 90.1 to 97.2% and specificity ranging from 68.4 to 97.4%. The models composed of clinical-laboratory variables and the rk39 rapid test showed the best performance.

CONCLUSION

The predictive models showed to be a potential useful tool to assist healthcare systems and control programs in their strategical choices, contributing to more efficient and more rational allocation of healthcare resources.

Clinical prediction rules in the physiotherapy management of low back pain: a systematic review

OBJECTIVE

To identify, appraise and determine the clinical readiness of diagnostic, prescriptive and prognostic Clinical Prediction Rules (CPRs) in the physiotherapy management of Low Back Pain (LBP).

DATA SOURCES

MEDLINE, EMBASE, CINAHL, AMED and the Cochrane Database of Systematic Reviews were searched from 1990 to January 2010 using sensitive search strategies for identifying CPR and LBP studies. Citation tracking and hand-searching of relevant journals were used as supplemental strategies.

STUDY SELECTION

Two independent reviewers used a two-phase selection procedure to identify studies that explicitly aimed to develop one or more CPRs involving the physiotherapy management of LBP. Diagnostic, prescriptive and prognostic studies investigating CPRs at any stage of their development, derivation, validation, or impact-analysis, were considered for inclusion using a priori criteria. 7453 unique records were screened with 23 studies composing the final included sample.

DATA EXTRACTION

Two reviewers independently extracted relevant data into evidence tables using a standardised instrument.

DATA SYNTHESIS

Identified studies were qualitatively synthesized. No attempt was made to statistically pool the results of individual studies. The 23 scientifically admissible studies described the development of 25 unique CPRs, including 15 diagnostic, 7 prescriptive and 3 prognostic rules. The majority (65%) of studies described the initial derivation of one or more CPRs. No studies investigating the impact phase of rule development were identified.

CONCLUSIONS

The current body of evidence does not enable confident direct clinical application of any of the identified CPRs. Further validation studies utilizing appropriate research designs and rigorous methodology are required to determine the performance and generalizability of the derived CPRs to other patient populations, clinicians and clinical settings.

Critical appraisal of: The endotracheal tube air leak test does not predict extubation outcome in critically ill pediatric patients. Pediatr Crit Care Med 2008; 9:490-496

OBJECTIVE

To review and discuss the airway leak test in predicting extubation failure in critically ill infants and children.

DESIGN

A critical appraisal with literature review of Wratney AT, Benjamin DK Jr, Slonim AD, et al: The endotracheal tube air leak test does not predict extubation outcome in critically ill pediatric patients. Pediatr Crit Care Med 2008; 9:490-496.

FINDINGS

In this prospective, blinded, cohort study, the authors sought to determine whether the absence of an endotracheal tube airleak (airleak test >30 cm H2O) predicts extubation failure in infants and children. Absence of the airleak did not predict extubation failure. The airleak test was >30 cm H2O before extubation in 47% (28 of 59) of patients, yet 23 patients were extubated successfully (negative predictive value, 18%). An airway leak test >30 cm H2O did not increase the likelihood of postextubation reintubation, with a likelihood ratio of 1.2. The authors found that the positive and negative likelihood ratios for all airway leak test results were between 0.8 and 1.4, thus adding little information for the clinician assessment of extubation readiness.

CONCLUSION

Pediatric patients who are otherwise candidates for extubation but do not have an endotracheal airleak may successfully be extubated.

Prognostic value, clinical effectiveness, and cost-effectiveness of high-sensitivity C-reactive protein as a marker for major cardiac events in asymptomatic individuals: a health technology assessment report

OBJECTIVES

The aim of this study was to compare the predictive value, clinical effectiveness, and cost-effectiveness of high-sensitivity C-reactive protein (hs-CRP)-screening in addition to traditional risk factor screening in apparently healthy persons as a means of preventing coronary artery disease.

METHODS AND RESULTS

The systematic review was performed according to internationally recognized methods. Seven studies on risk prediction, one clinical decision-analytic modeling study, and three decision-analytic cost-effectiveness studies were included. The adjusted relative risk of high hs-CRP-level ranged from 0.7 to 2.47 (p < .05 in four of seven studies). Adding hs-CRP to the prediction models increased the areas under the curve by 0.00 to 0.027. Based on the clinical decision analysis, both individuals with elevated hs-CRP-levels and those with hyperlipidemia have a similar gain in life expectancy following statin therapy. One high-quality economic modeling study suggests favorable incremental cost-effectiveness ratios for persons with elevated hs-CRP and higher risk. However, many model parameters were based on limited evidence.

CONCLUSIONS

Adding hs-CRP to traditional risk factors improves risk prediction, but the clinical relevance and cost-effectiveness of this improvement remain unclear.

A simple method to adjust clinical prediction models to local circumstances

Introduction

Clinical prediction models estimate the risk of having or developing a particular outcome or disease. Researchers often develop a new model when a previously developed model is validated and the performance is poor. However, the model can be adjusted (updated) using the new data. The updated model is then based on both the development and validation data. We show how a simple updating method may suffice to update a clinical prediction model.

Methods

A prediction model that preoperatively predicts the risk of severe postoperative pain was developed with multivariable logistic regression from the data of 1944 surgical patients in the Academic Medical Center Amsterdam, the Netherlands. We studied the predictive performance of the model in 1,035 new patients, scheduled for surgery at a later time in the University Medical Center Utrecht, the Netherlands. We assessed the calibration (agreement between predicted risks and the observed frequencies of an outcome) and discrimination (ability of the model to distinguish between patients with and without postoperative pain). When the incidence of the outcome is different, all predicted risks may be systematically over- or underestimated. Hence, the intercept of the model can be adjusted (updating).

Results

The predicted risks were systematically higher than the observed frequencies, corresponding to a difference in the incidence of postoperative pain between the development (62%) and validation set (36%). The updated model resulted in better calibration.

Discussion

When a clinical prediction model in new patients does not show adequate performance, an alternative to developing a new model is to update the prediction model with new data. The updated model will be based on more patient data, and may yield better risk estimates.

[Pneumonia mortality risk in patients with dementia: nursing home physicians' use and evaluation of a prognostic score]

A validated prognostic score for mortality risk 14 days after antibiotics treatment of nursing home residents with dementia and pneumonia is available. Of the nursing homes contacted, 96% was prepared to participate in a clinical impact analysis to examine usefulness of the score in practice. After randomising nursing homes, physicians of 27 homes in the intervention group were asked to complete a questionnaire and use the score for the next case of pneumonia; the control group comprised physicians of the 27 other homes who only completed the questionnaire. The 38 respondents from the control group who all reported about a single patient did not differ from the respondents of the intervention group (31 physicians enrolled 34 patients). Only in 24 cases did physicians calculate the score. For 79% of those patients, the score was (at least somewhat) useful, but mostly to train prognostication competencies and for better documentation of prognosis; frequently treatment decisions had already been made. Of the total group of respondents, the majority was positive about the use of prognostic scores in general, but no-one in the participating homes had any experience with it. The prognostic score is potentially useful for an important group of patients with pneumonia, but further implementation research and inclusion of prognostic instruments in training curricula is needed.

Derivation and validation of a clinical prediction score for isolation of inpatients with suspected pulmonary tuberculosis

BACKGROUND

The use of a clinical prediction score to improve the practice of instituting airborne-transmission precautions in patients with suspected tuberculosis holds promise for increasing appropriate isolation and decreasing unnecessary isolation. The objective of this study was to derive and validate a clinical prediction score for patients with suspected tuberculosis.

METHODS

We used a case-control study design to evaluate differences between patients with a diagnosis of tuberculosis and those placed under airborne precautions who had negative culture results. We developed risk scores based on a multivariable analysis of independently significant factors associated with tuberculosis. Subsequently, we evaluated the sensitivity and specificity of the score in a separate (validation) cohort of patients.

RESULTS

Within our population, we found 4 clinical factors associated with tuberculosis: chronic symptoms (odds ratio [OR], 10.2 [95% confidence interval {CI}, 2.95-35.4]), upper lobe disease on chest radiograph (OR, 5.27 [95% CI, 1.6-17.23]), foreign-born status (OR, 7.01 [95% CI, 2.1-23.8]), and immunocompromised state other than human immunodeficiency virus infection (OR, 8.14 [95% CI, 2.08-31.8]). Shortness of breath (OR, 0.13 [95% CI, 0.04-0.45]) was found to be associated with non-tuberculosis diagnoses and considered a negative predictor in the model. Using a cut-off point to maximize sensitivity, we applied the prediction rule to the validation cohort, resulting in a sensitivity of 97% and a specificity of 42%.

CONCLUSION

The tuberculosis prediction rule derived from our patient population could improve utilization of airborne precautions. Clinical prediction rules continue to show their utility for improvement in isolation practices in different demographic areas.

The endotracheal tube air leak test does not predict extubation outcome in critically ill pediatric patients

OBJECTIVE

Endotracheal tube air leak pressures are used to predict postextubation upper airway compromise such as stridor, upper airway obstruction, or risk of reintubation. To determine whether the absence of an endotracheal tube air leak (air leak test >/=30 cm H2O) measured during the course of mechanical ventilation predicts extubation failure in infants and children.

DESIGN

Prospective, blinded cohort.

SETTING

Multidisciplinary pediatric intensive care unit of a university hospital.

PATIENTS

Patients younger than or equal to 18 yrs and intubated >/=24 hrs.

INTERVENTIONS

The pressure required to produce an audible endotracheal tube air leak was measured within 12 hrs of intubation and extubation. Unless prescribed by the medical care team, patients did not receive neuromuscular blocking agents during air leak test measurements.

MEASUREMENTS AND MAIN RESULTS

The need for reintubation (i.e., extubation failure) was recorded during the 24-hr postextubation period. Seventy-four patients were enrolled resulting in 59 observed extubation trials. The extubation failure rate was 15.3% (9 of 59). Seven patients were treated for postextubation stridor. Extubation failure was associated with a longer median length of ventilation, 177 vs. 78 hrs, p = 0.03. Extubation success was associated with the use of postextubation noninvasive ventilation (p = 0.04). The air leak was absent for the duration of mechanical ventilation (i.e., >/=30 cm H2O at intubation and extubation) in ten patients. Absence of the air leak did not predict extubation failure (negative predictive value 27%, 95% confidence interval 6-60). The air leak test was >/=30 cm H2O before extubation in 47% (28 of 59) of patients yet 23 patients extubated successfully (negative predictive value 18%).

CONCLUSIONS

An endotracheal tube air leak pressure >/=30 cm H2O measured in the nonparalyzed patient before extubation or for the duration of mechanical ventilation was common and did not predict an increased risk for extubation failure. Pediatric patients who are clinically identified as candidates for an extubation trial but do not have an endotracheal tube air leak may successfully tolerate removal of the endotracheal tube.

The value of initial ionized calcium as a predictor of mortality and triage tool in adult trauma patients

Ionized hypocalcemia is a common finding in critically ill patients, but the relationship between ionized hypocalcemia and mortality risk in trauma patients has not been well established. The aim of this study was to assess the usefulness of initial ionized calcium (iCa) in predicting mortality in the trauma population, and evaluate its superiority over the three other triage tools: base deficit, systemic inflammatory response syndrome (SIRS) score, and triage-revised trauma score (t-RTS). A pro-and retrospective study was performed on 255 consecutive trauma patients admitted to our Emergency Medical Center from January to December, 2005, who underwent arterial blood gas analysis. Multivariate logistic regression analysis confirmed iCa (<or=0.88 mM/L), low Glasgow coma scale score, and a large transfusion amount to be significant risk factors associated with mortality (p<0.05). The sensitivities of iCa, base deficit, SIRS score, and t-RTS were 82.9%, 76.4%, 67.1%, and 74.5%, and their specificities were 41.0%, 64.1%, 64.1%, and 87.2%, respectively. Receiver operating characteristic curve analysis determined the areas under the curves of these parameters to be 0.607+/-0.062, 0.736+/-0.056, 0.694+/-0.059, and 0.875+/-0.043, respectively (95% confidence interval). Although initial iCa (<or=0.88 mM/L) was confirmed as a significant risk factor associated with mortality, it exhibited a poorer discriminative power for mortality prediction than other predictors, especially t-RTS.

Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited

OBJECTIVE

To examine the Hosmer-Lemeshow test's sensitivity in evaluating the calibration of models predicting hospital mortality in large critical care populations.

DESIGN

Simulation study.

SETTING

Intensive care unit databases used for predictive modeling.

PATIENTS

Data sets were simulated representing the approximate number of patients used in earlier versions of critical care predictive models (n = 5,000 and 10,000) and more recent predictive models (n = 50,000). Each patient had a hospital mortality probability generated as a function of 23 risk variables.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Data sets of 5,000, 10,000, and 50,000 patients were replicated 1,000 times. Logistic regression models were evaluated for each simulated data set. This process was initially carried out under conditions of perfect fit (observed mortality = predicted mortality; standardized mortality ratio = 1.000) and repeated with an observed mortality that differed slightly (0.4%) from predicted mortality. Under conditions of perfect fit, the Hosmer-Lemeshow test was not influenced by the number of patients in the data set. In situations where there was a slight deviation from perfect fit, the Hosmer-Lemeshow test was sensitive to sample size. For populations of 5,000 patients, 10% of the Hosmer-Lemeshow tests were significant at p < .05, whereas for 10,000 patients 34% of the Hosmer-Lemeshow tests were significant at p < .05. When the number of patients matched contemporary studies (i.e., 50,000 patients), the Hosmer-Lemeshow test was statistically significant in 100% of the models.

CONCLUSIONS

Caution should be used in interpreting the calibration of predictive models developed using a smaller data set when applied to larger numbers of patients. A significant Hosmer-Lemeshow test does not necessarily mean that a predictive model is not useful or suspect. While decisions concerning a mortality model's suitability should include the Hosmer-Lemeshow test, additional information needs to be taken into consideration. This includes the overall number of patients, the observed and predicted probabilities within each decile, and adjunct measures of model calibration.

Antithrombin levels, morbidity, and mortality in a surgical intensive care unit

BACKGROUND

Antithrombin (AT) levels have been suggested as being predictive of outcome in intensive care unit (ICU) patients with septic shock. We investigated the time course of AT levels in a surgical ICU and tested the hypothesis that AT levels may be associated with morbidity and increased mortality rates in a cohort of surgical ICU patients.

METHODS

Three-hundred-twenty-seven consecutive patients admitted to the ICU with an estimated length of stay more than 48 h were included. AT levels were measured daily.

RESULTS

On admission to the ICU, AT levels were below the lower limit of normal in 84.1% (n = 275) of patients and increased significantly by 48 h after admission to reach normal values by the 7th ICU day in patients who never had sepsis (n = 208). This increase in AT levels was delayed in patients with sepsis. Patients with severe sepsis (n = 55) had consistently lower AT levels compared with other patients. Patients with lower AT levels were more likely to need blood products and had a greater maximum degree of organ dysfunction in the ICU than did other patients. The ICU length of stay was similar, regardless of the AT level on admission. Admission AT levels were not associated with increased ICU mortality in a multivariable analysis.

CONCLUSIONS

AT levels are low on admission to the ICU, regardless of the presence of sepsis. Although associated with the degree of organ dysfunction and the severity of sepsis, AT levels were not independently associated with worse outcome in this group of surgical ICU patients.

Review of the acuity scoring systems for the pediatric intensive care unit and their use in quality improvement

Acuity scoring systems quantitate the severity of clinical conditions and stratify patients according to presenting patient condition. In the pediatric intensive care unit, the complexity and number of clinical scoring systems are increasing as their applications for clinicians, health services researches, and quality improvement broaden. This article is a review of acuity scoring systems for the pediatric intensive care unit, including examples of scoring systems available, the methods used in assessing these tools, the ways in which these systems are used, and the utility of acuity scoring systems in accurate benchmarking. It is anticipated that with increasing health care costs and competition and increased focus on medical error reduction and quality improvement, the demands for risk-adjusted outcomes and institutional benchmarking will increase; therefore, as clinicians, academicians, and administrators, it is imperative that we be knowledgeable of the methods and applications of these acuity scoring systems to ensure their quality and appropriate use.

Infant and toddler disease score was useful for risk of hospitalization based on data from administrative claims

OBJECTIVE

To develop the infant and toddler disease score (IDS), a population-based predictive tool of morbidity status in infants and toddlers, based on data from administrative claims.

STUDY DESIGN AND SETTING

A prospective cohort study was conducted, including 35,580 children less than 2 years of age in June 2003 from the French "ERASME" database (mean follow-up 13 months). The outcome variable was incident hospitalization during the follow-up year, that is, before the second birthday for infants and before the third for toddlers. Risk factors before inclusion (age, health care use, medications) were assessed in a 50% random sample (construction sample) by a logistic regression model. Beta coefficients were summed up to obtain the IDS. The IDS was then validated for the remaining 50% of the study population (validation sample).

RESULTS

The major variables significantly associated with the outcome were long-term disability, younger age, and >or=1 hospitalization before inclusion. The risks of hospitalization estimated by the IDS were concordant in the construction and validation samples.

CONCLUSION

The IDS is a useful index for the risk of hospitalization of infants and toddlers in relation to their morbidity status and may be used for adjustment in pharmacoepidemiologic studies using administrative claims databases.

Variables associated with positive findings at relaparotomy in patients with secondary peritonitis

OBJECTIVE

The decision whether and when to perform a relaparotomy in secondary peritonitis is largely subjective and based on professional experience. No existing scoring system aids in this decisional process. Our aim was to search for variables that could predict positive findings at relaparotomy.

DESIGN

Retrospective, clinical study.

SETTING

Tertiary university hospital.

PATIENTS

Two hundred and nineteen patients of a consecutive series who underwent an emergency laparotomy for secondary peritonitis.

INTERVENTIONS

None. Sequential prediction models were constructed by accumulation of clinical information in chronological order using logistic regression to determine the strength of association between predictive variables and positive findings at relaparotomy outcome. Positive findings were defined as persistent peritonitis or a new infectious focus at relaparotomy.

MEASUREMENTS AND MAIN RESULTS

Relaparotomy (planned or on demand) for secondary peritonitis was performed in 117 of 219 patients (53%), yielding either positive (n=62) or negative (n=55) findings. Discriminatory power for positive findings at relaparotomy improved in the successive (multivariate) models: general patient characteristics (area under the curve, 0.60; 95% confidence interval, 0.52-0.68), adding peritonitis-related variables (area under the curve, 0.73; 95% confidence interval, 0.66-0.80), adding operation-related variables (area under the curve, 0.74; 95% confidence interval, 0.67-0.81), and adding postoperative variables (area under the curve, 0.87; 95% confidence interval, 0.82-0.92). Bootstrap resampling reduced the areas under the curve of the subsequent models only slightly. Sensitivity and specificity of the final model were 82% and 76%, respectively, at a total error rate of 16%. One preoperative predictor and five postoperative predictors significantly increased the need for relaparotomy: younger age, decreased hemoglobin levels, temperature>39 degrees C, lower Pao2/Fio2 ratio, increased heart rate, and increased sodium levels.

CONCLUSIONS

These data suggest that the causes of secondary peritonitis and findings at emergency laparotomy for peritonitis are poor indicators for whether patients will need a relaparotomy. Factors indicative of progressive or persistent organ failure during early postoperative follow-up are the best indicators for ongoing infection and associated positive findings at relaparotomy.

Meningococcal disease: identifying high-risk cases

In the previous issue of Critical Care, Vermont and colleagues presented a simple but well-executed observational study describing the levels of chemokines in the serum of 58 children with meningococcal sepsis. The chemokine levels correlated with disease severity and outcome. Significant correlations were demonstrated between admission chemokine levels and the Paediatric Risk of Mortality score, the Disseminated Intravascular Coagulopathy score, the Sequential Organ Failure Assessment score and laboratory parameters of disease severity. Additionally, nonsurvivors had much higher levels of chemokines compared with survivors, and the chemokine levels predicted mortality with a high degree of sensitivity and specificity. The findings are important as they indicate a possible mechanism for risk stratification in future trials of novel therapies in human sepsis, which as yet have not been successful.

Cardiac Troponin T Levels for Risk Stratification in Pediatric Open Heart Surgery

BACKGROUND

Cardiac troponin T has been found to be accurate predictor of complications and adverse clinical events after pediatric cardiac surgery. Contrary to adult cardiac surgery, the relationship of troponin T to patient survival after pediatric heart surgery has not been previously studied. The purpose of this study was to determine whether troponin T could predict death after pediatric open cardiac surgery.

METHODS

This was a retrospective cohort study in which data from 1001 consecutive children having cardiac surgery during a 5-year period were studied. Perioperative variables that could influence death at 30 postoperative days were evaluated.

RESULTS

Multivariate analysis, using a forward stepwise logistic regression, showed that troponin T measured on the first postoperative day was a strong independent predictor of death at 30 days. Level of troponin T greater than 5.9 microg/L on the first postoperative day predicted death (odds ratio, 10.7; 95% confidence interval: 5.2 to 22.1) as did admission lactate level greater than 5.2 mmol/L (odds ratio, 22.2; 95% confidence interval: 9.7 to 50.8) No other variable, including postoperative creatine kinase-MB mass concentration, age, diagnosis, surgical procedure, presence of cyanosis, chromosomal anomaly or ventriculotomy, duration of cardiopulmonary bypass, or aortic cross-clamp, had any independent effect on 30-day survival.

CONCLUSIONS

Cardiac troponin T level on the first postoperative day is a powerful independent risk marker of death in pediatric cardiac surgery.

Prognostic factors for early severity in a childhood multiple sclerosis cohort

OBJECTIVE

The goal was to identify prognostic factors for an early severe course in a cohort of patients with childhood-onset multiple sclerosis, for the construction of a predictive tool.

METHODS

The cohort consisted of 197 children from the French Kid Sclérose en Plaques neuropediatric cohort with relapsing/remitting multiple sclerosis beginning before the age of 16 years. Patients were included from 1990 to 2003. We used multivariate survival analysis (Cox model) to evaluate the prognostic value of clinical, MRI, and biological covariates at onset for the occurrence of a third attack or severe disability ("severity" outcome).

RESULTS

The cohort was monitored for a mean of 5.5 +/- 3.6 years. The "severity" outcome was recorded for 144 patients (73%). The risk of severity was higher for girls, for a time between the first and second attacks of < 1 year, for childhood-onset multiple sclerosis MRI criteria at onset, for an absence of severe mental state changes at onset, and for a progressive course. A derived childhood-onset multiple sclerosis potential index for early severity was found to have a positive predictive value for severity of > 35% for the upper 2 quartiles.

CONCLUSIONS

The clinical and MRI prognostic factors for early severity that were identified were used as the basis of a predictive tool, which will be validated in another cohort. This tool should make it possible to identify subgroups at risk of early severe disease and should facilitate therapeutic studies.

Clinical prediction rules: What are they and what do they tell us?

QUESTION

Clinical prediction rules are research-based tools that quantify the contributions of relevant patient characteristics to provide numeric indices that assist clinicians in making predictions. Clinical prediction rules have been used to describe the likelihood of the presence or absence of a condition, assist in determining patient prognosis, and help the classification of patients for treatment. The recent rapid rise in the use of clinical prediction rules raises questions about the conditions under which they may be used most appropriately. What is the potential role of clinical prediction rules in physiotherapy practice and what are the strategies by which clinicians can determine their appropriate use for a given clinical setting?

CONCLUSION

Clinical prediction rules use quantitative methods to build upon the body of literature and expert opinion and can provide quick and inexpensive estimates of probability. Clinical prediction rules can be of great value to assist clinical decision making but should not be used indiscriminately. They are not a replacement for clinical judgment and should complement rather than supplant clinical opinion and intuition. The development of valid clinical prediction rules should be a goal of physiotherapy research. Specific areas in need of attention include deriving and validating clinical prediction rules to screen patients for potentially serious conditions for which current tests lack adequate diagnostic accuracy or have unacceptable cost and risk, and to assist in classification of patients for treatments that are likely to result in substantially different outcomes in heterogeneous groups of patients.

Prognostic factors in advanced cancer patients: evidence-based clinical recommendations--a study by the Steering Committee of the European Association for Palliative Care

PURPOSE

To offer evidence-based clinical recommendations concerning prognosis in advanced cancer patients.

METHODS

A Working Group of the Research Network of the European Association for Palliative Care identified clinically significant topics, reviewed the studies, and assigned the level of evidence. A formal meta-analysis was not feasible because of the heterogeneity of published studies and the lack of minimal standards in reporting results. A systematic electronic literature search within the main available medical literature databases was performed for each of the following four areas identified: clinical prediction of survival (CPS), biologic factors, clinical signs and symptoms and psychosocial variables, and prognostic scores. Only studies on patients with advanced cancer and survival < or = 90 days were included.

RESULTS

A total of 38 studies were evaluated. Level A evidence-based recommendations of prognostic correlation could be formulated for CPS (albeit with a series of limitations of which clinicians must be aware) and prognostic scores. Recommendations on the use of other prognostic factors, such as performance status, symptoms associated with cancer anorexia-cachexia syndrome (weight loss, anorexia, dysphagia, and xerostomia), dyspnea, delirium, and some biologic factors (leukocytosis, lymphocytopenia, and C-reactive protein), reached level B.

CONCLUSION

Prognostication of life expectancy is a significant clinical commitment for clinicians involved in oncology and palliative care. More accurate prognostication is feasible and can be achieved by combining clinical experience and evidence from the literature. Using and communicating prognostic information should be part of a multidisciplinary palliative care approach.

Critical appraisal of Perez et al: Jugular venous oxygen saturation or arteriovenous difference of lactate content and outcome in children with severe traumatic brain injury

OBJECTIVE

To review the findings and discuss the implications of jugular venous bulb oxygenation monitoring in children with severe traumatic brain injury.

DESIGN

A critical appraisal of Perez et al, Jugular venous oxygen saturation or arteriovenous difference of lactate content and outcome in children with severe traumatic brain injury.

FINDINGS

Two episodes of jugular venous bulb desaturation and abnormal values of arteriovenous difference in lactate content are associated with poor neurologic outcome in children with severe traumatic brain injury-risk ratio 6.6 (95% confidence interval, 1.5-29.7) and risk ratio 17.6 (95% confidence interval, 2.5-122.5), respectively. This confirms the findings of previously reported adult studies.

CONCLUSIONS

This study is the first to demonstrate that jugular venous monitoring may aid in predicting the neurologic outcome of children with severe traumatic brain injury. More studies need to be performed (particularly on safety) before adopting jugular venous bulb oxygenation monitoring as a prediction tool or, ultimately, as a therapeutic intervention to help manage and improve outcome for children with severe traumatic brain injury.

Functional status after childhood traumatic brain injury

BACKGROUND

Identification of children after traumatic brain injury (TBI) likely to have functional deficits at trauma center discharge will facilitate care.

METHODS

Two logistic regression models were derived from data on 4,439 children after TBI 7 to 14 years old enrolled in the National Pediatric Trauma Registry between 1994 and 2001 to predict physical and cognitive disabilities.

RESULTS

Children with open or multiple fractures or closed fractures or injured by motor vehicles were 8.2, 3.5, or 2.5 times more likely, respectively, than those without those circumstances to have discharge physical disabilities. Likelihood of cognitive impairment was increased by factors of 3.2 and 5.8 in children obtunded or comatose on arrival. Preexisting cognitive deficits, injury severity, and intubation predicted physical and cognitive disabilities. The C statistic was 0.862 for the motor model and 0.860 for the cognitive model.

CONCLUSION

Predicting the likelihood of morbidity after acute management of childhood TBI can provide information pertinent to providing effective care.

Optimal search strategies for identifying sound clinical prediction studies in EMBASE

BACKGROUND

Clinical prediction guides assist clinicians by pointing to specific elements of the patient's clinical presentation that should be considered when forming a diagnosis, prognosis or judgment regarding treatment outcome. The numbers of validated clinical prediction guides are growing in the medical literature, but their retrieval from large biomedical databases remains problematic and this presents a barrier to their uptake in medical practice. We undertook the systematic development of search strategies ("hedges") for retrieval of empirically tested clinical prediction guides from EMBASE.

METHODS

An analytic survey was conducted, testing the retrieval performance of search strategies run in EMBASE against the gold standard of hand searching, using a sample of all 27,769 articles identified in 55 journals for the 2000 publishing year. All articles were categorized as original studies, review articles, general papers, or case reports. The original and review articles were then tagged as 'pass' or 'fail' for methodologic rigor in the areas of clinical prediction guides and other clinical topics. Search terms that depicted clinical prediction guides were selected from a pool of index terms and text words gathered in house and through request to clinicians, librarians and professional searchers. A total of 36,232 search strategies composed of single and multiple term phrases were trialed for retrieval of clinical prediction studies. The sensitivity, specificity, precision, and accuracy of search strategies were calculated to identify which were the best.

RESULTS

163 clinical prediction studies were identified, of which 69 (42.3%) passed criteria for scientific merit. A 3-term strategy optimized sensitivity at 91.3% and specificity at 90.2%. Higher sensitivity (97.1%) was reached with a different 3-term strategy, but with a 16% drop in specificity. The best measure of specificity (98.8%) was found in a 2-term strategy, but with a considerable fall in sensitivity to 60.9%. All single term strategies performed less well than 2- and 3-term strategies.

CONCLUSION

The retrieval of sound clinical prediction studies from EMBASE is supported by several search strategies.

Development of radiology prediction models using feature analysis

RATIONALE AND OBJECTIVES

This article provides an introduction to prediction models and their application in diagnostic imaging research. Prediction models capitalize on the different degrees of association among variables to make a prediction of a health state, formulate a rule, or quantify individual contributions of various predictor variables. The purpose of this article is to elucidate the rationale, implication, and interpretation of prediction models using imaging features.

MATERIALS AND METHODS

The techniques and challenges of developing, testing, and implementing prediction models are described. Prediction model development methods are similar to data-mining techniques.

RESULTS

Learning objectives are to review prediction rule (model) methods, learn how prediction models may be applied to feature analysis, and understand the challenges of developing, testing, and implementing prediction models.

Artificial neural networks as prediction tools in the critically ill

The past 25 years have witnessed the development of improved tools with which to predict short-term and long-term outcomes after critical illness. The general paradigm for constructing the best known tools has been the logistic regression model. Recently, a variety of alternative tools, such as artificial neural networks, have been proposed, with claims of improved performance over more traditional models in particular settings. However, these newer methods have yet to demonstrate their practicality and usefulness within the context of predicting outcomes in the critically ill.