Interobserver variability of the measurement of the mortality probability models (MPM II) in the assessment of severity of illness

Development and Validation of a Simple-to-Use Nomogram for Predicting In-Hospital Mortality in Patients With Acute Heart Failure Undergoing Continuous Renal Replacement Therapy

Background: Patients with acute heart failure (AHF) who require continuous renal replacement therapy (CRRT) have a high risk of in-hospital mortality. It is clinically important to screen high-risk patients using a model or scoring system. This study aimed to develop and validate a simple-to-use nomogram consisting of independent prognostic variables for the prediction of in-hospital mortality in patients with AHF undergoing CRRT. Methods: We collected clinical data for 121 patients with a diagnosis of AHF who underwent CRRT in an AHF unit between September 2011 and August 2020 and from 105 patients in the medical information mart for intensive care III (MIMIC-III) database. The nomogram model was created using a visual processing logistic regression model and verified using the standard method. Results: Patient age, days after admission, lactic acid level, blood glucose concentration, and diastolic blood pressure were the significant prognostic factors in the logistic regression analyses and were included in our model (named D-GLAD) as predictors. The resulting model containing the above-mentioned five factors had good discrimination ability in both the training group (C-index, 0.829) and the validation group (C-index, 0.740). The calibration and clinical effectiveness showed the nomogram to be accurate for the prediction of in-hospital mortality in both the training and validation cohort when compared with other models. The in-hospital mortality rates in the low-risk, moderate-risk, and high-risk groups were 14.46, 40.74, and 71.91%, respectively. Conclusion: The nomogram allowed the optimal prediction of in-hospital mortality in adults with AHF undergoing CRRT. Using this simple-to-use model, the in-hospital mortality risk can be determined for an individual patient and could be useful for the early identification of high-risk patients. An online version of the D-GLAD model can be accessed at https://ahfcrrt-d-glad.shinyapps.io/DynNomapp/. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT0751838.

[Verification of validity of MPM II for neurological patients in intensive care units]

PURPOSE

Mortality Probability Model (MPM) II is a model for predicting mortality probability of patients admitted to ICU. This study was done to test the validity of MPM II for critically ill neurological patients and to determine applicability of MPM II in predicting mortality of neurological ICU patients.

METHODS

Data were collected from medical records of 187 neurological patients over 18 yr of age who were admitted to the ICU of C University Hospital during the period from January 2008 to May 2009. Collected data were analyzed through χ(2) test, t-test, Mann-Whiteny test, goodness of fit test, and ROC curve.

RESULTS

As to mortality according to patients' general and clinically related characteristics, mortality was statistically significantly different for ICU stay, hospital stay, APACHE III score, APACHE predicted death rate, GCS, endotracheal intubation, and central venous catheter. Results of Hosmer-Lemeshow goodness-of-fit test were MPM II(0) (χ(2)=0.02, p=.989), MPM II(24) (χ(2)=0.99 p=.805), MPM II(48) (χ(2)=0.91, p=.822), and MPM II(72) (χ(2)=1.57, p=.457), and results of the discrimination test using the ROC curve were MPM II(0), .726 (p<.001), MPM II(24), .764 (p<.001), MPM II(48), .762 (p<.001), and MPM II(72), .809 (p<.001).

CONCLUSION

MPM II was found to be a valid mortality prediction model for neurological ICU patients.

Outcome prediction in critical care: the Mortality Probability Models

PURPOSE OF REVIEW

The comparison of morbidity, mortality, and length-of-stay outcomes in patients receiving critical care requires adjustment based on their presenting illness. These adjustments are made with severity-of-illness models. These models must be periodically updated to reflect current medical practices. This article will review the history of the Mortality Probability Model (MPM), discuss why and how it was recently updated, and outline examples of MPM use.

RECENT FINDINGS

All severity-of-illness models have limitations, especially if a unit's patient population becomes highly specialized. In these situations, customized models may provide better accuracy. The MPMs include those calculated at admission (MPM0) and additional models at 24, 48, and 72 h (MPM 24, MPM 48, and MPM 72). The model is now in its third iteration (MPM 0-III). Length of stay (LOS) and subgroup models have also been developed.

SUMMARY

Understanding appropriate application of models such as MPM is important as transparency in healthcare drives demand for severity-adjusted outcomes data.

Severity scoring in the ICU: a review

BACKGROUND

Patients in the intensive care unit (ICU) require huge resources because of the dysfunction of several of their vital organs. The heterogeneity and complexity of the ICU patient have generated interest in systems able to measure severity of illness as a method of predicting outcome, comparing quality-of-care and stratification for clinical trials.

METHODS

By searching Medline and EMBASE for publications describing scoring systems in the ICU, the most frequently used systems, defined as resulting in more than 50 references, are included in this review. Scoring systems belong to one of four classes prognostic, single-organ failure, trauma scores and organ dysfunction (OD). The different systems are described and discussed.

RESULTS

Three different prognostic scoring systems, including several versions, four single OD scores and three OD scores, were included in this review.

CONCLUSION

Different forms of scoring systems are frequently used in the ICU. They have become a necessary tool to describe ICU populations and to explain differences in mortality. As there are several pitfalls related to the interpretation of the numbers supplied by the systems, they should not be used without knowledge on the science of severity scoring.

Reproducibility of physiological track-and-trigger warning systems for identifying at-risk patients on the ward

OBJECTIVE

Physiological track-and-trigger warning systems are used to identify patients on acute wards at risk of deterioration, as early as possible. The objective of this study was to assess the inter-rater and intra-rater reliability of the physiological measurements, aggregate scores and triggering events of three such systems.

DESIGN

Prospective cohort study.

SETTING

General medical and surgical wards in one non-university acute hospital.

PATIENTS AND PARTICIPANTS

Unselected ward patients: 114 patients in the inter-rater study and 45 patients in the intra-rater study were examined by four raters.

MEASUREMENTS AND RESULTS

Physiological observations obtained at the bedside were evaluated using three systems: the medical emergency team call-out criteria (MET); the modified early warning score (MEWS); and the assessment score of sick-patient identification and step-up in treatment (ASSIST). Inter-rater and intra-rater reliability were assessed by intra-class correlation coefficients, kappa statistics and percentage agreement. There was fair to moderate agreement on most physiological parameters, and fair agreement on the scores, but better levels of agreement on triggers. Reliability was partially a function of simplicity: MET achieved a higher percentage of agreement than ASSIST, and ASSIST higher than MEWS. Intra-rater reliability was better then inter-rater reliability. Using corrected calculations improved the level of inter-rater agreement but not intra-rater agreement.

CONCLUSION

There was significant variation in the reproducibility of different track-and-trigger warning systems. The systems examined showed better levels of agreement on triggers than on aggregate scores. Simpler systems had better reliability. Inter-rater agreement might improve by using electronic calculations of scores.

A comparison of Child-Pugh, APACHE II and APACHE III scoring systems in predicting hospital mortality of patients with liver cirrhosis

BACKGROUND

The aim of this study was to assess the prognostic accuracy of Child-Pugh and APACHE II and III scoring systems in predicting short-term, hospital mortality of patients with liver cirrhosis.

METHODS

200 admissions of 147 cirrhotic patients (44% viral-associated liver cirrhosis, 33% alcoholic, 18.5% cryptogenic, 4.5% both viral and alcoholic) were studied prospectively. Clinical and laboratory data conforming to the Child-Pugh, APACHE II and III scores were recorded on day 1 for all patients. Discrimination was evaluated using receiver operating characteristic (ROC) curves and area under a ROC curve (AUC). Calibration was estimated using the Hosmer-Lemeshow goodness-of-fit test.

RESULTS

Overall mortality was 11.5%. The mean Child-Pugh, APACHE II and III scores for survivors were found to be significantly lower than those of nonsurvivors. Discrimination was excellent for Child-Pugh (ROC AUC: 0.859) and APACHE III (ROC AUC: 0.816) scores, and acceptable for APACHE II score (ROC AUC: 0.759). Although the Hosmer-Lemeshow statistic revealed adequate goodness-of-fit for Child-Pugh score (P = 0.192), this was not the case for APACHE II and III scores (P = 0.004 and 0.003 respectively)

CONCLUSION

Our results indicate that, of the three models, Child-Pugh score had the least statistically significant discrepancy between predicted and observed mortality across the strata of increasing predicting mortality. This supports the hypothesis that APACHE scores do not work accurately outside ICU settings.

Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit

INTRODUCTION

The purpose of this study is to assess the performance of Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, Mortality Probability Model MPM II0 and MPM II24 systems in a major tertiary care hospital in Riyadh, Saudi Arabia.

METHODS

The following data were collected prospectively on all consecutive patients admitted to the Intensive Care Unit between 1 March 1999 and 31 December 2000: demographics, APACHE II and SAPS II scores, MPM variables, ICU and hospital outcome. Predicted mortality was calculated using original regression formulas. Standardized mortality ratio (SMR) was computed with 95% confidence intervals (CI). Calibration was assessed by calculating Lemeshow-Hosmer goodness-of-fit C statistics. Discrimination was evaluated by calculating the Area Under the Receiver Operating Characteristic Curves (ROC AUC).

RESULTS

Predicted mortality by all systems was not significantly different from actual mortality [SMR for MPM II0: 1.00 (0.91-1.10), APACHE II: 1.00 (0.8-1.11), SAPS II: 1.09 (0.97-1.21), MPM II24 0.92 (0.82-1.03)]. Calibration was best for MPM II24 (C-statistic: 14.71, P = 0.06). Discrimination was best for MPM II0 (ROC AUC:0.85) followed by MPM II24 (0.84), APACHE II (0.83) then SAPS II (0.79).

CONCLUSIONS

In our ICU population: 1) Overall mortality prediction, estimated by standardized mortality ratio, was accurate, especially for MPM II0 and APACHE II. 2) MPM II24 has the best calibration. 3) SAPS II has the lowest calibration and discrimination. The local performance of MPM II24 in addition to its ease-to-use makes it an attractive model for mortality prediction in Saudi Arabia.

[Probabilistic models of mortality for patients hospitalized in conventional units]

BACKGROUND

We have developed a tool to measure disease severity of patients hospitalized in conventional units in order to evaluate and compare the effectiveness and quality of health care in our setting.

PATIENTS AND METHOD

A total of 2,274 adult patients admitted consecutively to inpatient units from the Medicine, Surgery and Orthopaedic Surgery, and Trauma Departments of the Corporació Sanitària Parc Taulí of Sabadell, Spain, between November 1, 1997 and September 30, 1998 were included. The following variables were collected: demographic data, previous health state, substance abuse, comorbidity prior to admission, characteristics of the admission, clinical parameters within the first 24 hours of admission, laboratory results and data from the Basic Minimum Data Set of hospital discharges. Multiple logistic regression analysis was used to develop mortality probability models during the hospital stay.

RESULTS

The mortality probability model at admission (MPMHOS-0) contained 7 variables associated with mortality during hospital stay: age, urgent admission, chronic cardiac insufficiency, chronic respiratory insufficiency, chronic liver disease, neoplasm, and dementia syndrome. The mortality probability model at 24-48 hours from admission (MPMHOS-24) contained 9 variables: those included in the MPMHOS-0 plus two statistically significant laboratory variables: hemoglobin and creatinine.

CONCLUSIONS

Severity measures, in particular those presented in this study, can be helpful for the interpretation of hospital mortality rates and can guide mortality or quality committees at the time of investigating health care-related problems.

Daily assessment of severity of illness and mortality prediction for individual patients

OBJECTIVE

To refine the prognosis of critically ill patients using a statistical model that incorporates the daily probabilities of hospital mortality during the first week of stay in the intensive care unit (ICU).

DESIGN

Prospective inception cohort.

SETTING

Fifteen adult medical and surgical ICUs in Spain.

PATIENTS

A total of 1,441 patients aged > or =18 yrs who were consecutively admitted from April 1, 1995, through July 31, 1995.

INTERVENTIONS

Prospective data collection during the stay of the patient in the ICU. Data collected included vital status at hospital discharge as well as all variables necessary for computing the Mortality Probability Models II system at admission and during the first 7 days of stay in the ICU.

MEASUREMENTS AND MAIN RESULTS

Four logistic regression models were obtained. These models contained survival status at hospital discharge as a dependent variable and the following explanatory variables: (model 1) only the probability of dying at admission; (model 2) only the probability of dying during the current day; (model 3) the probability of dying at admission and during the current day; and (model 4) the probabilities of dying at admission and during the previous and current days. Models were evaluated using the Hosmer-Lemeshow statistic and the area under the receiver operating characteristic curve. For survivor and nonsurvivor patients, mortality probabilities obtained using the aforementioned models were compared using linear regression and the paired Student's t-test. Although severity at admission was a statistically significant variable, models 2 and 3 produced almost the same probabilities of hospital mortality, as shown with the linear regression and paired Student's t-test results.

CONCLUSIONS

To have an accurate measurement of the prognosis, it is necessary to update the severity measure. The best estimate of hospital mortality was the probability of death on the current day. Severity at admission and at previous days did not improve the assessment of prognosis.