APACHE II, data accuracy and outcome prediction

A novel nomogram for predicting the prognosis of critically ill patients with EEG patterns exhibiting stimulus-induced rhythmic, periodic, or ictal discharges

OBJECTIVES

To explore the factors associated with poor prognosis in critically ill patients with Electroencephalogram (EEG) patterns exhibiting stimulus-induced rhythmic, periodic, or ictal discharges (SIRPIDs), and to construct a prognostic prediction model.

METHODS

This study included a total of 53 critically ill patients with EEG patterns exhibiting SIRPIDs who were admitted to the First Affiliated Hospital of Chongqing Medical University from May 2023 to March 2024. Patients were divided into two groups based on their Modified Rankin Scale (mRS) scores at discharge: good prognosis group (0-3 points) and poor prognosis group (4-6 points). Retrospective analyses were performed on the clinical and EEG parameters of patients in both groups. Logistic regression analysis was applied to identify the risk factors related to poor prognosis in critically ill patients with EEG patterns exhibiting SIRPIDs; a risk prediction model for poor prognosis was constructed, along with an individualized predictive nomogram model, and the predictive performance and consistency of the model were evaluated.

RESULTS

Multivariate logistic regression analysis revealed that APACHE II score (OR=1.217, 95 %CI=1.030∼1.438), slow frequency bands or no obvious brain electrical activity (OR=8.720, 95 %CI=1.220∼62.313), and no sleep waveforms (OR=9.813, 95 %CI=1.371∼70.223) were independent risk factors for poor prognosis in patients. A regression model established based on multivariate logistic regression analysis had an area under the curve of 0.902. The model's accuracy was 90.60 %, with a sensitivity of 92.86 % and a specificity of 89.70 %. The nomogram model, after internal validation, showed a concordance index of 0.904.

CONCLUSIONS

A high APACHE II score, EEG patterns with slow frequency bands or no obvious brain electrical activity, and no sleep waveforms were independent risk factors for poor prognosis in patients with SIRPIDs. The nomogram model constructed based on these factors had a favorably high level of accuracy in predicting the risk of poor prognosis and held certain reference and application value for clinical neurofunctional assessment and prognostic determination.

Role of arterial blood glucose and interstitial fluid glucose difference in evaluating microcirculation and clinical prognosis of patients with septic shock: a prospective observational study

BACKGROUND

Microcirculation abnormality in septic shock is closely associated with organ dysfunction and mortality rate. It was hypothesized that the arterial blood glucose and interstitial fluid (ISF) glucose difference (GA-I) as a marker for assessing the microcirculation status can effectively evaluate the severity of microcirculation disturbance in patients with septic shock.

METHODS

The present observational study enrolled patients with septic shock admitted to and treated in the intensive care unit (ICU) of a tertiary teaching hospital. The parameters reflecting organ and tissue perfusion, including lactic acid (Lac), skin mottling score, capillary refill time (CRT), venous-to-arterial carbon dioxide difference (Pv-aCO2), urine volume, central venous oxygen saturation (ScvO2) and GA-I of each enrolled patient were recorded at the time of enrollment (H0), H2, H4, H6, and H8. With ICU mortality as the primary outcome measure, the ICU mortality rate at any GA-I interval was analyzed.

RESULTS

A total of 43 septic shock patients were included, with median sequential organ failure assessment (SOFA) scores of 10.5 (6-16), and median Acute Physiology and Chronic Health Evaluation (APACHAE) II scores of 25.7 (9-40), of whom 18 died during ICU stay. The GA-I levels were negative correlation with CRT (r = 0.369, P < 0.001), Lac (r = -0.269, P < 0.001), skin mottling score (r=-0.223, P < 0.001), and were positively associated with urine volume (r = 0.135, P < 0.05). The ICU mortality rate of patients with septic shock presenting GA-I ≤ 0.30 mmol/L and ≥ 2.14 mmol/L was significantly higher than that of patients with GA-I at 0.30-2.14 mmol/L [65.2% vs. 15.0%, odds ratio (OR) = 10.625, 95% confidence interval (CI): 2.355-47.503].

CONCLUSION

GA-I was correlated with microcirculation parameters, and with differences in survival. Future studies are needed to further explore the potential impact of GA-I on microcirculation and clinical prognosis of septic shock, and the bedside monitoring of GA-I may be beneficial for clinicians to identify high-risk patients.

Liver autotransplantation and atrial reconstruction on a patient with multiorgan alveolar echinococcosis: a case report

BACKGROUND

Alveolar echinococcosis (AE) primarily affects the liver and potentially spreads to other organs. Managing recurrent AE poses significant challenges, especially when it involves critical structures and multiple major organs.

CASE PRESENTATION

We present a case of a 59-year-old female with recurrent AE affecting the liver, heart, and lungs following two previous hepatectomies, the hepatic lesions persisted, adhering to major veins, and imaging revealed additional diaphragmatic, cardiac, and pulmonary involvement. The ex vivo liver resection and autotransplantation (ELRA), first in human combined with right atrium (RA) reconstruction were performed utilizing cardiopulmonary bypass, and repairs of the pericardium and diaphragm. This approach aimed to offer a potentially curative solution for lesions previously considered inoperable without requiring a donor organ or immunosuppressants. The patient encountered multiple serious complications, including atrial fibrillation, deteriorated liver function, severe pulmonary infection, respiratory failure, and acute kidney injury (AKI). These complications necessitated intensive intraoperative and postoperative care, emphasizing the need for a comprehensive management strategy in such complicated high-risk surgeries.

CONCLUSIONS

The multidisciplinary collaboration in this case proved effective and yielded significant therapeutic outcomes for a rare case of advanced hepatic, cardiac, and pulmonary AE. The combined approach of ELRA and RA reconstruction under extracorporeal circulation demonstrated distinct advantages of ELRA in treating complex HAE. Meanwhile, assessing diaphragm function during the perioperative period, especially in patients at high risk of developing pulmonary complications and undergoing diaphragmectomy is vital to promote optimal postoperative recovery. For multi-resistant infection, it is imperative to take all possible measures to mitigate the risk of AKI if vancomycin administration is deemed necessary.

Association of serum ferritin and all-cause mortality in AKI patients: a retrospective cohort study

Background

Serum ferritin (SF) is clinically found to be elevated in many disease conditions, and our research examines serum ferritin in patients with acute kidney injury (AKI) and its implication on the risk of short-term mortality in AKI.

Methods

Data were extracted from the Medical Information Mart for Intensive Care IV 2.2 (MIMIC-IV 2.2) database. Adult patients with AKI who had serum ferritin tested on the first day of ICU admission were included. The primary outcome was 28-day mortality. Kaplan-Meier survival curves and Cox proportional hazards models were used to test the relationship between SF and clinical outcomes. Subgroup analyses based on the Cox model were further conducted.

Results

Kaplan-Meier survival curves showed that a higher SF value was significantly associated with an enhanced risk of 28-day mortality, 90-day mortality, ICU mortality and hospital mortality (log-rank test: p < 0.001 for all clinical outcomes). In multivariate Cox regression analysis, high level of SF with mortality was significantly positive in all four outcome events (all p < 0.001). This result remains robust after adjusting for all variables. Subgroup analysis of SF with 28-day mortality based on Cox model-4 showed that high level of SF was associated with high risk of 28-day mortality in patients regardless of the presence or absence of sepsis (p for interaction = 0.730). Positive correlations of SF and 28-day mortality were confirmed in all other subgroups (p for interaction>0.05).

Conclusion

High level of SF is an independent prognostic predictor of 28-day mortality in patients with AKI.

Error Rates of Data Processing Methods in Clinical Research: A Systematic Review and Meta-Analysis of Manuscripts Identified Through PubMed

Background

In clinical research, prevention of systematic and random errors of data collected is paramount to ensuring reproducibility of trial results and the safety and efficacy of the resulting interventions. Over the last 40 years, empirical assessments of data accuracy in clinical research have been reported in the literature. Although there have been reports of data error and discrepancy rates in clinical studies, there has been little systematic synthesis of these results. Further, although notable exceptions exist, little evidence exists regarding the relative accuracy of different data processing methods. We aim to address this gap by evaluating error rates for 4 data processing methods.

Methods

A systematic review of the literature identified through PubMed was performed to identify studies that evaluated the quality of data obtained through data processing methods typically used in clinical trials: medical record abstraction (MRA), optical scanning, single-data entry, and double-data entry. Quantitative information on data accuracy was abstracted from the manuscripts and pooled. Meta-analysis of single proportions based on the Freeman-Tukey transformation method and the generalized linear mixed model approach were used to derive an overall estimate of error rates across data processing methods used in each study for comparison.

Results

A total of 93 papers (published from 1978 to 2008) meeting our inclusion criteria were categorized according to their data processing methods. The accuracy associated with data processing methods varied widely, with error rates ranging from 2 errors per 10,000 fields to 2,784 errors per 10,000 fields. MRA was associated with both high and highly variable error rates, having a pooled error rate of 6.57% (95% CI: 5.51, 7.72). In comparison, the pooled error rates for optical scanning, single-data entry, and double-data entry methods were 0.74% (0.21, 1.60), 0.29% (0.24, 0.35) and 0.14% (0.08, 0.20), respectively.

Conclusions

Data processing and cleaning methods may explain a significant amount of the variability in data accuracy. MRA error rates, for example, were high enough to impact decisions made using the data and could necessitate increases in sample sizes to preserve statistical power. Thus, the choice of data processing methods can likely impact process capability and, ultimately, the validity of trial results.

The Addition of the Geriatric Nutritional Risk Index to the Prognostic Scoring Systems Did Not Improve Mortality Prediction in Trauma Patients in the Intensive Care Unit

Background

Malnutrition is prevalent among critically ill patients and has been associated with a poor prognosis. This study sought to determine whether the addition of a nutritional indicator to the various variables of prognostic scoring models can improve the prediction of mortality among trauma patients in the intensive care unit (ICU).

Methods

This study's cohort included 1,126 trauma patients hospitalized in the ICU between January 1, 2018, and December 31, 2021. Two nutritional indicators, the prognostic nutrition index (PNI), a calculation based on the serum albumin concentration and peripheral blood lymphocyte count, and the geriatric nutritional risk index (GNRI), a calculation based on the serum albumin concentration and the ratio of current body weight to ideal body weight, were examined for their association with the mortality outcome. The significant nutritional indicator was served as an additional variable in prognostic scoring models of the Trauma and Injury Severity Score (TRISS), the Acute Physiology and Chronic Health Evaluation (APACHE II), and the mortality prediction models (MPM II) at admission, 24, 48, and 72 h in the mortality outcome prediction. The predictive performance was determined by the area under the receiver operating characteristic curve.

Results

Multivariate logistic regression revealed that GNRI (OR, 0.97; 95% CI, 0.96-0.99; p=0.007), but not PNI (OR, 0.99; 95% CI, 0.97-1.02; p=0.518), was independent risk factor for mortality. However, none of these predictive scoring models showed a significant improvement in prediction when the GNRI variable is incorporated.

Conclusions

The addition of GNRI as a variable to the prognostic scoring models did not significantly enhance the performance of the predictors.

Ranson score to stratify severity in Acute Pancreatitis remains valid - Old is gold

Introduction: Acute pancreatitis (AP) is a common gastrointestinal disease with a wide spectrum of severity and morbidity. Developed in 1974, the Ranson score was the first scoring system to prognosticate AP. Over the past decades, while the Ranson score remains widely used, it was identified to have certain limitations, such as having low predictive power. It has also been criticized for its 48-hour requirement for computation of the final score, which has been argued to potentially delay management. With advancements in our understanding of AP, is the Ranson score still relevant as an effective prognostication system for AP?Areas covered: This review summarizes the available evidence comparing Ranson score with other conventional and novel scoring systems, in terms of prognostic accuracy, benefits, limitations and clinical applicability. It also evaluates the effectiveness of Ranson score with regard to the Revised Atlanta Classification.Expert opinion: The Ranson score consistently exhibits comparable prognostic accuracy to other newer scoring systems, and the 48-hour timeframe for computing the full Ranson score is an inherent strength, not a weakness. These aspects, coupled with relative ease of use, practicality and universality of the score, advocate for the continued relevance of the Ranson score in modern clinical practice.

Overcoming challenges to data quality in the ASPREE clinical trial

BACKGROUND

Large-scale studies risk generating inaccurate and missing data due to the complexity of data collection. Technology has the potential to improve data quality by providing operational support to data collectors. However, this potential is under-explored in community-based trials. The Aspirin in reducing events in the elderly (ASPREE) trial developed a data suite that was specifically designed to support data collectors: the ASPREE Web Accessible Relational Database (AWARD). This paper describes AWARD and the impact of system design on data quality.

METHODS

AWARD's operational requirements, conceptual design, key challenges and design solutions for data quality are presented. Impact of design features is assessed through comparison of baseline data collected prior to implementation of key functionality (n = 1000) with data collected post implementation (n = 18,114). Overall data quality is assessed according to data category.

RESULTS

At baseline, implementation of user-driven functionality reduced staff error (from 0.3% to 0.01%), out-of-range data entry (from 0.14% to 0.04%) and protocol deviations (from 0.4% to 0.08%). In the longitudinal data set, which contained more than 39 million data values collected within AWARD, 96.6% of data values were entered within specified query range or found to be accurate upon querying. The remaining data were missing (3.4%). Participant non-attendance at scheduled study activity was the most common cause of missing data. Costs associated with cleaning data in ASPREE were lower than expected compared with reports from other trials.

CONCLUSIONS

Clinical trials undertake complex operational activity in order to collect data, but technology rarely provides sufficient support. We find the AWARD suite provides proof of principle that designing technology to support data collectors can mitigate known causes of poor data quality and produce higher-quality data. Health information technology (IT) products that support the conduct of scheduled activity in addition to traditional data entry will enhance community-based clinical trials. A standardised framework for reporting data quality would aid comparisons across clinical trials.

TRIAL REGISTRATION

International Standard Randomized Controlled Trial Number Register, ISRCTN83772183. Registered on 3 March 2005.

Evaluating ICU Clinical Severity Scoring Systems and Machine Learning Applications: APACHE IV/IVa Case Study

Clinical scoring systems have been developed for many specific applications, yet they remain underutilized for common reasons such as model inaccuracy and difficulty of use. For intensive care units specifically, the Acute Physiology and Chronic Health Evaluation (APACHE) score is used as a decision-making tool and hospital efficacy measure. In an attempt to alleviate the general underlying limitations of scoring instruments and demonstrate the utility of readily available medical databases, machine learning techniques were used to evaluate APACHE IV and IVa prediction measures in an open-source, teleICU research database. The teleICU database allowed for large-scale evaluation of APACHE IV and IVa predictions by comparing predicted values to the actual, recorded patient outcomes along with preliminary exploration of new predictive models for patient mortality and length of stay in both the hospital and the ICU. An increase in performance was observed in the newly developed models trained on the APACHE input variables highlighting avenues of future research and illustrating the utility of teleICU databases for model development and evaluation.

Efficacy of IgM-enriched Immunoglobulin for Vasopressor-resistant Vasoplegic Shock After Liver Transplantation

BACKGROUND

Vasoplegia is a clinical condition typically manifested by cardiovascular instability unresponsive to the usual doses of inotropes or vasopressors. It can occur in a variety of clinical settings including liver transplantation (LT). Immunoglobulins have been used to treat sepsis-related vasoplegia. We performed a retrospective study to evaluate the efficacy of IgM-enriched immunoglobulin (IgMIg) on 30-day mortality and its ability to reverse vasoplegia in patients undergoing LT.

METHODS

Between May 2013 and November 2017, 473 LT were performed at our institution. We identified 21 patients who received IgMIg for 3 days to treat vasoplegia. Patients included in the study met the criteria for having vasoplegia and required noradrenaline administration greater than 1 μg·kg·min for more than 24 hours to maintain a mean arterial pressure of 70 mm Hg or greater. Procalcitonin and interleukin-6 (IL-6) levels were used as surrogate markers for inflammation and were measured at the beginning and end of IgM treatment.

RESULTS

After IgMIg administration, median noradrenaline infusion rates could be significantly reduced from 1.6 μg·kg·min (1.3-2 μg·kg·min) to 0.16 μg·kg·min (0.08-0.34 μg·kg·min) (P < 0.001). In addition, after treatment, procalcitonin levels decreased significantly from 44 ng/mL (24-158) to 26.1 ng/mL (10.9-48.7) (P < 0.001) and IL-6 levels decreased significantly from 63 pg/mL (29-102) to 20 pg/mL (11-20) (P < 0.001). Thirty-day morality was 14.3%.

CONCLUSIONS

The administration of IgMIg in patients with vasoplegia after LT is associated with a return of hemodynamic stability. Despite a predicted mortality of over 90% by Sepsis-Related Organ Failure Assessment score, the mortality rate of patients receiving IgMIg in our study was less than 20%.

The Connected Intensive Care Unit Patient: Exploratory Analyses and Cohort Discovery From a Critical Care Telemedicine Database

BACKGROUND

Many intensive care units (ICUs) utilize telemedicine in response to an expanding critical care patient population, off-hours coverage, and intensivist shortages, particularly in rural facilities. Advances in digital health technologies, among other reasons, have led to the integration of active, well-networked critical care telemedicine (tele-ICU) systems across the United States, which in turn, provide the ability to generate large-scale remote monitoring data from critically ill patients.

OBJECTIVE

The objective of this study was to explore opportunities and challenges of utilizing multisite, multimodal data acquired through critical care telemedicine. Using a publicly available tele-ICU, or electronic ICU (eICU), database, we illustrated the quality and potential uses of remote monitoring data, including cohort discovery for secondary research.

METHODS

Exploratory analyses were performed on the eICU Collaborative Research Database that includes deidentified clinical data collected from adult patients admitted to ICUs between 2014 and 2015. Patient and ICU characteristics, top admission diagnoses, and predictions from clinical scoring systems were extracted and analyzed. Additionally, a case study on respiratory failure patients was conducted to demonstrate research prospects using tele-ICU data.

RESULTS

The eICU database spans more than 200 hospitals and over 139,000 ICU patients across the United States with wide-ranging clinical data and diagnoses. Although mixed medical-surgical ICU was the most common critical care setting, patients with cardiovascular conditions accounted for more than 20% of ICU stays, and those with neurological or respiratory illness accounted for nearly 15% of ICU unit stays. The case study on respiratory failure patients showed that cohort discovery using the eICU database can be highly specific, albeit potentially limiting in terms of data provenance and sparsity for certain types of clinical questions.

CONCLUSIONS

Large-scale remote monitoring data sources, such as the eICU database, have a strong potential to advance the role of critical care telemedicine by serving as a testbed for secondary research as well as for developing and testing tools, including predictive and prescriptive analytical solutions and decision support systems. The resulting tools will also inform coordination of care for critically ill patients, intensivist coverage, and the overall process of critical care telemedicine.

Evaluation of Random Plasma Cortisol and the Low Dose Corticotropin Test as Indicators of Adrenal Secretory Capacity in Critically Ill Patients: A Prospective Study

It is unclear whether a random plasma cortisol measurement and the corticotropin (ACTH) test adequately reflect glucocorticoid secretory capacity in critical illness. This study aimed to determine whether these tests provide information representative of the 24 hour period.

Plasma cortisol was measured hourly for 24 hours in 21 critically ill septic patients followed by a corticotropin test with 1 μg dose administered intravenously. Serum and urine were analysed for ACTH and free cortisol respectively. Marked hourly variability in plasma cortisol was evident (coefficient of variation 8–30%) with no demonstrable circadian rhythm. The individual mean plasma cortisol concentrations ranged from 286±59 nmol/l to 796± 83 nmol/l. The 24 hour mean plasma cortisol was strongly correlated with both random plasma cortisol (r2 0.9, P<0.0001) and the cortisol response to corticotropin (r2 0.72, P<0.001). Only nine percent of patients increased their plasma cortisol by 250 nmol/l after corticotropin (euadrenal response). However, 35% of non-responders had spontaneous hourly rises >250 nmol/l thus highlighting the limitations of a single point corticotropin test. Urinary free cortisol was elevated (865±937 nmol) in both corticotropin responders and non-responders suggesting elevated plasma free cortisol. No significant relationship was demonstrable between plasma cortisol and ACTH.

We conclude that although random cortisol measurements and the low dose corticotropin tests reliably reflect the 24 hour mean cortisol in critical illness, they do not take into account the pulsatile nature of cortisol secretion. Consequently, there is the potential for erroneous conclusions about adrenal function based on a single measurement. We suggest that caution be exercised when drawing conclusions on the adequacy of adrenal function based on a single random plasma cortisol or the corticotropin test.

Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India

BACKGROUND

There are few published standards or methodological guidelines for integrating Data Quality Assurance (DQA) protocols into large-scale health systems research trials, especially in resource-limited settings. The BetterBirth Trial is a matched-pair, cluster-randomized controlled trial (RCT) of the BetterBirth Program, which seeks to improve quality of facility-based deliveries and reduce 7-day maternal and neonatal mortality and maternal morbidity in Uttar Pradesh, India. In the trial, over 6300 deliveries were observed and over 153,000 mother-baby pairs across 120 study sites were followed to assess health outcomes. We designed and implemented a robust and integrated DQA system to sustain high-quality data throughout the trial.

METHODS

We designed the Data Quality Monitoring and Improvement System (DQMIS) to reinforce six dimensions of data quality: accuracy, reliability, timeliness, completeness, precision, and integrity. The DQMIS was comprised of five functional components: 1) a monitoring and evaluation team to support the system; 2) a DQA protocol, including data collection audits and targets, rapid data feedback, and supportive supervision; 3) training; 4) standard operating procedures for data collection; and 5) an electronic data collection and reporting system. Routine audits by supervisors included double data entry, simultaneous delivery observations, and review of recorded calls to patients. Data feedback reports identified errors automatically, facilitating supportive supervision through a continuous quality improvement model.

RESULTS

The five functional components of the DQMIS successfully reinforced data reliability, timeliness, completeness, precision, and integrity. The DQMIS also resulted in 98.33% accuracy across all data collection activities in the trial. All data collection activities demonstrated improvement in accuracy throughout implementation. Data collectors demonstrated a statistically significant (p = 0.0004) increase in accuracy throughout consecutive audits. The DQMIS was successful, despite an increase from 20 to 130 data collectors.

CONCLUSIONS

In the absence of widely disseminated data quality methods and standards for large RCT interventions in limited-resource settings, we developed an integrated DQA system, combining auditing, rapid data feedback, and supportive supervision, which ensured high-quality data and could serve as a model for future health systems research trials. Future efforts should focus on standardization of DQA processes for health systems research.

TRIAL REGISTRATION

ClinicalTrials.gov identifier, NCT02148952 . Registered on 13 February 2014.

Bacterial species-specific hospital mortality rate for intra-abdominal infections

BACKGROUND

Intra-abdominal infections (IAIs) are a major cause of morbidity and death. We hypothesized that the involvement of specific organisms would predict death independently.

PATIENTS AND METHODS

All patients with IAIs treated at an academic tertiary-care facility over eight years (June 1999-June 2007) were included. The data collected were demographics, co-morbidities, source of infection, intra-abdominal culture results, type of infection (community-acquired vs. nosocomial), type of intervention (operative vs. percutaneous drainage), and outcome. The Charlson Comorbidity Index and multiple organ dysfunction score (MODS) were used in the analysis.

RESULTS

A total of 389 patients were admitted for 452 infection episodes (IEs) during the study period. None of the 129 patients with appendiceal-related infections died, and these patients were excluded from further analysis. Thus, 323 non-appendiceal IEs were evaluated. The overall mortality rate was 8.7%. The mean age of the patients was 54 y, and 50% of them were male. Intra-abdominal cultures were obtained from 303 IEs (93.8%). The most common cause of IAI was post-operative infection (44%). There were 49 distinct species isolated. The most common were Enterococcus (105), Escherichia coli (75), Streptococcus (62), Staphylococcus (51), and Bacteroides (46). Bivariable analysis revealed multiple risk factors associated with death. Logistic regression demonstrated that independent risk factors for death were age ≥65 years (odds ratio [OR] 3.92), cardiac event (OR=8.17), catheter-related blood stream infection (OR=6.16), and growth of Clostridium (OR=13.03). The growth of Streptococcus was predictive of survival. The C statistic was 0.89.

CONCLUSIONS

In addition to age and intrinsic patient factors, the presence of specific bacterial organisms independently predicts death in patients with non-appendiceal IAI.

New scoring system (APACHE-HF) for predicting adverse outcomes in patients with acute heart failure: evaluation of the APACHE II and Modified APACHE II scoring systems

BACKGROUND

No scoring system for assessing acute heart failure (AHF) has been reported.

METHODS AND RESULTS

Data for 824 AHF patients were analyzed. The subjects were divided into an alive (n=750) and a dead group (n=74). We constructed a predictive scoring system based on eight significant APACHE II factors in the alive group [mean arterial pressure (MAP), pulse, sodium, potassium, hematocrit, creatinine, age, and Glasgow Coma Scale (GCS); giving each one point], defined as the APACHE-HF score. The patients were assigned to five groups by the APACHE-HF score [Group 1: point 0 (n=70), Group 2: points 1 and 2 (n=343), Group 3: points 3 and 4 (n=294), Group 4: points 5 and 6 (n=106), and Group 5: points 7 and 8 (n=11)]. A higher optimal balance was observed in the APACHE-HF between sensitivity and specificity [87.8%, 63.9%; area under the curve (AUC)=0.779] at 2.5 points than in the APACHE II (47.3%, 67.3%; AUC=0.558) at 17.5 points. The multivariate Cox regression model identified belonging to Group 5 [hazard ratio (HR): 7.764, 95% confidence interval (CI) 1.586-38.009], Group 4 (HR: 6.903, 95%CI 1.940-24.568) or Group 3 (HR: 5.335, 95%CI 1.582-17.994) to be an independent predictor of 3-year mortality. The Kaplan-Meier curves revealed a poorer prognosis, including all-cause death and HF events (death, readmission-HF), in Group 5 and Group 4 than in the other groups, in Group 3 than in Group 2 or Group 1, and in Group 2 than in Group 1.

CONCLUSIONS

The new scoring system including MAP, pulse, sodium, potassium, hematocrit, creatinine, age, and GCS (APACHE-HF) can be used to predict adverse outcomes of AHF.

A centralized research data repository enhances retrospective outcomes research capacity: a case report

This paper describes our considerations and methods for implementing an open-source centralized research data repository (CRDR) and reports its impact on retrospective outcomes research capacity in the urology department at Columbia University. We performed retrospective pretest and post-test analyses of user acceptance, workflow efficiency, and publication quantity and quality (measured by journal impact factor) before and after the implementation. The CRDR transformed the research workflow and enabled a new research model. During the pre- and post-test periods, the department's average annual retrospective study publication rate was 11.5 and 25.6, respectively; the average publication impact score was 1.7 and 3.1, respectively. The new model was adopted by 62.5% (5/8) of the clinical scientists within the department. Additionally, four basic science researchers outside the department took advantage of the implemented model. The average proximate time required to complete a retrospective study decreased from 12 months before the implementation to <6 months after the implementation. Implementing a CRDR appears to be effective in enhancing the outcomes research capacity for one academic department.

Critical Care Nurses Inadequately Assess SAPS II Scores of Very Ill Patients in Real Life

Background. Reliable ICU severity scores have been achieved by various healthcare workers but nothing is known regarding the accuracy in real life of severity scores registered by untrained nurses. Methods. In this retrospective multicentre audit, three reviewers independently reassessed 120 SAPS II scores. Correlation and agreement of the sum-scores/variables among reviewers and between nurses and the reviewers' gold standard were assessed globally and for tertiles. Bland and Altman (gold standard—nurses) of sum scores and regression of the difference were determined. A logistic regression model identifying risk factors for erroneous assessments was calculated. Results. Correlation for sum scores among reviewers was almost perfect (mean ICC = 0.985). The mean (±SD) nurse-registered SAPS II sum score was 40.3 ± 20.2 versus 44.2 ± 24.9 of the gold standard (P < 0.002 for difference) with a lower ICC (0.81). Bland and Altman assay was +3.8 ± 27.0 with a significant regression between the difference and the gold standard, indicating overall an overestimation (underestimation) of lower (higher; >32 points) scores. The lowest agreement was found in high SAPS II tertiles for haemodynamics (k = 0.45–0.51). Conclusions. In real life, nurse-registered SAPS II scores of very ill patients are inaccurate. Accuracy of scores was not associated with nurses' characteristics.

"Summary Page": a novel tool that reduces omitted data in research databases

Background

Data entry errors are common in clinical research databases. Omitted data are of particular concern because they are more common than erroneously inserted data and therefore could potentially affect research findings. However, few affordable strategies for their prevention are available.

Methods

We have conducted a prospective observational study of the effect of a novel tool called "Summary Page" on the frequency of correction of omitted data errors in a radiation oncology research database between July 2008 and March 2009. "Summary Page" was implemented as an optionally accessed screen in the database that visually integrates key fields in the record. We assessed the frequency of omitted data on the example of the Date of Relapse field. We considered the data in this field to be omitted for all records that had empty Date of Relapse field and evidence of relapse elsewhere in the record.

Results

A total of 1,156 records were updated and 200 new records were entered in the database over the study period. "Summary Page" was accessed for 44% of all updated records and for 69% of newly entered records. Frequency of correction of the omitted date of cancer relapse was six-fold higher in records for which "Summary Page" was accessed (p = 0.0003).

Conclusions

"Summary Page" was strongly associated with an increased frequency of correction of omitted data errors. Further, controlled, studies are needed to confirm this finding and elucidate its mechanism of action.

The Lombardia Stroke Unit Registry: 1-year experience of a web-based hospital stroke registry

This paper presents methodological aspects of the Lombardia Stroke Registry. At the registry start-up, 36 recruiting centres were identified according to a regional survey. The registry recruits consecutive patients with acute stroke or transient ischaemic attacks (TIAs). A 3-month follow-up was planned to correlate acute care with outcomes. On 31st December 2007, data concerning 6,181 patients discharged alive were available. The registry aims at measuring performance parameters, identifying guidelines non-compliance and analysing care processes. In this first phase, 30% of the Lombardia acute stroke and 10% of TIA patients have been enrolled, thus the sample can be considered informative for the disease care in the region. The proportion of completed data items is very high with very small differences among items. The following critical points were highlighted: (1) lack of data input staff for 30% of centres, and (2) difficulty of obtaining the informed consent for post-discharge follow-up.

Designing and implementing an Australian and New Zealand intensive care data audit study

PURPOSE

The aim of this pilot audit study is to develop and test a model to examine existing adult patient database (APD) data quality.

DESIGN/METHODOLOGY/APPROACH

A database was created to audit 50 records per site to determine accuracy. The audited records were randomly selected from the calendar year 2004 and four sites participated in the pilot audit study. A total of 41 data elements were assessed for data quality--those elements required for APACHE II scoring system.

FINDINGS

Results showed that the audit was feasible; missing audit data were an unplanned problem; analysis was complicated owing to the way the APACHE calculations are performed and 50 records per site was too time-consuming.

ORIGINALITY/VALUE

This is the first audit study of intensive care data within the ANZICS APD and demonstrates how to determine data quality in a large database containing individual patient records.

Reliability and accuracy of practitioner-calculated Acute Physiology and Chronic Health Evaluation II scores for determining the appropriateness of drotrecogin alfa (activated)

PURPOSE

The reliability and accuracy of practitioner-calculated Acute Physiology and Chronic Health Evaluation (APACHE) II scores for determining the appropriateness of drotrecogin alfa (activated) in critically ill patients were evaluated.

METHODS

Three adjudicated clinical cases of sepsis were developed using composites of real patient scenarios. The patients' APACHE II scores were independently assessed by randomly selected critical care practitioners (physicians and nonphysicians). Each case contained at least one reason to consider withholding drotrecogin alfa (activated), but none had a definitive contraindication to drotrecogin alfa (activated). Intraobserver and interobserver variabilities were assessed using kappa correlation. Accuracy was assessed by comparing median scores to the adjudicated scores and evaluating correctly classified APACHE II scores.

RESULTS

A total of 21 (42%) physicians and 14 (56%) nonphysicians completed all assessments. Intraobserver and interobserver variabilities were 0.16 and 0.49 for the total APACHE II score, respectively. Median calculated APACHE II scores significantly differed for case 1 (p = 0.003) and case 3 (p < 0.0001). The percentage of error in calculating the total APACHE II score approached 85%. The main reasons for administering drotrecogin alfa (activated) were an APACHE II score of >or=25 and multiple organ failures. The main reason for therapy was a high bleeding risk or an APACHE II score of <25.

CONCLUSION

Weak intraobserver agreement, modest interobserver reliability, a high error rate, and low accuracy limited the clinical application of the APACHE II score by untrained practitioners, indicating that the APACHE II score should not be the only determinant for the use of drotrecogin alfa (activated).

Reliability of intensive care unit admitting and comorbid diagnoses, race, elements of Acute Physiology and Chronic Health Evaluation II score, and predicted probability of mortality in an electronic intensive care unit database

BACKGROUND

Although reliability of severity of illness and predicted probability of hospital mortality have been assessed, interrater reliability of the abstraction of primary and other intensive care unit (ICU) admitting diagnoses and underlying comorbidities has not been studied.

METHODS

Patient data from one ICU were originally abstracted and entered into an electronic database by an ICU nurse. A research assistant reabstracted patient demographics, ICU admitting diagnoses and underlying comorbidities, and elements of Acute Physiology and Chronic Health Evaluation II (APACHE II) score from 100 random patients of 474 admitted during 2005 using an identical electronic database. Chamberlain's percent positive agreement was used to compare diagnoses and comorbidities between the 2 data abstractors. A kappa statistic was calculated for demographic variables, Glasgow Coma Score, APACHE II chronic health points, and HIV status. Intraclass correlation was calculated for acute physiology points and predicted probability of hospital mortality.

RESULTS

Percent positive agreement for ICU primary and other admitting diagnoses ranged from 0% (primary brain injury) to 71% (sepsis), and for underlying comorbidities, from 40% (coronary artery bypass graft) to 100% (HIV). Agreement as measured by kappa statistic was strong for race (0.81) and age points (0.95), moderate for chronic health points (0.50) and HIV (0.66), and poor for Glasgow Coma Score (0.36). Intraclass correlation showed a moderate-high agreement for acute physiology points (0.88) and predicted probability of hospital mortality (0.71).

CONCLUSION

Reliability for ICU diagnoses and elements of the APACHE II score is related to the objectivity of primary data in the medical charts.

Accuracy of Sequential Organ Failure Assessment (SOFA) scoring in clinical practice

BACKGROUND

The Sequential Organ Failure Assessment (SOFA) score is used to quantify the severity of illness daily during intensive care. Our aim was to evaluate how accurately SOFA is recorded in clinical practice, and whether this can be improved by a refresher course in scoring rules.

METHODS

The scores recorded by physicians in a university hospital intensive care unit (ICU) were compared with the gold standard determined by two expert assessors. Data concerning all consecutive patients during two 6-week-long observation periods (baseline and after the refresher course) were compared.

RESULTS

SOFA was accurate on 75/158 (48%) patient days at baseline. The cardiovascular, coagulation, liver, and renal component scores showed excellent accuracy (>or=82%, weighted kappa >or=0.92), while the neurological score showed only moderate (70%, weighted kappa 0.51) and the respiration score showed good accuracy (75%, weighted kappa 0.79). After the refresher course, the number of >or=2 point errors decreased (P<0.01). Sedation precluded neurological evaluation on 135/311 (43%) days. The accuracy of the assumed neurological scores was lower than those based on timely data: 89/135 (66%, weighted kappa 0.55) vs. 125/176 (71%, weighted kappa 0.81) (P<0.01).

CONCLUSION

Only half of the SOFA scores were accurate. In most cases, they were accurate enough to allow the recognition of organ failure and detection of change. The component scores showed good to excellent accuracy, except the neurological score. After the refresher course, the results improved slightly. The moderate accuracy of the neurological score was not amended. A simpler neurological classification tool than the Glasgow Coma Scale is needed in the ICU.

Quantifying data quality for clinical trials using electronic data capture

Background

Historically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes.

Methods and Principal Findings

The National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions.

Conclusions

Historically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

Inter-rater reliability of data elements from a prototype of the Paul Coverdell National Acute Stroke Registry

BACKGROUND

The Paul Coverdell National Acute Stroke Registry (PCNASR) is a U.S. based national registry designed to monitor and improve the quality of acute stroke care delivered by hospitals. The registry monitors care through specific performance measures, the accuracy of which depends in part on the reliability of the individual data elements used to construct them. This study describes the inter-rater reliability of data elements collected in Michigan's state-based prototype of the PCNASR.

METHODS

Over a 6-month period, 15 hospitals participating in the Michigan PCNASR prototype submitted data on 2566 acute stroke admissions. Trained hospital staff prospectively identified acute stroke admissions, abstracted chart information, and submitted data to the registry. At each hospital 8 randomly selected cases were re-abstracted by an experienced research nurse. Inter-rater reliability was estimated by the kappa statistic for nominal variables, and intraclass correlation coefficient (ICC) for ordinal and continuous variables. Factors that can negatively impact the kappa statistic (i.e., trait prevalence and rater bias) were also evaluated.

RESULTS

A total of 104 charts were available for re-abstraction. Excellent reliability (kappa or ICC > 0.75) was observed for many registry variables including age, gender, black race, hemorrhagic stroke, discharge medications, and modified Rankin Score. Agreement was at least moderate (i.e., 0.75 > kappa >/=; 0.40) for ischemic stroke, TIA, white race, non-ambulance arrival, hospital transfer and direct admit. However, several variables had poor reliability (kappa < 0.40) including stroke onset time, stroke team consultation, time of initial brain imaging, and discharge destination. There were marked systematic differences between hospital abstractors and the audit abstractor (i.e., rater bias) for many of the data elements recorded in the emergency department.

CONCLUSION

The excellent reliability of many of the data elements supports the use of the PCNASR to monitor and improve care. However, the poor reliability for several variables, particularly time-related events in the emergency department, indicates the need for concerted efforts to improve the quality of data collection. Specific recommendations include improvements to data definitions, abstractor training, and the development of ED-based real-time data collection systems.

Use of the All Patient Refined-Diagnosis Related Group (APR-DRG) Risk of Mortality Score as a Severity Adjustor in the Medical ICU

Objective:

To evaluate the performance of APR-DRG (All Patient Refined—Diagnosis Related Group) Risk of Mortality (ROM) score as a mortality risk adjustor in the intensive care unit (ICU).

Design:

Retrospective analysis of hospital mortality.

Setting:

Medical ICU in a university hospital located in metropolitan New York.

Patients:

1213 patients admitted between February 2004 and March 2006.

Main results:

Mortality rate correlated significantly with increasing APR-DRG ROM scores (p < 0.0001). Multiple logistic regression analysis demonstrated that, after adjusting for patient age and disease group, APR-DRG ROM was significantly associated with mortality risk in patients, with a one unit increase in APR-DRG ROM associated with a 3-fold increase in mortality.

Conclusions:

APR-DRG ROM correlates closely with ICU mortality. Already available for many hospitalized patients around the world, it may provide a readily available means for severity-adjustment when physiologic scoring is not available.

Cumulative index of health deficiencies as a characteristic of long life

OBJECTIVES

To describe the accumulation of aging-associated health disorders using a cumulative measure known as a frailty index (FI) and to evaluate its ability to differentiate long- and short-life phenotypes as well as the FI's connection to aging-associated processes in older people.

DESIGN

Retrospective cross-sectional and longitudinal studies.

SETTING

The National Long-Term Care Survey (NLTCS) data that assessed health and functioning of U.S. older individuals (> or =65) in 1982, 1984, 1989, 1994, and 1999 were analyzed. The NLTCS sample in each survey represents a mixture of longitudinal and cross-sectional components.

PARTICIPANTS

Approximately 5,000 individuals in each survey.

MEASUREMENTS

A cumulative index of health and well-being deficiencies (disabilities, signs, diseases) was calculated as a count of deficits observed in an individual divided by the total number of all considered deficits.

RESULTS

Men and women who died before the age of 75 and those who died after the age of 85 exhibited remarkably similar FI frequency patterns despite the 10-year age difference between age profiles in these samples. Long life is consistently characterized in longitudinal analyses by lower FIs. FI dynamics are found to be strongly sex sensitive.

CONCLUSION

The FI appears to be a sensitive age-independent indicator of sex-specific physiological decline in aging individuals and a sex-specific discriminator of survival chances. The FI is a promising characteristic suitable for improving sex-sensitive forecasts of risks of adverse health outcomes in older people.

Interrater Reliability of APACHE II Scores for Medical-Surgical Intensive Care Patients: A Prospective Blinded Study

Background Despite widespread use of the Acute Physiology and Chronic Health Evaluation II (APACHE II), its interrater reliability has not been well studied.

Objective To determine interrater reliability of APACHE II scores among 1 intensive care nurse and 2 research clerks.

Methods In a prospective, blinded, observational study, 3 raters collected APACHE II scores on 37 consecutive patients in a medical-surgical intensive care unit. One research clerk was blinded to the study’s start date to minimize observer bias. The nurse and the other research clerk were blinded to each other’s scores and did not communicate with the first research clerk about the study. The data analyst was blinded to the identity and source of all 3 raters’ scores. Intraclass correlation coefficients and 95% confidence intervals were assessed.

Results Mean (standard deviation) APACHE II scores were 21.8 (9.2) for the nurse, 20.4 (7.7) for research clerk 1, and 20.5 (8.1) for research clerk 2. Among the 3 raters, the intraclass correlation coefficient (95% confidence interval) was 0.90 (0.84, 0.94) for the APACHE II total score. Within APACHE II score components, the highest reliability was for age (0.98 [0.97, 0.99]), with lower reliabilities for the Chronic Health Index (0.64 [0.50, 0.80]) and the verbal component of the Glasgow Coma Scale (0.40 [0.20, 0.60]). Results were similar between pairs of raters.

Conclusions Use of trained nonmedical personnel to collect illness severity scores for clinical, research, and administrative purposes is reasonable. This method could be used to assess reliability of other illness severity scores.

Genetic variants in the NOD2/CARD15 gene are associated with early mortality in sepsis patients

OBJECTIVE

Genetic variants in the NOD2/CARD15 gene resulting in a diminished capacity to activate NF-kappaB in response to bacterial cell wall products have been associated with Crohn's disease (CD). Recently, we found an association between the variant Leu1007fsinsC of the NOD2/CARD15 gene (SNP13) and a significantly increased rate of transplant related mortality (TRM) due to intestinal and pulmonary complications in stem cell transplantation (SCT). To assess a possible contribution of variants in the NOD2/CARD15 gene to sepsis related mortality (SRM) we investigated 132 prospectively characterised, consecutive patients with sepsis.

DESIGN AND PATIENTS

The three most common NOD2/CARD15 variants (Arg702Trp, Gly908Arg, and Leu1007fsinsC) were determined in 132 prospectively characterised patients with sepsis attended to three intensive care units at the University of Regensburg by Taqman PCR. NOD2/CARD15 genotype and major patients' characteristics were correlated with SRM.

RESULTS

Patient groups with and without NOD2/CARD15 variants did not differ in their clinical characteristics such as median age, gender, reason for admission or APACHE score; however, SRM (day 30) was increased in patients with NOD2/CARD15 coding variants (42 vs. 31%) and was highest (57%) in 8 patients carrying the Leu1007fsinsC variant (p < 0.05). Multivariate analysis demonstrated the Leu1007fsinsC genetic variant as an independent risk factor for SRM.

CONCLUSION

Our findings indicate a major role of NOD2/CARD15 coding variants for SRM. This may be indicative for a role of impaired barrier function and bacterial translocation in the pathophysiology of early sepsis related death.

Analysis of data-collection methods for an acute stroke care registry

This study aims to assess and compare the completeness and reliability of data collected by prospective and retrospective methods for the Paul Coverdell National Acute Stroke Registry. The prototypes consisted of eight states that used the same data elements but differed in their collection approach. Three prototypes employed retrospective case ascertainment (n=1218), and five prototypes used prospective or a combination of prospective and retrospective case ascertainment (n=1602). RTI International performed an audit analysis of the eight prototypes. Completeness, exact match, and discrepancy analyses were performed with data elements grouped into 12 categories for this analysis. A sample of 2820 (37.6%) from a total of 7494 records from 91 hospitals was studied. The "in-hospital complications" section had the highest percentage of completeness (99.6%), followed by "demographic data" (97.7%), and "in-hospital diagnostic procedures" (93.4%). The section with the lowest percentage of completeness was "thrombolytic treatment" (53.5%), followed by "reasons for nontreatment with thrombolytics" (57.1%), and "signs and symptoms onset" (63.5%). Across all prototype elements, exact matches with audit data ranged from 62.8% to 95.9%. Documentation of the date/time of stroke onset and of arrival in the emergency department had a high number of discrepancies with audit data, with exact match percentages of 69.7% and 64.5%, respectively. No significant difference was found between retrospective and prospective case ascertainment in completeness or matching with audit data. Combined retrospective and prospective data-collection approaches for different types of data elements may be best in terms of both completeness and accuracy.

Optimal prediction of mortality after abdominal aortic aneurysm repair with statistical models

OBJECTIVE

To identify the best method for the prediction of postoperative mortality in individual abdominal aortic aneurysm surgery (AAA) patients by comparing statistical modelling with artificial neural networks' (ANN) and clinicians' estimates.

METHODS

An observational multicenter study was conducted of prospectively collected postoperative Acute Physiology and Chronic Health Evaluation II data for a 9-year period from 24 intensive care units (ICU) in the Thames region of the United Kingdom. The study cohort consisted of 1205 elective and 546 emergency AAA patients. Four independent physiologic variables-age, acute physiology score, emergency operation, and chronic health evaluation-were used to develop multiple regression and ANN models to predict in-hospital mortality. The models were developed on 75% of the patient population and their validity tested on the remaining 25%. The results from these two models were compared with the observed outcome and clinicians' estimates by using measures of calibration, discrimination, and subgroup analysis.

RESULTS

Observed in-hospital mortality for elective surgery was 9.3% (95% confidence interval [CI], 7.7% to 11.1%) and for emergency surgery, 46.7% (95% CI, 42.5 to 51.0%). The ANN and the statistical models were both more accurate than the clinicians' predictions. Only the statistical model was internally valid, however, when applied to the validation set of observations, as evidenced by calibration (Hosmer-Lemeshow C statistic, 14.97; P = .060), discrimination properties (area under receiver operating characteristic curve, 0.869; 95% CI, 0.824 to 0.913), and subgroup analysis.

CONCLUSIONS

The prediction of in-hospital mortality in AAA patients by multiple regression is more accurate than clinicians' estimates or ANN modelling. Clinicians can use this statistical model as an objective adjunct to generate informed prognosis.

Impact of operator expertise on collection of the APACHE II score and on the derived risk of death and standardized mortality ratio

We assessed the impact of operator expertise on collection of the APACHE II score, the derived risk of death and standardized mortality ratio in 465 consecutive patients admitted to a multi-disciplinary tertiary hospital ICU. Research coordinators and junior clinical staff independently collected the APACHE II variables; experts (senior clinical staff) rescored 20% of the records. Agreement was moderate between junior clinical staff and research coordinators or senior clinical staff for most variables of the acute physiology score (weighted kappa<0.6); agreement between research coordinators and senior clinical staff data collectors was good (weighted kappa >0.75). The APACHE II score and its derived risk of death (ROD) were significantly lower using the junior clinical staff dataset compared to research coordinators and senior clinical staff (APACHE II score: 13.4+/-9.2 vs 16.8+/-8.5 vs 17.1+/-7 7, P<0.001; ROD: 14.7%+/-22.4% vs 21.6%+/-22.6% vs 20.8%+/-22.4%, P<0.01 respectively). The discriminative capacity was not altered by the lack of agreement (area under Receiver Operator Characteristic curve >0.8) but calibration of ROD from the junior clinical staff dataset was poor (Goodness-of-fit: P= 0.001). The standardized mortality ratio (SMR) was higher with the junior clinical staff dataset (SMR: 1.22, 95% CI: 0.96-1.52 vs 0.87, 95% CI: 0.70-1.06 vs 0.76, 95% CI: 0.40-1.3 calculated from junior clinical staff research coordinators and senior clinical staff datasets respectively). We conclude that the expertise of data collectors significantly influences the APACHE II score, the derived risk of death and the standardized mortality ratio. Given the importance of such scores, ICUs should be provided with sufficient resources to train and employ dedicated data collectors.

A nationwide survey of intensive care unit discharge practices

OBJECTIVE

To describe intensive care unit (ICU) discharge practices, examine factors associated with physicians' discharge decisions, and explore ICU and hospital characteristics and clinical determinants associated with the discharge process.

DESIGN

Survey in adult ICUs affiliated with the Swiss Society of Intensive Care Medicine.

INTERVENTIONS

Questionnaire inquiring about ICU structure and organization mailed to 73 medical directors. Level of monitoring, intravenous medications, and physiological variables were proposed as elements of discharge decision. Five clinical situations were presented with request to assign a discharge disposition.

MEASUREMENTS AND RESULTS

Fifty-five ICUs participated, representing 75% of adult Swiss ICUs. Responsibility for patient management was assigned in 91% to the ICU team directing patient care. Only 22% of responding centers used written discharge guidelines. One-half of the respondents considered at least 10 of 15 proposed criteria to decide patient discharge. ICUs in central referral hospitals used fewer criteria than community and private hospitals. The availability of intermediate care units was significantly greater in university hospitals. The ICU director's level of experience was not associated with the number of criteria used. In the five clinical scenarios there was wide variation in discharge decision.

CONCLUSIONS

Our data indicate that there is marked heterogeneity in ICUs discharge practices, and that discharge decisions may be influenced by institutional factors. University teaching hospitals had more intermediate care facilities available. Written discharge guidelines were not widely used.

The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance

OBJECTIVE

To determine the impact of missing Acute Physiology Score (APS) values on risk-adjusted mortality.

DESIGN

Retrospective review of prospectively collected Acute Physiology and Chronic Health Evaluation (APACHE) III database.

SETTING

The intensive care units (ICUs) of an academic medical center.

PATIENTS

38,411 patients admitted to ICU between October 1994 and December 2003.

MEASUREMENTS AND RESULTS

Data were collected on ICU type, missing first ICU day APS values, predicted and observed hospital mortality, standardized mortality ratio (SMR), 95% confidence interval (CI), odds ratio (OR). The overall observed and predicted hospital mortality rates were 8.7% and 10.8%, respectively, with SMR of 0.806 (95% CI 0.779-0.834). Complete data were available in 829 (2.2%). Vital signs were missing in almost none and serum albumin and bilirubin in over 80% of the patients. The number of missing variables was higher in less sick and surgical ICU patients. Logistic regression analysis showed that the risk of dying in the hospital was significantly associated with the number of missing APS variables (OR 1.058, 95% CI 1.027-1.090) when adjusted for the severity of illness. The risk of death was also associated with the type of missing variables.

CONCLUSIONS

Since missing APS values may lead to underestimation of the predicted mortality rates, the number and type of missing variables should be taken into consideration when assessing the performance of an ICU. Unless data collection is standardized, future prognostic models should use variables that are routinely measured in most critically ill patients without sacrificing statistical precision.

Quantification of mortality risk after abdominal aortic aneurysm repair

Background

The study was designed to evaluate the Acute Physiology And Chronic Health Evaluation (APACHE) II risk scoring system in abdominal aortic aneurysm (AAA) surgery. The aim was to create an APACHE-based risk stratification model for postoperative death.

Methods

Prospective postoperative APACHE II data were collected from patients undergoing AAA repair over a 9-year interval from 24 intensive care units (ICUs) in the Thames region. A multilevel logistic regression model (APACHE-AAA) for in-hospital mortality was developed to adjust for both case mix and the variation in outcome between ICUs.

Results

A total of 1896 patients were studied. The in-hospital mortality rate among the 1289 patients who had elective AAA repair was 9·6 (95 per cent confidence interval (c.i.) 8·0 to 11·2) per cent and that among the 605 patients who had an emergency repair was 46·9 (95 per cent c.i. 43·0 to 50·9) per cent. Four independent predictors of death were identified: age (odds ratio (OR) 1·05 (95 per cent c.i. 1·03 to 1·07) per year increase), Acute Physiology Score (OR 1·14 (95 per cent c.i. 1·12 to 1·17) per unit increase), emergency operation (OR 4·86 (95 per cent c.i. 3·64 to 6·52)) and chronic health dysfunction (OR 1·43 (95 per cent c.i. 1·04 to 1·97)). The APACHE-AAA model was internally valid, as shown by calibration (Hosmer–Lemeshow C statistic: χ2 = 6·14, 8 d.f., P = 0·632), discrimination properties (area under receiver–operator characteristic curve 0·845) and subgroup analysis. There was no significant variation in outcome between hospitals.

Conclusion

APACHE-AAA was shown to be an accurate risk-stratification model that could be used to quantify the risk of death after AAA surgery. It might also be used to determine the relative impact of ICU over high-dependency unit care.

The longer patients are in hospital before Intensive Care admission the higher their mortality

OBJECTIVE

To explore the relationship between hospital mortality and time spent by patients on hospital wards before admission to the intensive care unit (ICU).

DESIGN

Observational study of prospectively collected data.

SETTING

Participating intensive care units within the North East Thames Regional Database.

PATIENTS AND PARTICIPANTS

Patients, 7,190, admitted to ICU from the hospital wards of 24 hospitals.

INTERVENTIONS

None.

MEASUREMENTS AND RESULTS

Of ICU admissions from the wards, 40.1% were in hospital for more than 3 days and 11.7% for more than 15 days. ICU patients who died in hospital were in-patients longer (p=0.001) before admission (median 3 days; interquartile range 1-9) than those discharged alive (median 2 days; interquartile range 1-5). Hospital mortality increased significantly (p<0.0001) in relation to time on hospital wards before ICU: 47.1% (standardised mortality ratio 1.09) for patients in hospital 0-3 days before ICU admission up to 67.2% (standardised mortality ratio 1.39) for patients on the wards for more than 15 days before ICU. Length of stay before ICU admission was an independent predictor of hospital mortality (odds ratio per day 1.019; 95% confidence interval 1.014-1.024). There were significant differences (p<0.001) in patient age, APACHE II score and predicted mortality in relation to time on wards before ICU admission.

CONCLUSIONS

Mortality was high among patients admitted from the wards to ICU; many were inpatients for days or weeks before admission. The longer these patients were in hospital before ICU admission, the higher their mortality. Patients with delayed admission differed in some respects compared to those admitted earlier.

ICU nurses' knowledge of, and attitudes towards, the APACHE II scoring system

AIMS AND OBJECTIVES

This study was undertaken to determine whether intensive care unit nurses possess adequate knowledge to implement the Acute Physiological and Chronic Health Evaluation (APACHE) II scoring system and to understand nurses' attitudes towards this scoring system.

DESIGN

A questionnaire, which contained 20 multiple-choice questions to test knowledge of the scoring system and 27 statements with a five-points Likert type scale to assess attitude, was developed by researchers for this study.

METHODS

The participants consisted of 102 intensive care unit nurses working in adult, emergency, or coronary intensive care units in a large teaching hospital (Veterans General Hospital) in Taiwan, where the APACHE II scoring system is used routinely by nurses.

RESULTS

On the knowledge section of the questionnaire, 76% of the questions were answered correctly. However, more than 50% of the nurses were uncertain about how to score a patient's chronic history status, only 44.1% of the nurses knew that APACHE II total scores range from 0 to 71, and only 37.3% understood that the value for bicarbonate in venous samples could replace the arterial pH value. The results of the attitude questionnaire revealed that nurses believed APACHE II was useful mainly for statistical purposes by the administration rather than for patient care, but acknowledged that the scores could provide a reference for more aggressive treatment.

CONCLUSIONS

The questionnaire answers indicated that greater knowledge concerning the APACHE II is needed.

RELEVANCE TO CLINICAL PRACTICE

Information and training concerning the proper use and purpose of APACHE II needs to be provided, especially for those intensive care unit nurses using this evaluation tool to score patients' conditions routinely.

Procalcitonin, C-reactive protein, and endotoxin after bone marrow transplantation: identification of children at high risk of morbidity and mortality from sepsis

We prospectively evaluated the capacity of serum procalcitonin (PCT), compared with serum levels of C-reactive protein (CRP) and endotoxin, to identify children at high risk for mortality from sepsis after BMT. Of 47 pediatric bone marrow transplantation patients studied, 22 had an uneventful course post-transplant (Group 1), 17 survived at least one septic event (Group 2), and eight died from multiorgan failure (MOF) following septic shock (Group 3). Median concentrations of PCT over the course of the study were 1.3, 15.2, and 102.8 ng/ml, respectively, in each of the three groups (P<0.002 for each comparison). Median concentrations of CRP were 91, 213, and 260 mg/l, respectively (P<0.001 for Group 1 vs Group 2 and Group 3; P=NS for Group 2 vs Group 3). Median concentrations of endotoxin were 0.21, 0.30, and 0.93 U/l, respectively (P=NS for each comparison). Median concentrations of PCT, in contrast to serum CRP and endotoxin, correlated with the severity of sepsis (8.2 ng/ml in 'sepsis' and 22.3 ng/ml in 'severe sepsis', P=0.028) and provided useful prognostic information during septic episodes.

Training in data definitions improves quality of intensive care data

BACKGROUND

Our aim was to assess the contribution of training in data definitions and data extraction guidelines to improving quality of data for use in intensive care scoring systems such as the Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II in the Dutch National Intensive Care Evaluation (NICE) registry.

METHODS

Before and after attending a central training programme, a training group of 31 intensive care physicians from Dutch hospitals who were newly participating in the NICE registry extracted data from three sample patient records. The 5-hour training programme provided participants with guidelines for data extraction and strict data definitions. A control group of 10 intensive care physicians, who were trained according the to train-the-trainer principle at least 6 months before the study, extracted the data twice, without specific training in between.

RESULTS

In the training group the mean percentage of accurate data increased significantly after training for all NICE variables (+7%, 95% confidence interval 5%-10%), for APACHE II variables (+6%, 95% confidence interval 4%-9%) and for SAPS II variables (+4%, 95% confidence interval 1%-6%). The percentage data error due to nonadherence to data definitions decreased by 3.5% after training. Deviations from 'gold standard' SAPS II scores and predicted mortalities decreased significantly after training. Data accuracy in the control group did not change between the two data extractions and was equal to post-training data accuracy in the training group.

CONCLUSION

Training in data definitions and data extraction guidelines is an effective way to improve quality of intensive care scoring data.

Defining and improving data quality in medical registries: a literature review, case study, and generic framework

Over the past years the number of medical registries has increased sharply. Their value strongly depends on the quality of the data contained in the registry. To optimize data quality, special procedures have to be followed. A literature review and a case study of data quality formed the basis for the development of a framework of procedures for data quality assurance in medical registries. Procedures in the framework have been divided into procedures for the co-ordinating center of the registry (central) and procedures for the centers where the data are collected (local). These central and local procedures are further subdivided into (a) the prevention of insufficient data quality, (b) the detection of imperfect data and their causes, and (c) actions to be taken / corrections. The framework can be used to set up a new registry or to identify procedures in existing registries that need adjustment to improve data quality.

Assessment of performance of four mortality prediction systems in a Saudi Arabian intensive care unit

INTRODUCTION

The purpose of this study is to assess the performance of Acute Physiology and Chronic Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, Mortality Probability Model MPM II0 and MPM II24 systems in a major tertiary care hospital in Riyadh, Saudi Arabia.

METHODS

The following data were collected prospectively on all consecutive patients admitted to the Intensive Care Unit between 1 March 1999 and 31 December 2000: demographics, APACHE II and SAPS II scores, MPM variables, ICU and hospital outcome. Predicted mortality was calculated using original regression formulas. Standardized mortality ratio (SMR) was computed with 95% confidence intervals (CI). Calibration was assessed by calculating Lemeshow-Hosmer goodness-of-fit C statistics. Discrimination was evaluated by calculating the Area Under the Receiver Operating Characteristic Curves (ROC AUC).

RESULTS

Predicted mortality by all systems was not significantly different from actual mortality [SMR for MPM II0: 1.00 (0.91-1.10), APACHE II: 1.00 (0.8-1.11), SAPS II: 1.09 (0.97-1.21), MPM II24 0.92 (0.82-1.03)]. Calibration was best for MPM II24 (C-statistic: 14.71, P = 0.06). Discrimination was best for MPM II0 (ROC AUC:0.85) followed by MPM II24 (0.84), APACHE II (0.83) then SAPS II (0.79).

CONCLUSIONS

In our ICU population: 1) Overall mortality prediction, estimated by standardized mortality ratio, was accurate, especially for MPM II0 and APACHE II. 2) MPM II24 has the best calibration. 3) SAPS II has the lowest calibration and discrimination. The local performance of MPM II24 in addition to its ease-to-use makes it an attractive model for mortality prediction in Saudi Arabia.

Accuracy and reliability of APACHE II scoring in two intensive care units Problems and pitfalls in the use of APACHE II and suggestions for improvement

Acute Physiology and Chronic Health Evaluation (APACHE) II scoring is widely used as an index of illness severity, for outcome prediction, in research protocols and to assess intensive care unit performance and quality of care. Despite its widespread use, little is known about the reliability and validity of APACHE II scores generated in everyday clinical practice. We retrospectively re-assessed APACHE II scores from the charts of 186 randomly selected patients admitted to our medical and surgical intensive care units. These 'new' scores were compared with the original scores calculated by the attending physician. We found that most scores calculated retrospectively were lower than the original scores; 51% of our patients would have received a lower score, 26% a higher score and only 23% would have remained unchanged. Overall, the original scores changed by an average of 6.4 points. We identified various sources of error and concluded that wide variability exists in APACHE II scoring in everyday clinical practice, with the score being generally overestimated. Accurate use of the APACHE II scoring system requires adherence to strict guidelines and regular training of medical staff using the system.

Hospital mortality in relation to staff workload: a 4-year study in an adult intensive-care unit

BACKGROUND

Few studies have examined mortality rates in relation to the workload of hospital staff. We investigated this issue in one adult intensive-care unit (ICU) in the UK.

METHODS

We measured ICU workload per shift during each patient's stay for all admissions between 1992 and 1995 that met criteria for adjustment of mortality risk by the APACHE II equation (n=1050). APACHE II data were validated by one observer. Measures of workload in each patient's stay included occupancy, total ICU nursing requirement as defined by the UK Intensive Care Society, and the ratio of occupied to appropriately staffed beds. Over the period, staffing was appropriate for between 4.1 and 5.3 occupied beds (1.3 nurses per patient).

FINDINGS

There were 337 deaths, 49 more (95% CI 34-65) than predicted by the APACHE II equation. Median occupancy was 5.8 beds, and median nursing requirement was 1.6 per patient. On multiple logistic regression analysis, adjusted mortality was more than two times higher (odds ratio 3.1 [1.9-5.0]) in patients exposed to high than in those exposed to low ICU workload, defined by average nursing requirement per occupied bed and peak occupancy; the unadjusted odds ratio for this comparison was 4.0 (2.6-6.2). After exclusion of measures of nursing requirement, adjusted mortality increased with the ratio of occupied to appropriately staffed beds during each patient's stay. All logistic regression models fitted the data satisfactorily.

INTERPRETATION

Variations in mortality may be partly explained by excess ICU workload. This methodology may have implications for planning and clinical governance.

Severity scoring systems and the prediction of outcome from intensive care

Severity scoring systems are tools that provide a predicted mortality for a group of intensive care unit patients on the basis of derangement of their physiology and some past medical history. This predicted mortality can then be compared with the actual mortality to give some indicator of the effectiveness of the package of care delivered by the intensive care unit, corrected for differences in case-mix. Thus, their primary use is in audit, and they are designed for use on large populations of patients and not on individuals. In spite of a large number of publications on the development, refinement and testing of scoring systems, papers describing their use in comparative audit are very rare. This may be partly due to limitations in their ability to predict mortality outside the population on which they were developed, and to the change in calibration of the system with time and advances in medical science. This review briefly addresses the limitations of severity scoring systems in light of recent publications.