Interrater Reliability of APACHE II Scores for Medical-Surgical Intensive Care Patients: A Prospective Blinded Study

Delirium incidence, predictors and outcomes in the intensive care unit: A prospective cohort study

AIM

The aim of this study was to investigate the incidence, predictors, and outcomes of delirium in intensive care units.

BACKGROUND

Delirium is a common complication in intensive care units. In developing countries, it can be misdiagnosed or unrecognised.

DESIGN

Prospective cohort study reported according to the strengthening the reporting of observational studies in epidemiology criteria.

METHODS

We included patients who were conscious, >18 years old, and admitted to the intensive care units for at least 8 h between December 2019 and February 2020. Patients with a Richmond score of -4 or -5, mental disability, receptive aphasia and/or visual or auditory impairment were excluded from the study. Delirium was assessed using the Confusion Assessment Method for the ICU (CAM-ICU), whereas the functional outcome was assessed by the Katz Activity of Daily Living Index.

RESULTS

This study included 111 patients with a delirium incidence of 31.5%. The severity of illness was the only significant predictor of delirium. Patients with delirium had longer intensive care unit and in-hospital stays in contrast to those without delirium. Delirium was associated with in-hospital and 4-month mortality but not the activities of daily living.

CONCLUSIONS

Delirium is associated with increased length of stay and mortality. Further investigation to determine whether delirium management can improve outcomes is warranted.

Usefulness of severity scales for cardiogenic shock in-hospital mortality. Proposal for a new prognostic model

Cardiogenic shock (CS) is a condition comprising multiple etiologies, which associates high mortality rates. Some scoring systems have been shown to be good predictors of hospital mortality in patients admitted to Critical Care Units (CCU). The main objective of this study is to analyze their usefulness and validity in a cohort of CS patients.

METHODS

Observational unicentric study of a cohort of CS patients. SOFA, SAPS II and APACHE II scores were calculated in the first 24 h of CCU admission.

RESULTS

130 patients with CS were included. SOFA, SAPS II and APACHE II scores revealed good discrimination for hospital mortality: (AUC) ROC values (AUC: 0.711, 0.752 and 0.742 respectively; P = .6). Calibration, estimated by the Hosmer-Lemeshow test, was adequate in all cases. Acute coronary syndrome, lactate serum values, SAPS II score and vasoactive inotropic score (VIS) were found to be independent predictors for mortality, upon ICU admission. With these variables, a specific prognostic indicator was developed (SAPS-2-LIVE), which improved predictive capability for mortality in our series (AUC) ROC, 0.825 (95% CI 0.752-0.89).

CONCLUSION

In this contemporary CS cohort, the aforementioned scores have been shown to have good predictive ability for hospital mortality. These findings could contribute to a more accurate risk stratification in CS.

Determinants of pain assessment documentation in intensive care units

PURPOSE

The underassessment of pain is a major barrier to effective pain management, and the lack of pain assessment documentation has been associated with negative patient outcomes. This study aimed to 1) describe the contextual factors related to pain assessment and management in five Québec intensive care units (ICUs); 2) describe their pain assessment documentation practices; and 3) identify sociodemographic and clinical determinants related to pain assessment documentation.

METHODS

A descriptive-correlational retrospective design was used. Sociodemographic data (i.e., age, sex), clinical data (i.e., diagnosis, mechanical ventilation, level of consciousness, severity of illness, opioids, sedatives), and pain assessments were extracted from 345 medical charts of ICU admissions from five teaching hospitals between 2017 and 2019. Descriptive statistics and multiple linear regression were performed.

RESULTS

All sites reported using the 0-10 numeric rating scale, but the implementation of a behavioural pain scale was variable across sites. A median of three documented pain assessments were performed per 24 hr, which is below the minimal recommendation of eight to 12 pain assessments per 24 hr. Overall, pain assessment was present in 70% of charts, but only 20% of opioid doses were followed by documented pain reassessment within one hour post-administration. Higher level of consciousness (β = 0.37), using only breakthrough doses (β = 0.24), and lower opioid doses (β = -0.21) were significant determinants of pain assessment documentation (adjusted R2 = 0.25).

CONCLUSION

Pain assessment documentation is suboptimal in ICUs, especially for patients unable to self-report or those receiving higher opioid doses. Study findings highlight the need to implement tools to optimize pain assessment and documentation.

A simplified critical illness severity scoring system (CISSS): Development and internal validation

PURPOSE

To create a simplified critical illness severity scoring system with high prediction accuracy for 30-day mortality using only commonly available variables.

MATERIALS AND METHODS

This is a retrospective cohort study of ICU admissions 2010-2015 in 306 ICUs in 117 Veterans Affairs (VA) hospitals. We randomly divided our cohort into a training dataset (75%) and a validation dataset (25%). We created a critical illness severity scoring system (CISSS) using age, comorbidities, heart rate, mean arterial blood pressure, temperature, respiratory rate, hematocrit, white blood cell count, creatinine, sodium, glucose, albumin, bilirubin, bicarbonate, use of invasive mechanical ventilation, and whether the admission was surgical or not. We validated the performance of CISSS to predict 30-day mortality internally.

RESULTS

After excluding 31,743 re-admissions, we divided our sample (n = 534,001) into a training (n = 400,613) and a validation dataset (n = 133,388). In the training dataset, the area under the curve (AUC) of CISSS was 0.847(95%CI = 0.845-0.850). In the validation dataset, the AUC was 0.848 (95%CI = 0.844-0.852), the standardized mortality ratio (SMR) was 1.00 (95%CI = 0.98-1.02), and Brier's score for 30-day mortality was 0.058 (95%CI = 0.057-0.059). CISSS calibration was acceptable.

CONCLUSIONS

CISSS has very good performance and requires only commonly used variables that can be easily extracted by electronic health records.

Comparison of the Automated Pediatric Logistic Organ Dysfunction-2 Versus Manual Pediatric Logistic Organ Dysfunction-2 Score for Critically Ill Children

OBJECTIVES

The Pediatric Logistic Organ Dysfunction-2 is a validated score that quantifies organ dysfunction severity and requires complex data collection that is time-consuming and subject to errors. We hypothesized that a computer algorithm that automatically collects and calculates the Pediatric Logistic Organ Dysfunction-2 (aPELOD-2) score would be valid, fast and at least as accurate as a manual approach (mPELOD-2).

DESIGN

Retrospective cohort study.

SETTING

Single center tertiary medical and surgical pediatric critical care unit (Sainte-Justine Hospital, Montreal, Canada).

PATIENTS

Critically ill children participating in four clinical studies between January 2013 and August 2018, a period during which mPELOD-2 data were manually collected.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The aPELOD-2 was calculated for all consecutive admissions between 2013 and 2018 (n = 5,279) and had a good survival discrimination with an area under the receiver operating characteristic curve of 0.84 (95% CI, 0.81-0.88). We also collected data from four single-center studies in which mPELOD-2 was calculated (n = 796, 57% medical, 43% surgical) and compared these measurements to those of the aPELOD-2. For those patients, median age was 15 months (interquartile range, 3-73 mo), median ICU stay was 5 days (interquartile range, 3-9 d), mortality was 3.9% (n = 28). The intraclass correlation coefficient between mPELOD-2 and aPELOD-2 was 0.75 (95% CI, 0.73-0.77). The Bland-Altman showed a bias of 1.9 (95% CI, 1.7-2) and limits of agreement of -3.1 (95% CI, -3.4 to -2.8) to 6.8 (95% CI, 6.5-7.2). The highest agreement (Cohen's Kappa) of the Pediatric Logistic Organ Dysfunction-2 components was noted for lactate level (0.88), invasive ventilation (0.86), and creatinine level (0.82) and the lowest for the Glasgow Coma Scale (0.52). The proportion of patients with multiple organ dysfunction syndrome was higher for aPELOD-2 (78%) than mPELOD-2 (72%; p = 0.002). The aPELOD-2 had a better survival discrimination (area under the receiver operating characteristic curve, 0.81; 95% CI, 0.72-0.90) over mPELOD-2 (area under the receiver operating characteristic curve, 0.70; 95% CI, 0.59-0.82; p = 0.01).

CONCLUSIONS

We successfully created a freely available automatic algorithm to calculate the Pediatric Logistic Organ Dysfunction-2 score that is less labor intensive and has better survival discrimination than the manual calculation. Use of an automated system could greatly facilitate integration of the Pediatric Logistic Organ Dysfunction-2 score at the bedside and within clinical decision support systems.

Inter-Rater Reliability and Impact of Disagreements on Acute Physiology and Chronic Health Evaluation IV Mortality Predictions

Acute Physiology and Chronic Health Evaluation is a well-validated method to risk-adjust ICU patient outcomes. However, predictions may be affected by inter-rater reliability for manually entered elements. We evaluated inter-rater reliability for Acute Physiology and Chronic Health Evaluation IV manually entered elements among clinician abstractors and assessed the impacts of disagreements on mortality predictions.

An automated computerized critical illness severity scoring system derived from APACHE III: modified APACHE

PURPOSE

To evaluate the performance of an automated computerized ICU severity scoring derived from the APACHE III.

MATERIALS AND METHODS

Within a retrospective cohort of patients admitted to Veterans Health Administration ICUs between 2009 and 2015, we created an automated illness severity score(modified APACHE or mAPACHE), that we extracted from the electronic health records, using the same scoring as the APACHE III excluding the Glasgow Coma Scale, urine output, arterial blood gas components of APACHE III. We assessed the mAPACHE discrimination by using the area under the curve(AUC), and calibration by using the Hosmer-Lemeshow test and calculating the difference between observed and expected mortality across equal-sized risk deciles for death.

RESULTS

The ICU and 30-day mortality was 5.07% of 7.82%, respectively (n = 490,955 patients). The AUC of mAPACHE for ICU and 30-day mortality was 0.771 and 0.786, respectively. The Hosmer-Lemeshow test was significant for both ICU and 30-day mortality (p < .001). The absolute difference between observed and expected mortality did not exceed ±1.53% across equal-sized deciles of risk for death. The AUC for ICU mortality was >0.7 in all admission diagnosis categories except in endocrine, respiratory, and sepsis. The AUC for 30-day mortality was >0.7 in every category.

CONCLUSION

mAPACHE has adequate performance to predict mortality.

Factors Influencing the Reliability of the Glasgow Coma Scale: A Systematic Review

BACKGROUND

The Glasgow Coma Scale (GCS) characterizes patients with diminished consciousness. In a recent systematic review, we found overall adequate reliability across different clinical settings, but reliability estimates varied considerably between studies, and methodological quality of studies was overall poor. Identifying and understanding factors that can affect its reliability is important, in order to promote high standards for clinical use of the GCS.

OBJECTIVE

The aim of this systematic review was to identify factors that influence reliability and to provide an evidence base for promoting consistent and reliable application of the GCS.

METHODS

A comprehensive literature search was undertaken in MEDLINE, EMBASE, and CINAHL from 1974 to July 2016. Studies assessing the reliability of the GCS in adults or describing any factor that influences reliability were included. Two reviewers independently screened citations, selected full texts, and undertook data extraction and critical appraisal. Methodological quality of studies was evaluated with the consensus-based standards for the selection of health measurement instruments checklist. Data were synthesized narratively and presented in tables.

RESULTS

Forty-one studies were included for analysis. Factors identified that may influence reliability are education and training, the level of consciousness, and type of stimuli used. Conflicting results were found for experience of the observer, the pathology causing the reduced consciousness, and intubation/sedation. No clear influence was found for the professional background of observers.

CONCLUSION

Reliability of the GCS is influenced by multiple factors and as such is context dependent. This review points to the potential for improvement from training and education and standardization of assessment methods, for which recommendations are presented.

Feasibility and Accuracy of a Nonmedical Research Person in Assimilation and Calculation of Acute Physiologic Assessment and Chronic Health Evaluation Scores in an Indian Intensive Care Unit

Background

The Physiologic Assessment and Chronic Health Evaluation (APACHE) score assimilation and calculation, as well as other demographic data collection, is inherent to research and nonresearch related needs of intensive care. There may be a role for well-trained nonmedical personnel to collect this vital material to enhance research and the standard of care in the Intensive Care Units (ICUs) in countries that are poorly funded and resourced in terms of medical personnel.

Aims

The aim of this study is to verify the interrater reliability of a trained nonmedical personnel and ICU trainee in the collection and calculation APACHE scores.

Materials and Methods

In a prospective study, two raters who were blinded, one a trained nonmedical ward clerk and another an ICU trainee, assimilated data and calculated APACHE scores for 60 consecutive patients admitted to two tertiary mixed ICUs (with a total of 19 beds). Primary outcomes were to assess interrater and interclass correlation as well as the agreement of scores between the two raters.

Results

There was an excellent correlation of APACHE scores (Kappa coefficient of 0.92) and Bland-Altman plot depicted overall good agreement with low bias between raters.

Conclusions

A well-trained and supervised nonmedical research person can assimilate and calculate APACHE II scores with good agreement with an ICU trainee. This may help in deriving data from medically understaffed ICUs in India, thus promoting much-needed research from such ICUs.

Temporal Patterns and Influential Factors of Blood Glucose Levels During the First 10-Day Critical Period After Brain Injury

This study was conducted to document temporal patterns of blood glucose level changes during the first 10-day critical period and to identify factors that influence stress-induced hyperglycemia development in brain injury patients. The medical records of 190 brain injury patients were retrospectively reviewed. Blood glucose levels in the poor recovery group were significantly higher than in the good recovery group, particularly during the first 72 hr (158-172 mg/dl). The poor recovery group showed persistent, fluctuating hyperglycemia, whereas the good recovery group exhibited hyperglycemic peaks during the first 3 days that subsequently reduced linearly to normal. Gender, preexisting hypertension, disease severity at admission, total calorie intake, and steroid use were found to influence stress-induced hyperglycemia development significantly. In conclusion, close monitoring and adjustment are required to maintain safe blood glucose levels and the development of protocols for safe glycemic management is essential to improve critical care in brain injury patients.

Validating a Patient-Reported Comorbidity Measure with Respect to Quality of Life in End-Stage Renal Disease

Purpose

Medical record-derived comorbidity measures such as the Charlson Comorbidity Index (CCI) do not predict functional limitations or quality of life (QoL) in the chronically ill. Although these shortcomings are known since the 1980s, they have been largely ignored by the international literature. Recently, QoL has received growing interest as an end-point of interventional trials in Nephrology. The aim of this study is to compare a patient-reported comorbidity measure and the CCI with respect to its validity regarding QoL.

Methods

The German Self-Administered Comorbidity Questionnaire (SCQ-G) was completed by 780 adult end-stage renal disease-patients recruited from 55 dialysis units throughout Germany. Acceptance was evaluated via response rates. Content validity was examined by comparing the typical comorbidity pattern in dialysis patients and the pattern retrieved from our data. Convergent validity was assessed via kappa statistics. Data was compared to the CCI. Linear associations with QoL were examined (criterion validity).

Results

The SCQ-G was very well accepted by dialysis patients of all ages (response rate: 99%). Content validity can be interpreted as high (corresponding comorbidity items: 73.7%). Convergent validity was rather weak (.27≤ρ≤.29) but increased when comparing only concordant items (.39≤ρ≤.43). With respect to criterion validity, the SCQ-G performed better than the CCI regarding the correlation with QoL (e.g., SF-12-physical: SCQ-G total score: ρ = -.49 vs. CCI: ρ = -.36).

Conclusions

The patient-reported measure proved to be more valid than the external assessment when aiming at insights on QoL. Due to the inclusion of subjective limitations, the SCQ-G is more substantial with respect to patient-centered outcomes and might be used as additional measure in clinical trials.

The reliability of the Glasgow Coma Scale: a systematic review

INTRODUCTION

The Glasgow Coma Scale (GCS) provides a structured method for assessment of the level of consciousness. Its derived sum score is applied in research and adopted in intensive care unit scoring systems. Controversy exists on the reliability of the GCS. The aim of this systematic review was to summarize evidence on the reliability of the GCS.

METHODS

A literature search was undertaken in MEDLINE, EMBASE and CINAHL. Observational studies that assessed the reliability of the GCS, expressed by a statistical measure, were included. Methodological quality was evaluated with the consensus-based standards for the selection of health measurement instruments checklist and its influence on results considered. Reliability estimates were synthesized narratively.

RESULTS

We identified 52 relevant studies that showed significant heterogeneity in the type of reliability estimates used, patients studied, setting and characteristics of observers. Methodological quality was good (n = 7), fair (n = 18) or poor (n = 27). In good quality studies, kappa values were ≥0.6 in 85%, and all intraclass correlation coefficients indicated excellent reliability. Poor quality studies showed lower reliability estimates. Reliability for the GCS components was higher than for the sum score. Factors that may influence reliability include education and training, the level of consciousness and type of stimuli used.

CONCLUSIONS

Only 13% of studies were of good quality and inconsistency in reported reliability estimates was found. Although the reliability was adequate in good quality studies, further improvement is desirable. From a methodological perspective, the quality of reliability studies needs to be improved. From a clinical perspective, a renewed focus on training/education and standardization of assessment is required.

The ecology of antibiotic use in the ICU: homogeneous prescribing of cefepime but not tazocin selects for antibiotic resistant infection

Background

Antibiotic homogeneity is thought to drive resistance but in vivo data are lacking. In this study, we determined the impact of antibiotic homogeneity per se, and of cefepime versus antipseudomonal penicillin/β-lactamase inhibitor combinations (APP-β), on the likelihood of infection or colonisation with antibiotic resistant bacteria and/or two commonly resistant nosocomial pathogens (methicillin-resistant Staphylococcus aureus and Pseudomonas aeruginosa). A secondary question was whether antibiotic cycling was associated with adverse outcomes including mortality, length of stay, and antibiotic resistance.

Methods

We evaluated clinical and microbiological outcomes in two similar metropolitan ICUs, which both alternated cefepime with APP-β in four-month cycles. All microbiological isolates and commensal samples were analysed for the presence of antibiotic-resistant bacteria including MRSA and P. aeruginosa.

Results

Length of stay, mortality and overall antibiotic resistance were unchanged after sixteen months. However, increased colonisation and infection by antibiotic-resistant bacteria were observed in cefepime cycles, returning to baseline in APP-β cycles. Cefepime was the strongest risk factor for acquisition of antibiotic-resistant infection.

Conclusions

Ecological effects of different β-lactam antibiotics may be more important than specific activity against the causative agents or the effect of antibiotic homogeneity in selection for antibiotic resistance. This has important implications for antibiotic policy.

Critical Care Nurses Inadequately Assess SAPS II Scores of Very Ill Patients in Real Life

Background. Reliable ICU severity scores have been achieved by various healthcare workers but nothing is known regarding the accuracy in real life of severity scores registered by untrained nurses. Methods. In this retrospective multicentre audit, three reviewers independently reassessed 120 SAPS II scores. Correlation and agreement of the sum-scores/variables among reviewers and between nurses and the reviewers' gold standard were assessed globally and for tertiles. Bland and Altman (gold standard—nurses) of sum scores and regression of the difference were determined. A logistic regression model identifying risk factors for erroneous assessments was calculated. Results. Correlation for sum scores among reviewers was almost perfect (mean ICC = 0.985). The mean (±SD) nurse-registered SAPS II sum score was 40.3 ± 20.2 versus 44.2 ± 24.9 of the gold standard (P < 0.002 for difference) with a lower ICC (0.81). Bland and Altman assay was +3.8 ± 27.0 with a significant regression between the difference and the gold standard, indicating overall an overestimation (underestimation) of lower (higher; >32 points) scores. The lowest agreement was found in high SAPS II tertiles for haemodynamics (k = 0.45–0.51). Conclusions. In real life, nurse-registered SAPS II scores of very ill patients are inaccurate. Accuracy of scores was not associated with nurses' characteristics.

The association between bathing and weaning trial duration

OBJECTIVE

To describe patterns of bath care for patients who are weaning from prolonged mechanical ventilation (PMV) and to explore the association between bathing and weaning trial duration.

METHODS

Descriptive correlational study. Clinical records from 439 weaning trial days for 30 patients who required PMV were abstracted for bathing occurrences during weaning trials, within 1 hour before a trial, and nocturnally.

RESULTS

Most baths occurred during weaning trials (30.8%) or at night (35.3%), and less frequently (16%) within 1 hour before a trial. No significant effects were found on trial duration for nocturnal bathing or bathing within 1 hour before a trial. By using random coefficient modeling, weaning duration was shown to be longer when bathing occurred during a weaning trial (P < .05), even when controlling for age, severity of illness, and days on bedrest.

CONCLUSION

Bathing occurred during approximately one third of PMV weaning trials. Baths during PMV weaning trials were associated with longer weaning trial duration.

Reliability and accuracy of practitioner-calculated Acute Physiology and Chronic Health Evaluation II scores for determining the appropriateness of drotrecogin alfa (activated)

PURPOSE

The reliability and accuracy of practitioner-calculated Acute Physiology and Chronic Health Evaluation (APACHE) II scores for determining the appropriateness of drotrecogin alfa (activated) in critically ill patients were evaluated.

METHODS

Three adjudicated clinical cases of sepsis were developed using composites of real patient scenarios. The patients' APACHE II scores were independently assessed by randomly selected critical care practitioners (physicians and nonphysicians). Each case contained at least one reason to consider withholding drotrecogin alfa (activated), but none had a definitive contraindication to drotrecogin alfa (activated). Intraobserver and interobserver variabilities were assessed using kappa correlation. Accuracy was assessed by comparing median scores to the adjudicated scores and evaluating correctly classified APACHE II scores.

RESULTS

A total of 21 (42%) physicians and 14 (56%) nonphysicians completed all assessments. Intraobserver and interobserver variabilities were 0.16 and 0.49 for the total APACHE II score, respectively. Median calculated APACHE II scores significantly differed for case 1 (p = 0.003) and case 3 (p < 0.0001). The percentage of error in calculating the total APACHE II score approached 85%. The main reasons for administering drotrecogin alfa (activated) were an APACHE II score of >or=25 and multiple organ failures. The main reason for therapy was a high bleeding risk or an APACHE II score of <25.

CONCLUSION

Weak intraobserver agreement, modest interobserver reliability, a high error rate, and low accuracy limited the clinical application of the APACHE II score by untrained practitioners, indicating that the APACHE II score should not be the only determinant for the use of drotrecogin alfa (activated).

The interrater reliability of SAPS II and SAPS 3

PURPOSE

In this study we evaluated and compared the interrater reliability of SAPS II and SAPS 3 in order to measure the consistency of performance among different raters.

METHOD

Ten junior doctors working at two general ICUs were trained in the use of SAPS II and SAPS 3 using a 2.5-h training program. After training they scored 24 cases in both systems. Scores were analyzed using intraclass correlation coefficient (ICC) statistics. In order to identify variables with low reliability, subscores were analyzed using the ICC, and single-variables were compared to a template score using weighted kappa statistics.

RESULTS

The ICC (95% CI) of the scores was 0.84 (0.74, 0.91) in SAPS II and 0.80 (0.68, 0.89) in SAPS 3, which is considered adequate for both systems. Mean mortality predictions among the raters had a range of 0.12 in SAPS II and 0.19 in SAPS 3. Administrative data including age had high reliability, whereas variables based on diagnostic information had only moderate reliability. Laboratory data had consistently higher reliability than variables based on the interpretation of charts.

CONCLUSION

Both SAPS II and SAPS 3 have adequate interrater reliability, but the standardized mortality ratios are still likely to be influenced by the rater's scoring practice.

Validity of the FOUR score coma scale in the medical intensive care unit

OBJECTIVE

To evaluate the validity of the FOUR (Full Outline of UnResponsiveness) score (ranging from 0 to 16), a new coma scale consisting of 4 components (eye response, motor response, brainstem reflexes, and respiration pattern), when used by the staff members of a medical intensive care unit (ICU).

PATIENTS AND METHODS

This interobserver agreement study prospectively evaluated the use of the FOUR score to describe the condition of 100 critically ill patients from May 1, 2007, to April 30, 2008. We compared the FOUR score to the Glasgow Coma Scale (GCS) score. For each patient, the FOUR score and the GCS score were determined by a randomly selected staff pair (nurse/fellow, nurse/consultant, fellow/fellow, or fellow/consultant). Pair wise weighted kappa values were calculated for both scores for each observer pair.

RESULTS

The interrater agreement with the FOUR score was excellent (weighted kappa: eye response, 0.96; motor response, 0.97; brainstem reflex, 0.98; respiration pattern, 1.00) and similar to that obtained with the GCS (weighted kappa: eye response, 0.96; motor response, 0.97; verbal response, 0.98). In terms of the predictive power for poor neurologic outcome (Modified Rankin Scale score, 3-6), the area under the receiver operating characteristic curve was 0.75 for the FOUR score and 0.76 for the GCS score. The mortality rate for patients with the lowest FOUR score of 0 (89%) was higher than that for patients with the lowest GCS score of 3 (71%).

CONCLUSION

The interrater agreement of FOUR score results was excellent among medical intensivists. In contrast to the GCS, all components of the FOUR score can be rated even when patients have undergone intubation. The FOUR score is a good predictor of the prognosis of critically ill patients and has important advantages over the GCS in the ICU setting.

Reliability of intensive care unit admitting and comorbid diagnoses, race, elements of Acute Physiology and Chronic Health Evaluation II score, and predicted probability of mortality in an electronic intensive care unit database

BACKGROUND

Although reliability of severity of illness and predicted probability of hospital mortality have been assessed, interrater reliability of the abstraction of primary and other intensive care unit (ICU) admitting diagnoses and underlying comorbidities has not been studied.

METHODS

Patient data from one ICU were originally abstracted and entered into an electronic database by an ICU nurse. A research assistant reabstracted patient demographics, ICU admitting diagnoses and underlying comorbidities, and elements of Acute Physiology and Chronic Health Evaluation II (APACHE II) score from 100 random patients of 474 admitted during 2005 using an identical electronic database. Chamberlain's percent positive agreement was used to compare diagnoses and comorbidities between the 2 data abstractors. A kappa statistic was calculated for demographic variables, Glasgow Coma Score, APACHE II chronic health points, and HIV status. Intraclass correlation was calculated for acute physiology points and predicted probability of hospital mortality.

RESULTS

Percent positive agreement for ICU primary and other admitting diagnoses ranged from 0% (primary brain injury) to 71% (sepsis), and for underlying comorbidities, from 40% (coronary artery bypass graft) to 100% (HIV). Agreement as measured by kappa statistic was strong for race (0.81) and age points (0.95), moderate for chronic health points (0.50) and HIV (0.66), and poor for Glasgow Coma Score (0.36). Intraclass correlation showed a moderate-high agreement for acute physiology points (0.88) and predicted probability of hospital mortality (0.71).

CONCLUSION

Reliability for ICU diagnoses and elements of the APACHE II score is related to the objectivity of primary data in the medical charts.