Validity of using an electronic medical record for assessing quality of care in an outpatient setting

Retooling of Paper-based Outcome Measures to Electronic Format: Comparison of the NY State Public Risk Model and EHR-derived Risk Models for CABG Mortality

BACKGROUND

Risk adjustment is critical in the comparison of quality of care and health care outcomes for providers. Electronic health records (EHRs) have the potential to eliminate the need for costly and time-consuming manual data abstraction of patient outcomes and risk factors necessary for risk adjustment.

METHODS

Leading EHR vendors and hospital focus groups were asked to review risk factors in the New York State (NYS) coronary artery bypass graft (CABG) surgery statistical models for mortality and readmission and assess feasibility of EHR data capture. Risk models based only on registry data elements that can be captured by EHRs (one for easily obtained data and one for data obtained with more difficulty) were developed and compared with the NYS models for different years.

RESULTS

Only 6 data elements could be extracted from the EHR, and outlier hospitals differed substantially for readmission but not for mortality. At the patient level, measures of fit and predictive ability indicated that the EHR models are inferior to the NYS CABG surgery risk model [eg, c-statistics of 0.76 vs. 0.71 (P<0.001) and 0.76 vs. 0.74 (P=0.009) for mortality in 2010], although the correlation of the predicted probabilities between the NYS and EHR models was high, ranging from 0.96 to 0.98.

CONCLUSIONS

A simplified risk model using EHR data elements could not capture most of the risk factors in the NYS CABG surgery risk models, many outlier hospitals were different for readmissions, and patient-level measures of fit were inferior.

Development and Validation of Electronic Quality Measures to Assess Care for Patients With Transient Ischemic Attack and Minor Ischemic Stroke

BACKGROUND

Despite interest in using electronic health record (EHR) data to assess quality of care, the accuracy of such data is largely unknown. We sought to develop and validate transient ischemic attack and minor ischemic stroke electronic quality measures (eQMs) using EHR data.

METHODS AND RESULTS

A random sample of patients with transient ischemic attack or minor ischemic stroke, cared for in Veterans Health Administration facilities (fiscal year 2011), was identified. We constructed 31 eQMs based on existing quality measures. Chart review was the criterion standard for validating the eQMs. To evaluate eQMs in terms of eligibility, we calculated the proportion of patients who were genuinely not eligible to receive a process (based on chart review) and who were correctly identified as not eligible by the EHR data (specificity). To assess eQMs about classification of whether patients received a process, we calculated the proportion of patients who actually received the process (based on chart review) and who were classified correctly by the EHR data as passing (sensitivity). Seven hundred sixty-three patients were included. About eligibility, specificity varied from 25% (brain imaging; carotid imaging) to 99% (anticoagulation quality). About pass rates, sensitivity varied from 30% (antihypertensive class) to 100% (coronary risk assessment; international normalized ratio measured). The 16 eQMs with ≥70% specificity in eligibility and ≥70% sensitivity in pass rates included coronary risk assessment, international normalized ratio measured, HbA1c measurement, speech language pathology consultation, anticoagulation for atrial fibrillation, discharge on statin, lipid management, neurology consultation, Holter, deep vein thrombosis prophylaxis, oral hypoglycemic intensification, cholesterol medication intensification, antihypertensive intensification, antihypertensive class, carotid stenosis intervention, and substance abuse referral for alcohol.

CONCLUSIONS

It is feasible to construct valid eQMs for processes of transient ischemic attack and minor ischemic stroke care. Healthcare systems with EHRs should consider using electronic data to evaluate care for their patients with transient ischemic attack and to complement and expand quality measurement programs currently focused on patients with stroke.

Challenges in the use of the mental health information system in a resource-limited setting: lessons from Ghana

BACKGROUND

One of the most successful modes of record-keeping and data collection is the use of health management information systems, where patient information and management plans are uniformly entered into a database to streamline the information and for ease of further patient management. For mental healthcare, a Mental Health Information System (MHIS) has been found most successful since a properly established and operational MHIS is helpful for developing equitable and appropriate mental health care systems. Until 2010, the system of keeping patient records and information in the Accra Psychiatric Hospital of Ghana was old and outdated. In light of this and other factors, a complete reforming of the mental health information systems in three psychiatric hospitals in Ghana was undertaken in 2010. Four years after its implementation, we explored user experiences with the new system, and report here the challenges that were identified with use of the new MHIS.

METHODS

Individual semi-structured interviews were conducted with nine clinical and administrative staff of the Accra Psychiatric Hospital to examine their experiences with the new MHIS. Participants in the study were in three categories: clinical staff, administrator, and records clerk. Participants' knowledge of the system and its use, as well as the challenges they had experienced in its use were explored using an interpretative phenomenological approach.

RESULTS

The data suggest that optimal use of the current MHIS had faced significant implementation challenges in a number of areas. Central challenges reported by users included increased workload, poor staff involvement and training, and absence of logistic support to keep the system running.

CONCLUSIONS

Setting up a new system does not guarantee its success. As important as it is to have a mental health information system, its usefulness is largely dependent on proper implementation and maintenance. Further, the system can facilitate policy transformation only when the place of mental health in district, regional and national health discourse improves.

Comparison of Risk Factor Control in the Year After Discharge for Ischemic Stroke Versus Acute Myocardial Infarction

Background and Purpose—

The Veterans Health Administration has engaged in quality improvement to improve vascular risk factor control. We sought to examine blood pressure (<140/90 mm Hg), lipid (LDL [low-density lipoprotein] cholesterol <100 mg/dL), and glycemic control (hemoglobin A1c <9%), in the year post-hospitalization for acute ischemic stroke or acute myocardial infarction (AMI).

Methods—

We identified patients who were hospitalized (fiscal year 2011) with ischemic stroke, AMI, congestive heart failure, transient ischemic attack, or pneumonia/chronic obstructive pulmonary disease. The primary analysis compared risk factor control after incident ischemic stroke versus AMI. Facilities were included if they cared for ≥25 ischemic stroke and ≥25 AMI patients. A generalized linear mixed model including patient- and facility-level covariates compared risk factor control across diagnoses.

Results—

Forty thousand two hundred thirty patients were hospitalized (n=75 facilities): 2127 with incident ischemic stroke and 4169 with incident AMI. Fewer stroke patients achieved blood pressure control than AMI patients (64%; 95% confidence interval, 0.62–0.67 versus 77%; 95% confidence interval, 0.75–0.78; P <0.0001). After adjusting for patient and facility covariates, the odds of blood pressure control were still higher for AMI than ischemic stroke patients (odds ratio, 1.39; 95% confidence interval, 1.21–1.51). There were no statistical differences for AMI versus stroke patients in hyperlipidemia ( P =0.534). Among patients with diabetes mellitus, the odds of glycemic control were lower for AMI than ischemic stroke patients (odds ratio, 0.72; 95% confidence interval, 0.54–0.96).

Conclusions—

Given that hypertension control is a cornerstone of stroke prevention, interventions to improve poststroke hypertension management are needed.

Electronic approaches to making sense of the text in the adverse event reporting system

INTRODUCTION

Health care organizations working to eliminate preventable harm and to improve patient safety must have robust programs to collect and to analyze data on adverse events in order to use the information to affect improvement. Such adverse event reporting systems are based on frontline personnel reporting issues that arise in the course of their daily work. Limitations in how existing software systems handle these reports mean that use of this potentially rich information is resource intensive and prone to variable results.

AIM

The aim of this study was to develop an electronic approach to processing the text in medical event reports that would be reliable enough to be used to improve patient safety.

METHODS

At Connecticut Children's Medical Center, staff manually enter reports of adverse events into a web-based software tool. We evaluated the ability of 2 electronic methods-rule-based query and semi-supervised machine learning-to identify specific types of events ("use cases") versus a reference standard. Rule-based query was tested on 5 use cases and machine learning on a subset of 2 using 9164 events reported from February 2012-January 2014.

RESULTS

Machine learning found 93% of the weight-based errors and 92% of the errors in patient-identification. Rule-based query had accuracy of 99% or greater, high precision, and high recall for all use cases.

CONCLUSIONS

Electronic approaches to streamlining the use of adverse event reports are feasible to automate and valuable for categorizing this important data for use in improving patient safety.

Measuring Preventive Care Delivery: Comparing Rates Across Three Data Sources

INTRODUCTION

Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness.

METHODS

This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015.

RESULTS

Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96-1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83-0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67-0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services.

CONCLUSIONS

Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

A comparison of a multistate inpatient EHR database to the HCUP Nationwide Inpatient Sample

BACKGROUND

The growing availability of electronic health records (EHRs) in the US could provide researchers with a more detailed and clinically relevant alternative to using claims-based data.

METHODS

In this study we compared a very large EHR database (Health Facts©) to a well-established population estimate (Nationwide Inpatient Sample). Weighted comparisons were made using t-value and relative difference over diagnoses and procedures for the year 2010.

RESULTS

The two databases have a similar distribution pattern across all data elements, with 24 of 50 data elements being statistically similar between the two data sources. In general, differences that were found are consistent across diagnosis and procedures categories and were specific to the psychiatric-behavioral and obstetrics-gynecology services areas.

CONCLUSIONS

Large EHR databases have the potential to be a useful addition to health services researchers, although they require different analytic techniques compared to administrative databases; more research is needed to understand the differences.

Accessing primary care Big Data: the development of a software algorithm to explore the rich content of consultation records

OBJECTIVE

To develop a natural language processing software inference algorithm to classify the content of primary care consultations using electronic health record Big Data and subsequently test the algorithm's ability to estimate the prevalence and burden of childhood respiratory illness in primary care.

DESIGN

Algorithm development and validation study. To classify consultations, the algorithm is designed to interrogate clinical narrative entered as free text, diagnostic (Read) codes created and medications prescribed on the day of the consultation.

SETTING

Thirty-six consenting primary care practices from a mixed urban and semirural region of New Zealand. Three independent sets of 1200 child consultation records were randomly extracted from a data set of all general practitioner consultations in participating practices between 1 January 2008-31 December 2013 for children under 18 years of age (n=754,242). Each consultation record within these sets was independently classified by two expert clinicians as respiratory or non-respiratory, and subclassified according to respiratory diagnostic categories to create three 'gold standard' sets of classified records. These three gold standard record sets were used to train, test and validate the algorithm.

OUTCOME MEASURES

Sensitivity, specificity, positive predictive value and F-measure were calculated to illustrate the algorithm's ability to replicate judgements of expert clinicians within the 1200 record gold standard validation set.

RESULTS

The algorithm was able to identify respiratory consultations in the 1200 record validation set with a sensitivity of 0.72 (95% CI 0.67 to 0.78) and a specificity of 0.95 (95% CI 0.93 to 0.98). The positive predictive value of algorithm respiratory classification was 0.93 (95% CI 0.89 to 0.97). The positive predictive value of the algorithm classifying consultations as being related to specific respiratory diagnostic categories ranged from 0.68 (95% CI 0.40 to 1.00; other respiratory conditions) to 0.91 (95% CI 0.79 to 1.00; throat infections).

CONCLUSIONS

A software inference algorithm that uses primary care Big Data can accurately classify the content of clinical consultations. This algorithm will enable accurate estimation of the prevalence of childhood respiratory illness in primary care and resultant service utilisation. The methodology can also be applied to other areas of clinical care.

Outcome evaluation in pediatric orthopedics

Outcome evaluation is an objective procedure to determine the impact and success of a therapeutic program, focusing on the patient's well being in daily life. It is important not to confuse the different concepts found in the health status classification and quality of life evaluation. Both are suitable for outcome evaluation and may be used according to the principles of evidence-based medicine. Outcome evaluation of a therapeutic program may be compared to a search in the literature and the level of evidence. The goal is to achieve direct benefits for the patient and society. Strategies to improve daily practice may be developed. Scientific societies can create database for outcome evaluation and develop standardized protocols for longitudinal outcome follow-up, as well as proposing this type of study to health authorities. Based on these results, a global view of the patient can be taken into consideration to influence medical, socioeconomic and health management patient care.

Caveats for the use of operational electronic health record data in comparative effectiveness research

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

Measuring data reliability for preventive services in electronic medical records

Background

Improvements in the quality of health care services are often measured using data present in medical records. Electronic Medical Records (EMRs) contain potentially valuable new sources of health data. However, data quality in EMRs may not be optimal and should be assessed. Data reliability (are the same data elements being measured over time?) is a prerequisite for data validity (are the data accurate?). Our objective was to measure the reliability of data for preventive services in primary care EMRs during the transition to EMR.

Methods

Our data sources were randomly selected eligible patients’ medical records and data obtained from provincial administrative datasets. Eighteen community-based family physicians in Toronto, Ontario that implemented EMRs starting in 2006 participated in this study. We measured the proportion of patients eligible for a service (Pap smear, screening mammogram or influenza vaccination) that received the service. We compared the change in rates of selected preventive services calculated from the medical record audits with the change in administrative datasets.

Results

In the first year of EMR use (2006) services decreased by 8.7% more (95% CI −11.0%– − 6.4%, p < 0.0001) when measured through medical record audits as compared with administrative datasets. Services increased by 2.4% more (95% CI 0%–4.9%, p = 0.05) in the medical record audits during the second year of EMR use (2007).

Conclusion

There were differences between the change measured through medical record audits and administrative datasets. Problems could include difficulties with organizing new data entry processes as well as continued use of both paper and EMRs. Data extracted from EMRs had limited reliability during the initial phase of EMR implementation. Unreliable data interferes with the ability to measure and improve health care quality

Psychosocial correlates of healthy beliefs, choices, and behaviors in overweight and obese school-age children: a primary care healthy choices intervention pilot study

The relationships among weight, mental health, social competence, healthy lifestyle beliefs, choices, and behaviors in overweight and obese school-age children (9 to 12 years) who were participating in a pilot Healthy Choices Intervention Program were examined using a descriptive correlational design. Results indicated that the children's weight negatively correlated with the children's self-concept, social skills, and, most importantly, healthy lifestyle choices. Their healthy lifestyle beliefs demonstrated positive correlations with their intentions to choose a healthy lifestyle. The children's healthy lifestyle beliefs and choices also were associated with increased cooperation and empathy behaviors. A positive self-concept was associated with decreased anxiety and depressive symptomology. These data support the need for health care providers to understand the psychological and social issues that overweight and obese school-age children experience. Cognitive behavior skills building techniques, such as those in the Healthy Choices Intervention Program, may promote healthy lifestyle choices and behaviors.

How good are the data? Feasible approach to validation of metrics of quality derived from an outpatient electronic health record

Although electronic health records (EHRs) promise to be efficient resources for measuring metrics of quality, they are not designed for such population-based analyses. Thus, extracting meaningful clinical data from them is not straightforward. To avoid poorly executed measurements, standardized methods to measure and to validate metrics of quality are needed. This study provides and evaluates a use case for a generally applicable approach to validating quality metrics measured electronically from EHR-based data. The authors iteratively refined and validated 4 outpatient quality metrics and classified errors in measurement. Multiple iterations of validation and measurement resulted in high levels of sensitivity and agreement versus the "gold standard" of manual review. In contrast, substantial differences remained for measurement based on coded billing data. Measuring quality metrics using an EHR-based electronic process requires validation to ensure accuracy; approaches to validation such as those described in this study should be used by organizations measuring quality from EHR-based information.

A systematic review of primary healthcare provider education and training using the Chronic Care Model for childhood obesity

The purpose of this systematic review was to examine 15 studies which evaluated interventions aimed at improving primary care providers' identification, assessment, prevention and/or management of obesity in children and adolescents. Interventions were evaluated in terms of length, components addressing nutrition, physical activity and behavioural counselling consistent with expert recommendations, and inclusion of components of the Chronic Care Model. Overall, training interventions were delivered face to face or in a combination of lecture, assigned readings, preceptorship with experienced providers, and critiqued evaluations of interactions of the provider with the patient and family. Many studies incorporated training of providers as an initial step prior to delivering an obesity intervention for children and adolescents measuring weight loss and behaviour change as outcomes. Each study was evaluated for components of the Chronic Care Model. The interventions most frequently utilized the elements of self-management support (69%), decision support (100%), delivery system support (77%) and clinical information systems (23%). Although science in this area is emerging, results suggest that intervention programmes that included more components of the Chronic Care Model were more effective.

Review: electronic health records and the reliability and validity of quality measures: a review of the literature

Previous reviews of research on electronic health record (EHR) data quality have not focused on the needs of quality measurement. The authors reviewed empirical studies of EHR data quality, published from January 2004, with an emphasis on data attributes relevant to quality measurement. Many of the 35 studies reviewed examined multiple aspects of data quality. Sixty-six percent evaluated data accuracy, 57% data completeness, and 23% data comparability. The diversity in data element, study setting, population, health condition, and EHR system studied within this body of literature made drawing specific conclusions regarding EHR data quality challenging. Future research should focus on the quality of data from specific EHR components and important data attributes for quality measurement such as granularity, timeliness, and comparability. Finally, factors associated with poor or variability in data quality need to be better understood and effective interventions developed.

The challenge of measuring quality of care from the electronic health record

The electronic health record (EHR) is seen by many as an ideal vehicle for measuring quality of health care and monitoring ongoing provider performance. It is anticipated that the availability of EHR-extracted data will allow quality assessment without the expensive and time-consuming process of medical record abstraction. A review of the data requirements for the indicators in the Quality Assessment Tools system suggests that only about a third of the indicators would be readily accessible from EHR data. Other factors involving complexity of required data elements, provider documentation habits, and EHR variability make the task of quality measurement more difficult than may be appreciated. Accurately identifying eligible cases for quality assessment and validly scoring those cases with EHR-extracted data will pose significant challenges but could potentially plummet the cost and therefore expand the use of quality assessment.

Using technology to promote gastrointestinal outcomes research: a case for electronic health records

Electronic health records (EHRs) have been shown to reduce medication errors, improve patient outcomes, and create administrative efficiencies. Numerous public and private efforts are currently underway to achieve universal EHR adoption in the United States by the year 2014. EHRs hold a great potential to integrate clinical care and research by allowing input of clinical data in a structured format, facilitating electronic data capture for clinical trials and providing linkage with genomic information. The goal of this article is to inform the academic gastrointestinal community about the research opportunities created by the widespread adoption of EHRs and present a systematic approach in utilizing EHR-derived data for observational, experimental, or translational studies.

The Clinical Outcomes Assessment Toolkit: a framework to support automated clinical records-based outcomes assessment and performance measurement research

The Clinical Outcomes Assessment Toolkit (COAT) was created through a collaboration between the University of California, Los Angeles and Brigham and Women's Hospital to address the challenge of gathering, formatting, and abstracting data for clinical outcomes and performance measurement research. COAT provides a framework for the development of information pipelines to transform clinical data from its original structured, semi-structured, and unstructured forms to a standardized format amenable to statistical analysis. This system includes a collection of clinical data structures, reusable utilities for information analysis and transformation, and a graphical user interface through which pipelines can be controlled and their results audited by nontechnical users. The COAT architecture is presented, as well as two case studies of current implementations in the domain of prostate cancer outcomes assessment.

Recommendations for prevention of childhood obesity

The majority of US youth are of healthy weight, but the majority of US adults are overweight or obese. Therefore, a major health challenge for most American children and adolescents is obesity prevention-today, and as they age into adulthood. In this report, we review the most recent evidence regarding many behavioral and practice interventions related to childhood obesity, and we present recommendations to health care providers. Because of the importance, we also suggest approaches that clinicians can use to encourage obesity prevention among children, including specific counseling strategies and practice-based, systems-level interventions. In addition, we suggest how clinicians may interact with and promote local and state policy initiatives designed to prevent obesity in their communities.

Measuring performance directly using the veterans health administration electronic medical record: a comparison with external peer review

BACKGROUND

Electronic medical records systems (EMR) contain many directly analyzable data fields that may reduce the need for extensive chart review, thus allowing for performance measures to be assessed on a larger proportion of patients in care.

OBJECTIVE

This study sought to determine the extent to which selected chart review-based clinical performance measures could be accurately replicated using readily available and directly analyzable EMR data.

METHODS

A cross-sectional study using full chart review results from the Veterans Health Administration's External Peer Review Program (EPRP) was merged to EMR data.

RESULTS

Over 80% of the data on these selected measures found in chart review was available in a directly analyzable form in the EMR. The extent of missing EMR data varied by site of care (P<0.01). Among patients on whom both sources of data were available, we found a high degree of correlation between the 2 sources in the measures assessed (correlations of 0.89-0.98) and in the concordance between the measures using performance cut points (kappa: 0.86-0.99). Furthermore, there was little evidence of bias; the differences in values were not clinically meaningful (difference of 0.9 mg/dL for low-density lipoprotein cholesterol, 1.2 mm Hg for systolic blood pressure, 0.3 mm Hg for diastolic, and no difference for HgbA1c).

CONCLUSIONS

Directly analyzable data fields in the EMR can accurately reproduce selected EPRP measures on most patients. We found no evidence of systematic differences in performance values among these with and without directly analyzable data in the EMR.

Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures

New reimbursement policies and pay-for-performance programs to reward providers for producing better outcomes are proliferating. Although electronic health record (EHR) systems could provide essential clinical data upon which to base quality measures, most metrics in use were derived from administrative claims data. We compared commonly used quality measures calculated from administrative data to those derived from clinical data in an EHR based on a random sample of 125 charts of Medicare patients with diabetes. Using standard definitions based on administrative data (which require two visits with an encounter diagnosis of diabetes during the measurement period), only 75% of diabetics determined by manually reviewing the EHR (the gold standard) were identified. In contrast, 97% of diabetics were identified using coded information in the EHR. The discrepancies in identified patients resulted in statistically significant differences in the quality measures for frequency of HbA1c testing, control of blood pressure, frequency of testing for urine protein, and frequency of eye exams for diabetic patients. New development of standardized quality measures should shift from claims-based measures to clinically based measures that can be derived from coded information in an EHR. Using data from EHRs will also leverage their clinical content without adding burden to the care process.

Medical disease and alcohol use among veterans with human immunodeficiency infection: A comparison of disease measurement strategies

BACKGROUND

Many people with human immunodeficiency (HIV) infection drink alcohol. We asked whether level of exposure to alcohol is associated with medical disease in a linear or nonlinear manner, whether the association depends upon the proximity of alcohol use, and whether it varies by source used to measure disease (chart review vs. ICD-9 Diagnostic Codes).

METHODS

The Veterans Aging 3 Site Cohort Study (VACS 3) enrolled 881 veterans, 86% of all HIV-positive patients seen, at 3 VA sites from June 23, 1999, to July 28, 2000. To maximize the sensitivity for alcohol exposure, alcohol use was measured combining data from patient self-report, chart review, and ICD-9 codes. We assigned the greatest exposure level reported from any source. Alcohol use within the past 12 months was considered current. Data on comorbid and AIDS-defining medical diseases were collected via chart review and ICD-9 diagnostic codes. The association of alcohol use (level and timing) and disease was modeled only for diseases demonstrating > or =10% prevalence. Linearity was compared with nonlinearity of association using nested multivariate models and the likelihood ratio test. All multivariate models were adjusted for age, CD4 cell count, viral load, intravenous drug use, exercise, and smoking.

RESULTS

Of 881 subjects enrolled, 866 (98%) had sufficient data for multivariate analyses, and 876 (99%) had sufficient data for comparison of chart review with ICD-9 Diagnostic Codes. Of the 866, 42 (5%) were lifetime abstainers; 247 (29%) were past drinkers; and 577 (67%) were current users. Among the 824 reporting past or current alcohol use, 341 (41%) drank in moderation, 192 (23%) drank hazardously, and 291 (35%) carried a diagnosis of abuse or dependence. ICD-9 codes showed limited sensitivity, but overall agreement with chart review was good for 15 of 20 diseases (kappa > 0.4). The following diseases demonstrated a > or =10% prevalence with both measures (hepatitis C, hypertension, diabetes, obstructive lung disease, candidiasis, and bacterial pneumonia). All of these were associated with alcohol use (P < 0.05). Hepatitis C, hypertension, obstructive lung disease, candidiasis, and bacterial pneumonia demonstrated linear associations with level of alcohol use (P < 0.03). Past alcohol use increased the risk of hepatitis C and diabetes after adjustment for level of exposure (P < 0.01). With the exception of candidiasis, the associations between level and timing of alcohol use were similar when measured by ICD-9 codes or by chart review.

CONCLUSIONS

Past and current use of alcohol is common among those with HIV infection. Estimates of disease risk associated with alcohol use based upon ICD-9 Diagnostic Codes appear similar to those based upon chart review. After adjustment for level of alcohol exposure, past use is associated with similar (or higher) prevalence of disease as among current drinkers. Finally, level of alcohol use is linearly associated with medical disease. We find no evidence of a "safe" level of consumption among those with HIV infection.

Medical record review of deaths, unexpected intensive care unit admissions, and clinician referrals: detection of adverse events and insight into the system

AIMS AND METHODS

To determine whether a programme of continuous medical record review of deaths, unexpected intensive care unit (ICU) admissions, and admissions referred by medical and nursing staff for specific review, would provide a range of adverse events from which to gain insight into the healthcare system of a large paediatric referral hospital. A quality assurance programme was commenced in 1996.

RESULTS

Over a six year period there were 103 255 admissions; 1612 (1.6%) records were reviewed, from which 325 adverse events were detected. Events were associated with operations, procedures and anaesthesia (56.5%), diagnosis and therapy (24%), drug and fluid management (12.6%), and system issues (7%). Medical records were reviewed from 23 of the 28 clinical units. Review of the records and analysis of the adverse events triggered many system changes.

CONCLUSIONS

The findings suggest that continuous medical record review may be a valuable method for the detection of adverse events and identifying system issues in children's hospitals.