Reliability audit of a regional cardiac surgery registry

Assessment of data quality in an international multi-centre randomised trial of coronary artery surgery

Background

ART is a multi-centre randomised trial of cardiac surgery which provided a unique opportunity to evaluate the data from a large number of centres from a variety of countries. We attempted to assess data quality, including recruitment rates, timeliness and completeness of the data obtained from the centres in different socio-economic strata.

Methods

The analysis was based on the 2-page CRF completed at the 6 week follow-up. CRF pages were categorised into "clean" (no edit query) and "dirty" (any incomplete, inconsistent or illegible data). The timelines were assessed on the basis of the time interval from the visit and receipt of complete CRF. Data quality was defined as the number of data queries (in percent) and time delay (in days) between visit and receipt of correct data. Analyses were stratified according to the World Bank definitions into: "Developing" countries (Poland, Brazil and India) and "Developed" (Italy, UK, Austria and Australia).

Results

There were 18 centres in the "Developed" and 10 centres in the "Developing" countries. The rate of enrolment did not differ significantly by economic level ("Developing":4.1 persons/month, "Developed":3.7 persons/month). The time interval for the receipt of data was longer for "Developing" countries (median:37 days) compared to "Developed" ones (median:11 days) (p < 0.001). The median number of data queries was 23% in "Developed" countries compared to 19% in "Developing" ones (p = ns).

Conclusions

In this study we showed that data quality was comparable between centres from "Developed" and "Developing" countries. Data was received in a less timely fashion from Developing countries and appropriate systems should be instigated to minimize any delays. Close attention should be paid to the training of centres and to the central management of data quality.

Trial registration

ISRCTN46552265

Designing and implementing an Australian and New Zealand intensive care data audit study

PURPOSE

The aim of this pilot audit study is to develop and test a model to examine existing adult patient database (APD) data quality.

DESIGN/METHODOLOGY/APPROACH

A database was created to audit 50 records per site to determine accuracy. The audited records were randomly selected from the calendar year 2004 and four sites participated in the pilot audit study. A total of 41 data elements were assessed for data quality--those elements required for APACHE II scoring system.

FINDINGS

Results showed that the audit was feasible; missing audit data were an unplanned problem; analysis was complicated owing to the way the APACHE calculations are performed and 50 records per site was too time-consuming.

ORIGINALITY/VALUE

This is the first audit study of intensive care data within the ANZICS APD and demonstrates how to determine data quality in a large database containing individual patient records.

Quantifying data quality for clinical trials using electronic data capture

Background

Historically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes.

Methods and Principal Findings

The National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions.

Conclusions

Historically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

Do women spend longer on wait lists for coronary bypass surgery? Analysis of a population-based registry in British Columbia, Canada

Background

Studies have shown patients who are delayed for surgical cardiac revascularization are faced with increased risks of symptom deterioration and death. This could explain the observation that operative mortality among persons undergoing coronary artery bypass surgery (CABG) is higher among women than men. However, in jurisdictions that employ priority wait lists to manage access to elective cardiac surgery, there is little information on whether women wait longer than men for CABG. It is therefore difficult to ascertain whether higher operative mortality among women is due to biological differences or to delayed access to elective CABG.

Methods

Using records from a population-based registry, we compared the wait-list time between women and men in British Columbia (BC) between 1990 and 2000. We compared the number of weeks from registration to surgery for equal proportions of women and men, after adjusting for priority, comorbidity and age.

Results

In BC in the 1990s, 9,167 patients aged 40 years and over were registered on wait lists for CABG and spent a total of 136,071 person-weeks waiting. At the time of registration for CABG, women were more likely to have a comorbid condition than men. We found little evidence to suggest that women waited longer than men for CABG after registration, after adjusting for comorbidity and age, either overall or within three priority groups.

Conclusion

Our findings support the hypothesis that higher operative mortality during elective CABG operations observed among women is not due to longer delays for the procedure.

Cumulative incidence for wait-list death in relation to length of queue for coronary-artery bypass grafting: a cohort study

Background

In deciding where to undergo coronary-artery bypass grafting, the length of surgical wait lists is often the only information available to cardiologists and their patients. Our objective was to compare the cumulative incidence for death on the wait list according to the length of wait lists at the time of registration for the operation.

Methods

The study cohort included 8966 patients who registered to undergo isolated coronary-artery bypass grafting (82.4% men; 71.9% semi-urgent; 22.4% non-urgent). The patients were categorized according to wait-list clearance time at registration: either "1 month or less" or "more than 1 month". Cumulative incidence for wait-list death was compared between the groups, and the significance of difference was tested by means of regression models.

Results

Urgent patients never registered on a wait list with a clearance time of more than 1 month. Semi-urgent patients registered on shorter wait lists more often than non-urgent patients (79.1% vs. 44.7%). In semi-urgent and non-urgent patients, the observed proportion of wait-list deaths by 52 weeks was lower in category "1 month or less" than in category "more than 1 month" (0.8% [49 deaths] vs. 1.6% [39 deaths], P < 0.005). After adjustment, the odds of death before surgery were 64% higher in patients on longer lists, odds ratio [OR] = 1.64 (95% confidence interval [CI] 1.02–2.63). The observed death rate was higher in category "more than 1 month" than in category "1 month or less", 0.79 (95%CI 0.54–1.04) vs. 0.58 (95% CI 0.42–0.74) per 1000 patient-weeks, the adjusted OR = 1.60 (95%CI 1.01–2.53). Longer wait times (log-rank test = 266.4, P < 0.001) and higher death rates contributed to a higher cumulative incidence for death on the wait list with a clearance time of more than 1 month.

Conclusion

Long wait lists for coronary-artery bypass grafting are associated with increased probability that a patient dies before surgery. Physicians who advise patients where to undergo cardiac revascularization should consider the risk of pre-surgical death that is associated with the length of a surgical wait list.

The risk of death associated with delayed coronary artery bypass surgery

Background

The detrimental effect of delaying surgical revascularization has been estimated in randomized trials and observational studies. It has been argued that the Kaplan-Meier method used in quantifying the hazard of delayed treatment is not appropriate for summarizing the probability of competing outcomes. Therefore, we sought to improve the estimates of the risk of death associated with delayed surgical treatment of coronary artery disease.

Methods

Population-based prospective study of 8,325 patients registered to undergo first time isolated coronary artery bypass grafting (CABG) in any of the four tertiary hospitals that provide cardiac care to adult residents of British Columbia, Canada. The cumulative incidence of pre-operative death, the cumulative incidence of surgery, and the probability that a patient, who may die or undergo surgery, dies if not operated by certain times over the 52-week period after the decision for CABG were estimated. The risks were quantified separately in two groups: high-severity at presentation were patients with either persistent unstable angina or stable angina and extensive coronary artery disease, and low-severity at presentation were stable symptomatic patients with limited disease.

Results

The median waiting time for surgery was 10 weeks (interquartile range [IQR] 15 weeks) in the high-severity group and 21 weeks (IQR 30 weeks) in the low-severity group. One percent of patients died before surgery: 54 in the high-severity and 26 in the low-severity group. For 58 (72.5%) patients, death was related to CVD (acute coronary syndrome, 33; chronic CVD, 16; other CVD, 4; and sudden deaths, 5). The overall death rate from all causes was 0.61 (95% CI 0.48-0.74) per 1,000 patient-weeks, varying from 0.62 (95% CI 0.45-0.78) in the high-severity group to 0.59 (95% CI 0.37-0.82) in the low-severity group. After adjustment for age, sex, and comorbidity, the all-cause death rate in the low-severity group was similar to the high-severity group (OR = 1.02, 95% CI 0.64-1.62). The conditional probability of death was greater in the high-severity group than in the low-severity group both for all-cause mortality (p = 0.002) and cardiovascular deaths (p <0.001).

Conclusion

The probability of death conditional on not having undergone a required CABG increases with time spent on wait lists.

Do Longer Delays for Coronary Artery Bypass Surgery Contribute to Preoperative Mortality in Less Urgent Patients?

BACKGROUND

Priority wait lists are common for managing access to cardiac surgery in publicly funded health systems. We evaluated whether longer delays contribute to the probability of death before surgery among patients prioritized into the less urgent category.

METHODS

We studied records of 9233 patients registered for isolated coronary artery bypass graft (CABG) in British Columbia, Canada. The primary outcome was death before surgery. We estimated the probability that a patient, who could be removed from the list as a result of surgery, death, or other competing events, dies on or before a certain wait-list week.

RESULTS

Despite similar death rates in semiurgent and nonurgent groups, 0.63 (95% confidence interval, 0.46-0.80) versus 0.58 (0.36-0.80) per 1000 patient-weeks, nonurgent patients were remaining on the list longer, which contributed to higher cumulative incidence of all-cause death than in semiurgent group (adjusted odds ratio = 1.66; 1.03-2.68). By 52 weeks on the wait list, 0.9% (0.6-1.1) and 1.3% (0.8-1.8) of patients died in semiurgent and nonurgent groups, respectively (P < 0.01). Similar proportions of deaths related to cardiovascular disease estimated over wait-list time in both groups (P = 0.40) were the result of shorter delays in the semiurgent group despite a higher rate of death resulting from cardiovascular disease (0.50 [0.36-0.65] vs. 0.34 [0.17-0.51] per 1000 patient-weeks).

CONCLUSION

Queuing according to urgency of treatment contributed to a higher proportion of CABG candidates dying before surgery from all causes in the nonurgent compared with the semiurgent group despite similar weekly death rates observed in both groups. However, similar probabilities of death resulting from cardiovascular disease observed in both groups over wait-list time were the result of shorter delays in the semiurgent group despite a higher rate of cardiovascular death.

Time on wait lists for coronary bypass surgery in British Columbia, Canada, 1991-2000

BACKGROUND

In British Columbia, Canada, all necessary medical services are funded publicly. Concerned with growing wait lists in the mid-1990s, the provincial government started providing extra funding for coronary artery bypass grafting (CABG) operations annually. Although aimed at improving access, it is not known whether supplementary funding changed the time that patients spent on wait lists for CABG. We sought to determine whether the period of registration on wait lists had an effect on time to isolated CABG and whether the period effect was similar across priority groups.

METHODS

Using records from a population-based registry, we studied the wait-list time before and after supplementary funding became available. We compared the number of weeks from registration to surgery for equal proportions of patients in synthetic cohorts defined by five registration periods in the 1990s.

RESULTS

Overall, 9,231 patients spent a total of 137,126 person-weeks on the wait lists. The time to surgery increased by the middle of the decade, and decreased toward the end of the decade. Relative to the 1991-92 registration period, the conditional weekly probabilities of undergoing surgery were 30% lower among patients registered on the wait lists in 1995-96, hazard ratio (HR) = 0.70 (0.65-0.76), and 23% lower in 1997-98 patients, HR = 0.77 (0.71-0.83), while there were no differences with 1999-2000 patients, HR = 0.94 (0.88-1.02), after adjusting for priority group at registration, comorbidity, age and sex. We found that the effect of registration period was different across priority groups.

CONCLUSION

Our results provide evidence that time to CABG shortened after supplementary funding was provided on an annual basis to tertiary care hospitals within a single publicly funded health system. One plausible explanation is that these hospitals had capacity to increase the number of operations. At the same time, the effect was not uniform across priority groups indicating that changes in clinical practice should be considered when adding extra funding to reduce wait lists.

The Registration of Complications in Surgery: A Learning Curve

Registration of complications in surgery is an important method used for quality improvement. Unfortunately many different definitions and classification systems have been used, which influences the interpretation and the outcome of complication registration. Since 1986 complications have been registered on a daily basis in our surgical department. We focus in this article on the influence of changes in interpretation of the definition and registration methods used on the incidence of registered complications. Between 1986 and 1993 complications registered were strictly related to surgical procedures. In the second period, between 1993 and 2001, the interpretation of the definition changed and all adverse events were registered in a patient-centred way, not only related to the surgical procedure. The definition used in both periods did not change. In 1993 we started with the implementation of a fully automated registration system in our surgical department. In the first period 1699 (7%) complications in 24,201 surgical procedures were registered and in the second period 8335 (27%) complications were registered in 31,161 surgical procedures. A dramatic increase in the total number of registered complications was seen with the implementation of a fully automated registration system and a patient-centred way of registering complications. In the context of the evolving discussion of quality of care, a uniform definition and registration system has to be used to assure reliable outcome data in surgery and to form a basis for comparison.

Use of Physiologic Pacing After the Canadian Trial of Physiologic Pacing

The Canadian trial of physiologic pacing (CTOPP), published in 2000, demonstrated a reduction in atrial fibrillation (AF), stroke and death with preservation of atrioventricular synchrony, though only the lower rate of AF was statistically significant. The purpose of this study was to determine the effect of CTOPP on pacing mode selection in our region. The British Columbia Cardiac Registry contains prospectively entered data covering a population of 4 millions (M) and 17 implanting centers. It was examined for mode selection trends from 1997 to 2002. At examination, there were data on 22,446 pulse generators (PG) and 29,898 leads. New implant rates per M population were 1997:473; 1998:456; 1999:505; 2000:513; 2001:486; 2002:510. PG replacements also increased, resulting in a total implant rate of 667 PG per M in 2002. Over the 6-year period, DDD use decreased from 321 to 306, but DDDR use, more than doubled from 317 to 750 PG/year. VVI use steadily decreased from 741 to 410 PG/year, while VVIR use increased more modestly from 1997 to 1999, then remained stable. During the 6-year period bracketing CTOPP, use of modes maintaining AV synchrony increased by over 32%, to 53% of PG implanted in 2002. Our PG implant rate was much higher than expected from prior retrospective surveys, and similar to rates in Belgium, France, and Germany. CTOPP did not decrease our use of physiologic pacing but, instead, was associated with a brief pause, then progressively increased in both academic and community centers. Patients' need and widely accepted standards of care proved more important in clinical decision making than the results of a flawed randomized trial.

New-onset sustained ventricular tachycardia and fibrillation early after cardiac operations

BACKGROUND

Malignant ventricular tachyarrhythmia early after cardiac surgery is an uncommon arrhythmic complication but has a negative impact on mortality. The purpose of this study was to evaluate the incidence of new-onset sustained postoperative ventricular tachycardia-ventricular fibrillation and to identify risk factors for the dysrhythmia.

METHODS

Demographic, clinical, operative, and postoperative data, including a variable of postoperative ventricular tachycardia, were prospectively obtained from 4748 patients undergoing nonemergency coronary artery bypass graft and(or) valve replacement with no history of sustained ventricular tachycardia or sudden death. A detailed analysis was performed to define the risk factors for the ventricular tachycardia and the prognostic impact of the arrhythmia on 30-day mortality was evaluated.

RESULTS

Forty-five patients (0.95%) had sustained ventricular tachycardia or ventricular fibrillation and the initial episode occurred 3.9 +/- 5.2 days (mean +/- standard deviation) after surgery. By multivariate analysis, female sex (odds ratio, 1.982), left ventricular ejection fraction (< 35%: > 50%, 4.771), the presence of pulmonary hypertension (3.066), the presence of systemic hypertension (2.391), and pump time (per 10 minutes, 1.085) were independently associated with the dysrhythmias. Early mortality of patients with the arrhythmia was 28.9%, strikingly higher than that of patients without ventricular tachycardia/ventricular fibrillation (1.9%).

CONCLUSIONS

Left ventricular ejection fraction is the strongest risk factor for new-onset postoperative sustained ventricular tachycardia-ventricular fibrillation; female sex, pump time, pulmonary and systemic hypertension are independent predictors of the dysrhythmias; the arrhythmia is associated with increased 30-day mortality after cardiac surgery.

Systematic review of the quality of surgical mortality monitoring

BACKGROUND

Mortality is the most tightly defined and used adverse event for audit and performance monitoring in surgery. However, to identify cause and therefore scope for improvement, accurate and timely data are required. The aim of this study was to perform a systematic review of the quality of measurement, reporting and monitoring of mortality as an outcome after surgery.

METHODS

A systematic review of published literature was undertaken for the 7-year interval 1993-1999. Grey and unpublished literature was obtained through the Royal College of Surgeons of England, from UK national audits and routine national hospital data collections.

RESULTS

Eligible monitoring systems included six UK national surgical audits, and cardiac and vascular surgery monitoring systems from North America and the UK. The definitions of 'surgical death' varied in several respects and deaths after discharge from hospital were rarely ascertained unless there was routine linkage to national death registers. There were very few published studies on validation of the completeness and accuracy of the data collection.

CONCLUSION

A comprehensive data collection system is needed for improving clinical performance, with ownership, but not necessarily data collection, resting with the surgeons concerned. Recording of risk factors and deaths after discharge from hospital is essential, whatever data collection system is used.

Assessing data adequacy for clinical research: reliability and validity of a surgical database

As clinical databases are utilized more frequently for clinical research, it is essential that researchers assess the quality of databased information. While researchers have begun to report strategies to measure accuracy of databased information, knowledge remains limited. The purpose of this study was to assess the reliability and validity of databased information among selected study variables contained within a computerized coronary artery surgery clinical database using the written patient medical record as an external standard. Both reliability (N = 400) and validity (N = 100) samples were randomly selected from a databased sampling frame of 548 Medicare subjects who underwent coronary artery bypass grafting surgery in 1998. Reliability assessed by consistency rates were age (95%), race (94%), gender (99%), congestive heart failure (CHF) (60.5%), angina (91.5%), renal insufficiency (82%), hypertension (91.7%), diabetes mellitus (93.7%), chronic obstructive pulmonary disease (COPD) (75.5%), clinical status (97%), American Society of Anesthesiologists classification (99%), prior peripheral vascular surgery (15.5%), prior CABGS (99%), and duration of mechanical ventilation (87.5%). These percentages reflected a large portion of missing data for CHF, COPD, and prior peripheral vascular surgery. Validity assessed by sensitivity and specificity analyses were all greater than 80%. The majority of computerized databased information among selected study variables was the same information recorded in the written patient medical record. Using the same external standard to assess both reliability and validity was a significant limitation of this study, which resulted in the same measure of data adequacy by utilizing differing statistical methods.

Completeness of data entry in three cancer surgery databases

AIMS

Clinical databases are regularly used for audit and research purposes. The accuracy of data input is critical to the value of these tools, but little is known about the factors which influence the completeness of data recording. The aim of this study was to evaluate the influences affecting completeness of data recording in computerized clinical databases of cancer treatment.

METHODS

Data omission rates in three databases dealing with management of breast, colorectal and gastro-oesophageal cancers were calculated. The effects of (a) type of record; (b) nature of data and (c) training required to interpret data were evaluated by univariate and multivariate analyses.

RESULTS

The overall data omission rate was 21.9% (upper GI 27.6%, breast 19.6%, colorectal 32.7%, P=0.13). For different categories of data, omission rates varied from 0% to 55%. Fields requiring a 'text field' or 'numerical' entry, or containing demographic data, data required for the process of care or data which required no interpretation were associated with low omission rates. Clinical data, and fields requiring a 'yes/no' response were associated with high omission rates (45 and 48% respectively). Clinical data and data relating to patient demographic details were independently associated with high and low omission rates respectively (odds ratios for significant missing data 86.9 and 1 respectively).

CONCLUSION

Clinical data are poorly captured by current cancer surgery databases. Reasons for the poor completion of fields requiring input by clinical staff, particularly availability of time and training, and prioritization of work, should be addressed. Re-design of databases to ensure that data entry is simple and unambiguous may improve accuracy.

Pathology data in the central databases of multicenter randomized trials need to be based on pathology reports and controlled by trained quality managers

PURPOSE

Randomized multicenter trials form the basis of health care development. Regarding cancer research, pathology data are crucial. To maintain the quality of these trials, the auditing of subsequent processes is necessary. The aim of the present study was to examine the completeness and accuracy of data obtained from a special-purpose standardized pathology form compared with the data available through traditional hospital pathology reports.

PATIENTS AND METHODS

A retrospective comparison of pathology data case record forms with hospital pathology reports was performed using the data from 300 patients with primary rectal cancer. All of these patients had been included in a large multicenter trial in the Netherlands. Three independent audits were carried out. Special attention was given to the accuracy of parameters, which are important for prognosis and treatment decisions. Furthermore, various factors that possibly influence the occurrence of errors were investigated.

RESULTS

Quality control of the pathology data revealed a high accuracy of 86.5% of all data items. However, only one third of the forms were complete and correct. Missing values were most prominent in the number of lymph nodes examined, whereas most errors were made in relation to the circumferential margin. Trained review pathologists made fewer major errors. Discrepancies were detected in all control rounds.

CONCLUSION

Successive rounds of quality control are required for accuracy and completeness of pathology data in multicenter trials. In addition to the special-purpose pathology forms, original pathology reports have to be collected, and the data should also be controlled by a trained pathology quality manager.