The importance of reproducibility testing of computer programs for electrocardiographic interpretation: application to the automatic vectorcardiographic analysis program (AVA 3.4)

Variability in ECG computer interpretation. Analysis of individual complexes vs analysis of a representative complex

Variability in the electrocardiogram (ECG) can be due to extrinsic noise or can be caused by intrinsic factors, such as changes in the volume conductor or in the heart itself. Computer programs for the interpretation of the ECG base their diagnostic classification on one set of measurements that is derived from a representative PQRST complex or that is computed by taking the median from the measurements for each complex in the recording. However, these methods may fail to do justice to the intrinsic variability that may be present in the ECG. An alternative method is proposed: derive a set of measurements from each complex in the recording, classify each individual complex separately, and then combine the individual classifications into one final classification. This procedure has been evaluated on a validated database (n = 1,220) using an ECG computer program. Total accuracy against the clinical evidence increased from 69.8% for the interpretations of the averaged complexes to 71.2% for the combined interpretations of the individual complexes (p < 0.001). The effect of beat-to-beat variation on the measurements and classifications is demonstrated and the influence of extrinsic and intrinsic variability is assessed.

Stability of computer ECG amplitude measurements in the presence of noise. The CSE Working Party

An important feature of an ECG analysis program is its ability to provide reliable measurements under various operating conditions, e.g., on noise-free and noisy ECGs. Therefore, within the European cooperative project "Common Standards For Quantitative Electrocardiography" (CSE), the accuracy and stability of ECG measurements obtained by several computer programs has been compared. To investigate the stability of measurements two sets of 10 ECGs with and without seven different high- and low-frequency types of noise--altogether 160 electrocardiograms and 160 vectorcardiograms--have been analyzed by eight electrocardiographic and five vectorcardiographic computer programs. The stability of measurement was tested with respect to results obtained for the noise-free recordings. In a previous paper, the influence of noise on wave boundary recognition has been reported. In the present paper, the effect of noise on amplitude measurements and on problems of waveform definitions within the QRS complex are described. The results indicate that programs analyzing an averaged beat exhibit less variability than programs which measure every complex or a selected beat. Comparability and stability of measurements could be improved if a standardized procedure for amplitude references were to be introduced. In addition, the stability of QRS waveform labelling could be improved if waveforms' minimum amplitude and duration were to be validated against the noise level which itself should be determined by a standardized procedure.

The reproducibility of interpretation of 10 computer ECG systems by means of a microprocessor-based ECG signal generator

The use of computer ECG systems has become widespread in Japan, and many new models are available. In order to provide a method for comparative evaluation, a microprocessor-based ECG signal generator was built. This generator was used to compare 10 of the computer ECG systems which are currently available in Japan. Since the generator can produce a precisely defined wave form, it is particularly suited for testing the reproducibility of diagnostic interpretation. Several deficiencies of some current ECG analysis systems were noted. Specifically in three systems, there was interference of the interpretation of morphology with the interpretation of rhythm when WPW-like patterns were analyzed. The current evaluation concentrates on the reproducibility of interpretation in the tested systems. As the library of test ECG patterns is expanded in the future, the technique will also become applicable to the evaluation of diagnostic accuracy.

A new microcomputer-based ECG analysis system

A new automated ECG system using advances in microprocessor technology and computerized electrocardiography is described. This microcomputer-based system is self-contained and mobile. It acquires both the 12-lead and orthogonal lead (Frank) electrocardiograms and analyzes the latter within minutes. Software includes the program developed in the Veterans Administration which uses advanced statistical classification techniques and a large well-documented patient data base. Diagnostic probabilities are computed using a Bayesian approach. Diagnostic performance has been tested using independent clinical criteria and found to be quite accurate. This system enables the clinician to immediately review the computer's identifications, measurements, and diagnostic classifications and quickly use these results in clinical decision making. Serial comparisons are readily made since all previous recordings are stored on floppy diskettes. The use of microprocessors in this system makes it economically feasible for practicing physicians.

The impact of an ECG computer analysis program on the cardiologist's interpretation. A cooperative study

Nine experienced electrocardiographers and the ECG computer program developed in the Veterans Administration (AVA 4.0) were evaluated against ECG-independent evidence of 180 patients' true diagnoses. A cross section of cardiac abnormalities was included. Each reader was given the 12-lead and orthogonal 3-lead ECG. The impact of ECG computer reports on the interpretations by the nine readers was evaluated by comparing their interpretations before and after the addition of a computer report. Using only high probability statements, the average accuracy of ECG diagnosis by the nine readers was 54%. It increased to 62% when the computer report was added. Computer interpretation was correct in 76%. It was shown that the Bayesian classification method together with multivariate analysis, used in the VA program, are mainly responsible for the improvement in diagnostic accuracy.

XYZ data interpreted by a 12-lead computer program using the derived electrocardiogram

The 12-lead electrocardiogram (ECG) derived from the Frank xyz signals was compared with the conventional 12-lead ECG using the Telemed computer system. In 100 cases studied. Telemed's interpretations were essentially similar in 77, but substantially different in 23. In the 23 cases, interpretations of the derived tracings tended to be more accurate in 14 cases, and less accurate in four cases. In the diagnosis of infarction the probability that the interpretation of the derived tracing will be correct more often was 90%. The better performance may have been related to closer agreement with the vectorcardiogram (VCG). As a substitute for the conventional ECG, the derived ECG offers the prospect of a computerized system that is more practical and more versatile than most currently used systems.

Clinical evaluation of automated processing of electrocardiograms by the Veterans Administration program (AVA 3.4)

Automated processing of electrocardiograms by the Veterans Administration program was evaluated for both agreement with physician interpretation and interpretative accuracy as assessed with nonelectrocardiographic criteria. One thousand unselected electrocardiograms were analyzed by two reviewer groups, one familiar and the other unfamiliar with the computer program. A significant number of measurement errors involving repolarization changes and left axis deviation occurred; however, interpretative disagreements related to statistical decision were largely language-related. Use of a printout with a more traditional format resulted in agreement with physician interpretation by both reviewer groups in more than 80 percent of cases. Overall sensitivity based on agreement with nonelectrocardiographic criteria was significantly greater with use of the computer program than with use of the conventional criteria utilized by the reviewers. This difference was particularly evident in the subgroup analysis of myocardial infarction and left ventricular hypertrophy. The degree of overdiagnosis of left ventricular hypertrophy and posteroinferior infarction was initially unacceptable, but this difficulty was corrected by adjustment of probabilities. Clinical acceptability of the Veterans Administration program appears to require greater physician education than that needed for other computer programs of electrocardiographic analysis; the flexibility of interpretation by statistical decision offers the potential for better diagnostic accuracy.