Measuring agreement in method comparison studies

The importance of the level of the lip line and resting lip pressure in Class II, Division 2 malocclusion

Many clinicians hypothesize that retroclination of the maxillary central incisors in Class II, Division 2 malocclusion is caused by increased resting lip pressure against these teeth. The purpose of this study was (1) to verify this assumption by means of simultaneous lip-pressure measurements at two different levels on the maxillary central incisor crowns, and (2) to examine factors that could possibly contribute to the increased resting lip pressure. This is the first study to prove that individuals with Class II, Division 2 malocclusion (n = 21) have the upper central incisors exposed to significantly higher lip pressure than those with Class I malocclusion (n = 21). Our statistical evaluation revealed that this is primarily attributed to a high lip line and not to a hypertonic peri-oral musculature. We concluded that orthodontic treatment of Class II, Division 2 cases should include intrusion of the maxillary incisors, to eliminate the non-physiologically high pressure exerted by the lower lip on these teeth and, consequently, to reduce the high risk of a post-orthodontic relapse.

L-shell x-ray fluorescence measurements of lead in bone: accuracy and precision

This study aimed to quantify the accuracy and precision of a method for in vivo measurements of lead in bone using L-shell x-ray fluorescence (LXRF), the former via comparison with independent measurements of lead in bone obtained using electrothermal atomic absorption spectrometry (AAS) following acid digestion. Using LXRF. the lead content of adult human cadaver tibiae was measured, both as intact legs and as dissected tibiae with overlying tissue removed, the latter at several proximal-distal locations. After LXRF, each tibia was divided into nine cross-sectional segments, which were further separated into tibia core and surface samples for AAS measurement. The proximal-distal variability of AAS-measured core and surface tibia lead concentrations has been described elsewhere (the lead concentration was found to decrease towards both ends of the tibia). The subjects of this paper are the proximal-distal variability of the LXRF-measured lead concentrations, the measurement uncertainty and the statistical agreement between LXRF and AAS. There was no clear proximal-distal variability in the LXRF-measured concentrations; the degree of variability in actual tibia lead concentrations is far less than the LXRF measurement uncertainty. Measurement uncertainty was dominated by counting statistics and exceeded the estimate of lead concentration in most cases. The agreement between LXRF and AAS was reasonably good for bare bone measurements but poor for intact leg measurements. The variability of the LXRF measurements was large enough, for both bare bone and intact leg measurements, to yield grave concerns about the analytical use of the technique in vivo.

Assessing agreement between CD34 enumeration by flow cytometry and volumetric analysis

Prior to replacement of an established method for CD34 enumeration by an alternative approach, evaluation of the agreement between the methods is essential. In this study, the comparison of two assays was evaluated according to the recommendation of Bland and Altman describing the agreement between two methods where the true value is not known. CD34 enumeration was performed on blood or leukapheresis product from 105 patients by flow cytometry (dual platform assay) and volumetric analysis (single platform assay). Both the flow cytometric and the volumetric analysis showed poor reproducibility for measures lower than approximately 9 CD34+ cells/mm3. For values higher than 29 CD34+ cells/mm3, evaluation of the agreement demonstrated a difference between the single and dual platform assay, where CD34 enumeration by the volumetric analysis demonstrated values 73-80% of the flow cytometric value. The difference between the two assays could be due to several technical pitfalls which are discussed.

The feasibility of transesophageal echocardiograph-guided right and left ventricular oximetry in hemodynamically stable patients undergoing coronary artery bypass grafting

There are no techniques available for continuous noninvasive measurement of the oxygen saturation of blood flowing through the heart. We assessed the feasibility and accuracy of transesophageal echocardiograph (TEE)-guided left ventricular (SpO2 LV) and right ventricular (SpO2 RV) oximetry. Twenty hemodynamically stable, well-oxygenated anesthetized patients (ASA physical status III, aged 51-75 yr) undergoing coronary artery bypass grafting were studied. A TEE probe was modified by attaching a single-use pediatric reflectance pulse oximeter just proximal to the ultrasound transducer. The TEE probe was directed toward the LV by using the transgastric mid-short axis view or toward the RV by using the transgastric RV inflow view, in random order. Readings were taken every 30 s for 10 min during a hemodynamically stable period of anesthesia. Simultaneous blood samples were taken from the radial artery and pulmonary artery to determine arterial oxygen saturation (SaO2) and mixed venous oxygen saturation (SvO2), respectively. During SpO2 LV readings, simultaneous finger pulse oximetry (SpO2 finger) was also recorded. SpO2 LV was feasible in 20 of 20 patients, and SpO2 RV was feasible in 19 of 20 patients. The mean +/- SD (range) oxygen saturation for each method was the following: SpO2 LV, 98.7% +/- 0.6% (97%-100%); SaO2, 98.7% +/- 0.6% (96.6%-99.4%); SpO2 finger, 98.1% +/- 1.2% (97%-100%); SpO2 RV, 73.9% +/- 4.7% (64%-85%); and SvO2, 74.5% +/- 4.4% (66.8%-82.6%). SpO2 LV agreed closely with SaO2 (mean difference, 0.072%). SpO2 RV agreed closely with SvO2 (mean difference, 0.65%). SpO2 LV agreed more closely with SaO2 than finger oximetry (mean difference, -0.072 vs -0.692). TEE-guided SpO2 LV and SpO2 RV are feasible in hemodynamically stable anesthetized patients and provide similar readings to arterial and mixed venous blood samples. The technique merits further investigation.

IMPLICATIONS

Transesophageal echocardiograph-guided left and right ventricular oximetry is feasible in hemodynamically stable anesthetized patients and provides similar readings to arterial and mixed venous blood samples.

The HemoCue, a point of care B-hemoglobin photometer, measures hemoglobin concentrations accurately when mixed in vitro with canine plasma and three hemoglobin-based oxygen carriers (HBOC)

PURPOSE

Accuracy of measurement of low hemoglobin concentrations using the HemoCue, a B-hemoglobin photometer (HemoCue AB, Angelholm, Sweden) may exhibit significant variability. Infusion of hemoglobin-based oxygen carriers (HBOC) results in low concentrations of plasma hemoglobin. Our study assessed B-hemoglobin photometer measurement accuracy of three HBOC: (hemoglobin glutamer-200 (bovine; Oxyglobin, Biopure Corp., Cambridge, MA, USA); hemoglobin glutamer-250 (bovine; Hemopure, Biopure Corp, Cambridge, MA, USA), and hemoglobin-raffimer, (human; Hemolink, Hemosol, Inc., Toronto, Ontario, Canada).

METHODS

In the laboratory, 45 split canine plasma samples were mixed with hemoglobin glutamer-200 (8.13, 16.25, 32.5 g x L(-1) concentrations), 45 samples were mixed with hemoglobin glutamer-250 (8.13, 16.25, 32.5 g x L(-1) concentrations), 45 with hemoglobin-raffimer (12.5, 25.0, 50.0 g x L(-1) concentrations), and measured. Plasma samples without HBOC served as control. Hemoglobin concentration was determined by a laboratory analyzer (Coulter Corporation, Hiafeah, FL, USA) and B-hemoglobin photometer (HemoCue, Angelholm, Sweden). Two independent technicians performed blinded sample measurements and randomly tested each sample five times. Results were analyzed according to Bland and Altman analysis.

RESULTS

B-hemoglobin photometer demonstrated high repeatability for all three HBOCs. Repeatability coefficients were 0.37 g x L(-1) and 0.48 g x L(-1) for hemoglobin glutamer-200, 0.39 g x L(-1) and 0.4 g x L(-1) for hemoglobin glutamer-250 and 1.07 g x L(-1) and 0.85 g x L(-1) for hemoglobin-raffimer. An acceptable agreement was found between the B-hemoglobin photometer and the laboratory analyzer for all three HBOCs tested.

CONCLUSION

The B-hemoglobin photometer accurately determined the concentration of three HBOC solutions dissolved in canine plasma.

Effect of sevoflurane on the mid-latency auditory evoked potentials measured by a new fast extracting monitor

BACKGROUND

Mid-latency auditory evoked potentials (MLAEP) are widely suppressed during general anesthesia and may therefore be useful for assessment of the depth of anesthesia. However, interpretation of amplitudes and latencies in the AEP signal is time consuming. A new monitor (A-line) that quantifies the MLAEP into an index has therefore been developed. The present study aimed to assess the precision of a prototype of the new monitor and to test the hypothesis that the depth of anesthesia index shows a graded response with changing steady-state end-expiratory concentrations of sevoflurane.

METHODS

We studied 10 ASA physical status I or II patients undergoing elective hysterectomy under combined epidural and general anesthesia by sevoflurane. Baseline auditory evoked potentials were recorded in the conscious patient immediately before induction of general anesthesia. Depth of anesthesia indices were recorded before anesthesia and at decreasing end-expiratory steady-state sevoflurane concentrations of 2.0%, 1.5%, 1.0% and 0.5%. All indices were recorded in duplicate 6 s apart. By use of an autoregressive model with exogenous input (ARX-model), the monitor extracted the AEP within 6 s. The depth of anesthesia AEP index calculated in this way was defined as the A-line ARX index (AAI).

RESULTS

Approximately 95% of the differences between repeated recordings were 5 AAI-units or less. A wide interindividual variation was observed at each observation point. AAI at 1%, 1.5% and 2% end-expiratory concentration was significantly less than the baseline AAI obtained before induction of anesthesia (P < 0.001). AAI did not change significantly in the 1-2% concentration range.

CONCLUSION

The new monitor was precise. Attenuation of the A-line ARX-index (AAI) for mid-latency auditory evoked potentials (MLAEP) during general anesthesia was profound. However, the monitor did not show a graded response with changing end-expiratory steady-state concentrations of sevoflurane.

Fractional zinc absorption using a single isotope tracer

BACKGROUND

Fractional absorption of zinc (Zn) has been measured using dual isotopes of Zn given simultaneously. An oral test dose and an intravenous (i.v.) reference dose are administered, followed by the measurement of the double isotopic enrichment (E) in urine 48 h after administration. We postulated that an estimate of the %E in urine for a given i.v. dose of Zn may be used to eliminate the need for venipuncture and the second Zn isotope.

OBJECTIVES

To determine a constant (k) for the Zn enrichment of urine after i.v. administration of a dose of labeled Zn in Zn-replete subjects. To use 'k' to calculate fractional absorption of Zn, and to compare these values to values obtained using the standard dual isotope method.

DESIGN

Single-arm cohort.

SETTING

The Hospital for Sick Children, Toronto, Canada.

SUBJECTS

Twenty-three healthy adults were recruited from the Metropolitan Toronto area. Seventeen subjects completed the study.

INTERVENTIONS

A 2.29 mg i.v. dose of (67)Zn followed immediately by a 2.50 mg oral dose of (70)Zn.

RESULTS

Population mean percentage enrichment (%E) of (67)Zn in urine was 1.43 (95% CI 1.26, 1.60). The ratio of the i.v. dose to mean %E in urine (k) was estimated to be 1.60 mg (95% CI 1.43, 1.82). There was no difference in the mean fractional absorption of Zn calculated using the single compared to the dual isotope method: 12.58% (95% CI 2.22, 22.94) vs 12.68% (95% CI 4.52, 20.85), respectively (P=0.89). The correlation coefficient between the two methods was 0.81 (P<0.0001).

CONCLUSIONS

The dual isotope method may be replaced by using a constant (k) and a single oral dose of isotopic-enriched Zn to estimate fractional absorption of Zn within a population.

SPONSORSHIP

Gerber Products Company, Fremont, MI.

Uncertainty in estimating blood ethanol concentrations by analysis of vitreous humour

AIMS

To determine the concentrations of ethanol in femoral venous blood (FVB) and vitreous humour (VH) obtained during forensic necropsies. The ratios of ethanol concentrations in VH and FVB, the reference interval, and the associated confidence limits were calculated to provide information about the uncertainty in estimating FVB ethanol concentrations indirectly from that measured in VH.

METHODS

Ethanol concentrations were determined in specimens of FVB and VH obtained from 706 forensic necropsies. The specimens were analysed in duplicate by headspace gas chromatography (HS-GC), with a precision (coefficient of variation) of 1.5% at a mean ethanol concentration of 500 mg/litre. The limit of detection of ethanol in body fluids by HS-GC in routine casework was 100 mg/litre.

RESULTS

In 34 instances, ethanol was present in VH at a mean concentration of 154 mg/litre, whereas the FVB ethanol concentration was reported as negative (< 100 mg/litre). These cases were excluded from the statistical analysis. The concentration of ethanol in FVB was higher than in VH in 93 instances, with a mean difference of 160 mg/litre (range 0 to 900). The mean concentration of ethanol in FVB (n = 672) was 1340 mg/litre (SD, 990) compared with 1580 mg/litre (SD, 1190) in VH. The arithmetic mean VH/FVB ratio of ethanol was 1.19 (SD, 0.285) and the 95% range was 0.63 to 1.75. The mean and SD of the differences (log VH - log FVB) was 0.063 (SD, 0.109), which gives 95% limits of agreement (LOA) from -0.149 to 0.276. Transforming back to the original scale of measurement gives a geometric mean VH/FVB ratio of 1.16 and 95% LOA from 0.71 to 1.89. These parametric estimates are in good agreement, with a median VH/FVB ratio of 1.18 and 2.5th and 97.5th centiles of 0.63 and 1.92.

CONCLUSIONS

The ethanol distribution ratios (VH/FVB) show wide variation and this calls for caution when results of analysing VH at necropsy are used to estimate the concentration in FVB. Dividing the ethanol concentration in VH by 2.0 would provide a very conservative estimate of the ethanol content in FVB, being less than the true value, with a high degree of confidence.

Measurement of body fat using leg to leg bioimpedance

AIMS

(1) To validate a leg to leg bioimpedance analysis (BIA) device in the measurement of body composition in children by assessment of its agreement with dual energy x ray absorptiometry (DXA) and its repeatability. (2) To establish a reference range of percentage body fat in Hong Kong Chinese children.

METHODS

Sequential BIA and DXA methods were used to determine body composition in 49 children aged 7-18 years; agreement between the two methods was calculated. Repeatability for the BIA method was established from duplicate measurements. Body composition was then determined by BIA in 1139 girls and 1243 boys aged 7-16 years, who were randomly sampled in eight local primary and secondary schools to establish reference ranges.

RESULTS

The 95% limits of agreement between BIA and DXA methods were considered acceptable (-3.3 kg to -0.5 kg fat mass and -3.9 to 0.6% body fat). The percentage body fat increased with increasing age. Compared to the 1993 Hong Kong growth survey, these children had higher body mass index. Mean (SD) percentage body fat at 7 years of age was 17.2% (4.4%) and 14.0% (3.4%) respectively for boys and girls, which increased to 19.3% (4.8%) and 27.8% (6.3%) at age 16.

CONCLUSION

Leg to leg BIA is a valid alternative method to DXA for the measurement of body fat. Provisional reference ranges for percentage body fat for Hong Kong Chinese children aged 7-16 years are provided.

ADHD in Swedish 3- to 7-year-old children

OBJECTIVE

To study characteristics of attention-deficit/hyperactivity disorder (ADHD) in a representative group of clinically impaired young children in Sweden with the disorder.

METHOD

One hundred thirty-one children with ADHD (aged 3-7 years) were examined, and their parents were interviewed. Independent parent questionnaire data (Child Behavior Checklist, ADHD Rating Scale-IV, Conners) were collected. For comparison 131 children without ADHD were matched for age, gender, parents' marital status, child's adoption status, and social class.

RESULTS

Children with ADHD had extremely high ADHD symptom levels--on average four to eight times higher than the comparison group. Sociodemographic correlates of ADHD symptoms were more pronounced in parent questionnaire data than in parent interview data, underscoring the importance of diagnostic interview when dealing with clinical issues. Very few of the children with ADHD (6%) appeared "normal" with regard to attention/activity level at clinical examination.

CONCLUSIONS

Clinic children with a diagnosis of DSM-IV ADHD have typical and impairing symptoms already before starting school. The variance of ADHD in this age group appears to be accounted for by primary psychosocial factors only to a limited degree. It would seem reasonable to establish supportive and treatment measures for these young children so that the psychosocial and academic problems shown by so many individuals with ADHD later in their development might be reduced.

Fusion Proteins for Combined Analysis of Autoantibodies to the 65-kDa Isoform of Glutamic Acid Decarboxylase and Islet Antigen-2 in Insulin-dependent Diabetes Mellitus

Background: Prediction, risk assessment, and diagnosis of autoimmune diseases often rely on detection of autoantibodies directed to multiple target antigens, such as the 65-kDa isoform of glutamic acid decarboxylase (GAD65-abs) and the tyrosine phosphatase-like protein islet antigen-2 (IA2-abs), the two major subspecificities of islet cell antibodies (ICAs) associated with insulin-dependent diabetes mellitus. We hypothesized that a combination of autoantigens in a fusion protein unifying the important immunodominant epitopes could provide an efficient target for cost-effective, one-step screening of sera.

Methods: Chimeric proteins composed of GAD65 and IA2 residues were constructed, analyzed for their immune reactivity with monoclonal antibodies and sera, and used in a diagnostic assay with 35S-labeled protein as antigen.

Results: Length and order of GAD65 and IA2 sequences were critical for conservation of the conformational epitopes in the fusion protein. Among four chimera tested, only IA2(606–979)/GAD65(1–585) retained wild-type-like folding of GAD65 and IA2 domains and yielded a stable protein after baculovirus expression. Reactivity of GAD65 antibody- and IA2 antibody-positive sera from patients newly diagnosed with insulin-dependent diabetes mellitus, from ICA-positive prediabetics, and from ICA-positive first-degree relatives demonstrated conservation of the relevant autoreactive epitopes. The assay based on the in vitro translated fusion antigen had a sensitivity and specificity identical to those for detection of GAD65- and IA2-abs based on the two separate GAD65 and IA2 proteins.

Conclusions: Autoantigens such as GAD65 and IA2 can be combined successfully in a fusion protein of similar immune reactivity. This allows simultaneous detection of GAD65- and IA2-abs in a one-step screening assay and cost-effective identification of positive individuals at risk of diabetes or at onset of disease.

Computer algorithms for the automated measurement of retinal arteriolar diameters

AIMS

Quantification of retinal vascular change is difficult and manual measurements of vascular features are slow and subject to observer bias. These problems may be overcome using computer algorithms. Three automated methods and a manual method for measurement of arteriolar diameters from digitised red-free retinal photographs were compared.

METHODS

60 diameters (in pixels) measured by manual identification of vessel edges in red-free retinal images were compared with diameters measured by (1) fitting vessel intensity profiles to a double Gaussian function by non-linear regression, (2) a standard edge detection algorithm (Sobel), and (3) determination of points of maximum intensity variation by a sliding linear regression filter (SLRF). Method agreement was analysed using Bland-Altman plots and the repeatability of each method was assessed.

RESULTS

Diameter estimations obtained using the SLRF method were the least scattered although diameters obtained were approximately 3 pixels greater than those measured manually. The SLRF method was the most repeatable and the Gaussian method less so. The Sobel method was the least consistent owing to frequent misinterpretation of the light reflex as the vessel edge.

CONCLUSION

Of the three automated methods compared, the SLRF method was the most consistent (defined as the method producing diameter estimations with the least scatter) and the most repeatable in measurements of retinal arteriolar diameter. Application of automated methods of retinal vascular analysis may be useful in the assessment of cardiovascular and other diseases.

Typical error versus limits of agreement

We have shown that the SEM (typical error) and LOA are very similar when defined at the same level of abstraction. The calculation of these sample statistics does not depend on sample size, but the precision of their estimate for the population parameter does. Only the latter concept involves the t-statistic. They do differ in the type of measurement error that is described (true score error versus test-retest error) and the coverage probability of the reference interval (0.6 versus 0.95). Bland and Altman and Atdinson and Nevill have promoted the citation of either the SEM or the LOA to help researchers in their discussion of the impact of error to real uses of the measurement tol. What is vital in this discussion is the researcher having a thorough understanding of the underlying theory behind the measurement error statistic(s) that is/are employed, especially the definition of error and the coverage probability that is selected. Such issues have been built into the title of the 95% LOA statistic, and are also an inherent part of SEM. Whilst the concept of typical error appears to be easy to understand and teach, this is only because the underlying theory and definition of what the statistic actually represents is not communicated. Only if all researchers adopt one single statistic (e.g. typical error) for measurement error is it remotely possible to push underlying theory into the background, since there would be a baseline of comparison for all. Because this scenario is highly unlikely, it is important that any measurement error statistic is well defined and understood by researchers.

Modelling method comparison data

We explore a range of linear regression models that might be useful for either: (a) the relative calibration of two or more methods or (b) to evaluate their precisions relative to each other. Ideally, one should be able to use a single data set to carry out the jobs (a) and (b) together. Throughout this review we consider the constraints (assumptions) needed to attain identifiability of the models and the possible pitfalls to the unwary in having to introduce them. We also pay particular attention to the possible problems arising from the presence of random matrix effects (reproducible random measurement 'errors' that are characteristic of a given method when being used on a given specimen or sample, i.e. specimen specific biases or subject by method interactions). Finally, we stress the importance of a fully-informative design (using replicate measurements on each subject using at least three independent methods) and large sample sizes.

The interpretation of diagnostic tests

Laboratory diagnostic tests are central in the practice of modern medicine. Common uses include screening a specific population for evidence of disease and confirming or ruling out a tentative diagnosis in an individual patient. The interpretation of a diagnostic test result depends on both the ability of the test to distinguish diseased from nondiseased subjects and the particular characteristics of the patient and setting in which the test is being used. This article reviews statistical methodology for assessing laboratory diagnostic test accuracy and interpreting individual test results, with an emphasis on diagnostic tests that yield a continuous measurement. The article begins with a summary of basic concepts and terminology, then briefly discusses study design and reviews methods for assessing the accuracy of a single diagnostic test, comparing the accuracy of two or more diagnostic tests and interpreting individual test results.