Clinical decision-making as the basis for assessing agreement between measures of the International Normalized Ratio

Measurement of serum anti-Müllerian hormone by revised Gen II or automated assay: Reproducibility under various blood/serum storage conditions

OBJECTIVE

We investigated the agreement between anti-Müllerian hormone (AMH) levels measured with revised Gen II (rev-Gen II) and automated AMH (Access) assays and evaluated the reproducibility of each method under various blood/serum storage conditions.

METHODS

AMH levels in blood samples from 74 volunteers were measured by rev-Gen II and Access assays under various conditions: immediate serum separation and AMH measurement (fresh control); serum stored at -20 °C and AMH measured after 48 hours, 1 week, and 2 years; serum stored at 0 to 4 °C and AMH measured after 48 hours and 1 week; and blood kept at room temperature and delayed serum separation after 48 hours and 1 week, with immediate AMH measurement.

RESULTS

In fresh controls, all rev-Gen II-AMH values were higher than comparable Access-AMH values (difference, 8.3% to 19.7%). AMH levels measured with the two methods were strongly correlated for all sample conditions (r=0.977 to 0.995, all p<0.001). For sera stored at -20 °C or 0 to 4 °C for 48 hours, Access-AMH values were comparable to control measurements, but rev-Gen II-AMH values were significantly lower. AMH levels in sera stored at -20 °C or 0 to 4 °C for 1 week were significantly lower than in fresh controls, irrespective of method. Across methods, long-term storage at -20 °C for 2 years yielded AMH measurements significantly higher than control values. When serum separation was delayed, rev-Gen II-AMH values were significantly lower than control measurements, but Access-AMH values varied.

CONCLUSION

The rev-Gen II and Access-AMH assays showed varying reproducibility across blood/serum storage conditions, but automated Access yielded superior stability to rev-Gen II.

Predicting vertical ground reaction force in rearfoot running: A wavelet neural network model and factor loading

This study proposed a simple method for selecting input variables by factor loading and inputting these variables into a wavelet neural network (WNN) model to predict vertical ground reaction force (vGRF). The kinematic data and vGRF of 9 rearfoot strikers at 12, 14, and 16 km/h were collected using a motion capture system and an instrumented treadmill. The input variables were screened by factor loading and utilized to predict vGRF with the WNN. Nine kinematic variables were selected, corresponding to nine principal components, mainly focusing on the knee and ankle joints. The prediction results of vGRF were effective and accurate at different speeds, namely, the coefficient of multiple correlation (CMC) > 0.98 (0.984-0.988), the normalized root means square error (NRMSE) < 15% (9.34-11.51%). The NRMSEs of impact force (8.18-10.01%), active force (4.92-7.42%), and peak time (7.16-12.52%) were less than 15%. There was a small number (peak, 4.12-6.18%; time, 4.71-6.76%) exceeding the 95% confidence interval (CI) using the Bland-Altman method. The knee joint was the optimal location for estimating vGRF, followed by the ankle. There were high accuracy and agreement for predicting vGRF with the peak and peak time at 12, 14, and 16 km/h. Therefore, factor loading could be a valid method to screen kinematic variables in artificial neural networks.

Agreement between Coaguchek XS and STA-R Evolution (Hepato Quick) INR results depends on the level of INR

INTRODUCTION

Introducing point-of-care (POC) INR measurement to monitor anticoagulant therapy may be beneficial for both patients and anticoagulation clinics. However, agreement between POC and laboratory INR results is still unclear, especially at sub- and supratherapeutic levels. Therefore we investigated the analytical and clinical agreement between POC INR results of the Coaguchek XS and laboratory INR results of the STA-R Evolution.

MATERIALS AND METHODS

Paired POC and laboratory INR results were obtained and analyzed in 3257 patients aged 18-104 years between August 2008 and March 2014.

RESULTS

Mean difference between POC and laboratory results ranged from -0.18 (95%CI -0.20;-0.16) INR point for POC results 2.0-3.0, up to 1.14 (95%CI 0.87;1.42) INR point for POC results 7.1-8.0. In the therapeutic range (POC INR 2.0-4.0), mean difference between POC and laboratory results was -0.13 (95%CI -0.15;-0.12) INR point. At subtherapeutic (POC INR <2.0) and supratherapeutic (POC INR >4.0) INR levels, mean differences were -0.13 (95%CI -0.15;-0.11) and 0.72 (95%CI 0.63;0.80) INR point, respectively. Clinical agreement regarding therapeutic range was present in 92.0% (POC within range), 67.7% (POC below range) and 87.6% (POC above range) of the paired measurements. We observed ≥15% INR difference between the POC and laboratory result in 14.8% (POC INR 2.0-4.0), 17.0% (POC INR<2.0) and 47.8% (POC INR >4.0) of the paired measurements.

CONCLUSIONS

POC and laboratory INR results were strongly correlated within the therapeutic range and differences between results become larger with increasing INR. Clinical disagreement between laboratory and POC results occurs often at both sub- and supratherapeutic INR levels.

Transiently increased variation between a Point-of-Care and laboratory INR method after a long period of correlation: a case study demonstrating the need for ongoing correlation of POC with the central laboratory

OBJECTIVES

To perform long-term comparison between laboratory Stago and Point-of-Care (POC) i-STAT methods for determining the international normalized ratio (INR).

METHODS

This was a multicenter method comparison of patient INR results and factors related to performance variance.

RESULTS

For 5 years, the assays demonstrated close patient correlation within and above the 3.5 INR therapeutic range cutoff (bias, 0.23 INR units). Patient results above 3.5 INR were bimodal, with 60% demonstrating an i-STAT INR bias of less than 0.5. Several patient conditions were associated with the presence of a higher i-STAT bias. In year 6, a broader range i-STAT bias developed, increasing to 0.73 INR units. The increased bias persisted for 3 years, then returned to initial levels following i-STAT adjustments. The substantial increase in i-STAT bias after a long period of stability was partly corrected by renewed correlation to the international reference preparation. Additional assay drift is discussed in relation to thromboplastin reagents and other testing variables.

CONCLUSIONS

This study emphasizes the need for continual laboratory correlation with POC devices and caution in using published comparisons.

A comparison of coagulation study results between heparinized peripherally inserted central catheters and venipunctures

PURPOSE/OBJECTIVE

The purpose of this study was to test an evidence-based procedure of drawing blood samples for coagulation testing from heparinized peripherally inserted central catheter (PICC) by comparing results with blood drawn from venipuncture (VP).

DESIGN

A prospective, quasi-experimental design using purposive sampling was used.

SETTING

The setting was a 230-bed community hospital located in the Southwest. The hospital is part of a 15-hospital system.

SAMPLE

: The sample was composed of 30 hospitalized patients with heparinized PICCs.

METHODS

Informed consent was obtained. Using aseptic technique, samples of blood were drawn via VP and from the PICC using the evidence-based procedure. Data were analyzed using Pearson product moment correlations and Bland-Altman analysis.

FINDINGS

For 5 coagulation tests studied, correlations between PICC values and VP values ranged from 0.990 to 0.998, indicating almost perfect correlations. In Bland-Altman analyses, mean biases and SDs were small to moderate for prothrombin time, 0.13 seconds (-0.55 to 0.81 seconds) (P = 0.0484); international normalized ratio, 0.010 (-0.050 to 0.070) (P = 0.085); partial thromboplastin time, 2.16 seconds (-5.10 to 9.43 seconds) (P = 0.0033); and fibrinogen, -18.2 mg (-70.4 to 34.1 mg) (P = 0.0033) and 0.52 (-0.73 to 1.77) seconds (P = 0.0003). Correlations of absolute difference versus average ranged from 0.18 to 0.49. Only the paired international normalized ratio samples had P values suggesting nonagreement.

CONCLUSIONS

Drawing blood samples from heparinized PICCs for coagulation tests using the evidence-based procedure developed for this study resulted in accurate coagulation test results in 4 of the 5 tests: prothrombin time, partial thromboplastin time, and fibrinogen in seconds and in milligrams.

Comparative performance of two methods that assess the quality of international normalized ratio measures

OBJECTIVES

Two methods, Petersen's error grid analysis and Shermock's method to detect clinically important differences, were recently developed to advance the assessment of analytic performance of point-of-care INR devices. Both methods predict when alternate INR measures lead to different clinical decisions. Our goal was to compare their performance characteristics.

DESIGN AND METHODS

Performance characteristics were assessed by comparing the models' predictions to clinical decisions that were directly measured in a previous experiment.

RESULTS

Shermock's method (82% of predictions correct) demonstrated superior predictive performance compared with the error grid analysis (75% of predictions correct, p=0.008). Shermock's method was particularly superior at identifying the clinical decisions that actually disagreed (79% for Shermock's method vs. 47% for error grid). Consequently, Shermock's method was superior at identifying a POC device with poor performance (79% accuracy vs. 70%, p=0.006).

CONCLUSION

Shermock's method had superior performance characteristics and should be integrated into analytic strategies to assess POC INR devices.

Novel analysis of clinically relevant diagnostic errors in point-of-care devices

BACKGROUND

To ensure proper clinical decision-making and avoid preventable harm, the quality of point-of-care (POC) device measures is routinely assessed. Traditional analyses may not reveal clinically important diagnostic errors.

OBJECTIVES

To compare results between a novel analytic framework and traditional analyses.

METHODS

Patients in four anticoagulation clinics provided two measures of the International Normalized Ratio (INR) at the same visit as part of routine quality assurance: one via a venous sample and one fingerstick. These were assessed with Hemochron POC devices. Traditional, quarterly, quality assurance assessments emphasized correlation analysis. The novel analysis used enhanced graphics and a validated assessment of clinical decision-making.

RESULTS

1518 paired INRs were analyzed. The correlation between the POC and laboratory assessments ranged between 0.84 and 0.91. Traditional quality assurance showed that the Hemochron devices were acceptable for continued use in each quarterly analysis. Enhanced graphical analysis demonstrated that the Hemochron devices never reported seven common INR values. The Hemochron devices systematically inflated values < 3 and deflated values > 4, biasing results towards the target INR range. Consequently, the Hemochron devices lead to a different clinical decision than the clinical laboratory measure in 31% of cases (458/1466; 95% confidence interval [CI] 29-34). When the reference INR was low, the Hemochron devices would not result in appropriate dose increases in 52% of cases (95% CI 48-56), placing these patients at risk for a significant adverse drug event.

CONCLUSIONS

Our novel, clinically relevant analysis revealed previously undetected deficiencies in our POC INR devices, and our approach should be adopted by industry, regulators, and institutions.

Establishment of outcome-related analytic performance goals

BACKGROUND

Accrediting organizations require laboratories to establish analytic performance criteria that ensure their tests provide results of the high quality required for patient care. However, the procedures for instituting performance criteria that are directly linked to the needs of medical practice are not well established, and therefore alternative strategies often are used to create and implement surrogate performance standards.

CONTENT

We reviewed 6 approaches for establishing outcome-related analytic performance goals: (a) limits defined by regulations and external assessment programs, (b) limits based on biologic variation, (c) limits based on surveys of clinicians about their needs, (d) limits based on effects on guideline driven medical decisions, (e) limits based on analysis of patterns for ordering follow-up clinical tests, and (f) limits based on formal medical decision models. Performance criteria were tabulated for 12 common chemistry analytes and 4 routine hematology tests.

CONCLUSIONS

There is no consensus currently about the preferred methods for establishing medically necessary analytic performance limits. The various methods reviewed give considerably different performance limits. The analytic performance limits claimed by a laboratory should correspond to those limits that can be reliably maintained based on validated QC monitoring systems. These limits generally are larger than the observed CVs and bias parameters collected for assay validation. There is a major need for increased communication among laboratorians and clinicians on this topic, especially when the analytic performance limits that can be consistently maintained by a laboratory are inconsistent with the expectations of health care providers.