Median of patient results as a tool for assessment of analytical stability

Patient result monitoring of HbA1c shows small seasonal variations and steady decrease over more than 10 years

OBJECTIVES

Internal and external quality assurance materials often use highly processed matrixes. This can render the materials non-commutable. Monitoring laboratory methods with patient medians helps in identifying and correcting systematic errors that may affect diagnostic accuracy. The aim of the present study was to use HbA1c patient results for monitoring of method performance over time.

METHODS

Test HbA1c results from 2010 to 2022 was analyzed (n=722,553) regarding changes over time and seasonal variation. The HbA1c testing was initially performed on a Cobas 501 instrument using immunological detection but in May 2017 the method was replaced by capillary electrophoresis on Capillarys 3 Tera.

RESULTS

There was a steady decrease in HbA1c values. From 2011 to 2021 the decrease was for 0.10 percentile 6.6 %, lower quartile 7.9 %, median 10.2 %, mean values 9 %, upper quartile 11.2 %, and 0.90 percentile 9.3 %. No clear shift in HbA1c levels was observed due to the shift in methods. The median HbA1c values per month was approximately 44 mmol/mol (6.2 %, DCCT/NGSP). The only month with a median HbA1c that differed by more than 1 mmol/mol was July with a median value of 42 mmol/mol (6.0 %).

CONCLUSIONS

The patient data showed a similar decrease as in the National Diabetes Register which indicates that the method is stable over time without any sudden changes and that the seasonal variation is low. The continuous decrease in HbA1c values over time is most likely to a shift towards earlier detection of patient with diabetes and improved treatment.

The effect of ratios upon improving patient-based real-time quality control (PBRTQC) performance

OBJECTIVES

Large biological variation hinders application of patient-based real-time quality control (PBRTQC). The effect of analyte ratios on the ability of PBRTQC to improve error detection was investigated.

METHODS

Four single analyte-ratio pairs (alanine aminotransferase [ALT] vs. ALT to aspartate aminotransferase ratio [RALT]; creatinine [Cr] vs. Cr to cystatin C ratio [RCr]; lactate dehydrogenase [LDH] vs. LDH to hydroxybutyrate dehydrogenase ratio [RLDH]; total bilirubin [TB] vs. TB to direct bilirubin ratio [RTB]) were chosen for comparison. Various procedures, including four conventional algorithms (moving average [MA], moving median [MM], exponentially weighted moving average [EWMA] and moving standard deviation [MSD]) were assessed. A new algorithm that monitors the number of defect reports per analytical run (NDR) was also evaluated.

RESULTS

When a single analyte and calculated ratio used the same PBRTQC parameters, fewer samples were needed to detect systematic errors (SE) by taking ratios (p<0.05). Application of ratios in MA, MM and EWMA significantly enhanced their ability to detect SE. The influence of ratio on random error (RE) detection depended upon the analytes and PBRTQC parameters, as consistent advantage was not demonstrated. The NDR method performed well when appropriate parameters were used, but was only effective for unilateral SE. Rearrangement of sample order led to a significant deterioration of conventional algorithms' performance, while NDR remained almost unaffected.

CONCLUSIONS

For analytes with large variation and poor PBRTQC performance, using ratios as PBRTQC indexes may significantly improve performance and achieve better anti-interference ability, providing a new class of monitoring indicators for PBRTQC.

Development and Evaluation of "The Delta Plus-Minus Even Distribution Check": A Novel Patient-Based Real-Time Quality Control Method for Laboratory Tests

BACKGROUND

Laboratory testing of large sample numbers necessitates high-volume rapid processing, and these test results require immediate validation and a high level of quality assurance. Therefore, real-time quality control including delta checking is an important issue. Delta checking is a process of identifying errors in individual patient results by reviewing differences from previous results of the same patient (Δ value). Under stable analytical conditions, Δ values are equally positively and negatively distributed.

METHODS

The previous 20 Δ values from 3 tests (cholesterol, albumin, and urea nitrogen) were analyzed by calculating the R-value: "the positive Δ value ratio minus 0.5." This method of monitoring optimized R-values is referred to as the even-check method (ECM) and was compared with quality control (QC) testing in terms of error detection.

RESULTS

Bias was observed on 4 of the 120 days for the 3 analytes measured. When QC detected errors, the ECM captured the same systematic errors and more rapidly. In contrast, the ECM did not generate an alarm for the one random error that occurred in QC. While QC did not detect any errors, the percentage of R-values exceeding the acceptable range was under 2%, the number of days generating alarms was between 16 and 21 days, with short alarm periods, and a median number of samples per alarm period between 7 and 9 samples.

CONCLUSIONS

The ECM is a practical real-time QC method, controlled by setting R-value conditions, that quickly detects bias values.

Practical application of the patient data-based quality control method: the potassium example

Introduction

Internal quality control (IQC) is a core pillar of laboratory quality control strategies. Internal quality control commercial materials lack the same characteristics as patient samples and IQC contributes to the costs of laboratory testing. Patient data-based quality control (PDB-QC) may be a valuable supplement to IQC; the smaller the biological variation, the stronger the ability to detect errors. Using the potassium concentration in serum as an example study compared error detection effectiveness between PDB-QC and IQC.

Materials and methods

Serum potassium concentrations were measured by using an indirect ion-selective electrode method. For the training database, 23,772 patient-generated data and 366 IQC data from April 2022 to September 2022 were used; 15,351 patient-generated data and 246 IQC data from October 2022 to January 2023 were used as the testing database. For both PDB-QC and IQC, average values and standard deviations were calculated, and z-score charts were plotted for comparison purposes.

Results

Five systematic and three random errors were detected using IQC. Nine systematic errors but no random errors were detected in PDB-QC. The PDB-QC showed systematic error warnings earlier than the IQC.

Conclusions

The daily average value of patient-generated data was superior to IQC in terms of the efficiency and timeliness of detecting systematic errors but inferior to IQC in detecting random errors.

Analyte stability in whole blood using experimental and datamining approaches

The analytical stability of laboratory tests relies mostly on internal and external quality control procedures. Summarized patient data has in several studies been shown to be a good supplement for monitoring analytical stability. In our present investigation, we evaluate a datamining method for retrospective evaluation and assessment of analyte stability in whole blood. Results from the laboratory information system were used as the basis for the datamining approach. Blood tests were requested by the general practitioners and drawing of the blood sample was either at the general practitioner's or at the hospital outpatient clinics. We were able to split data into groups based on sample collection place and time to analysis. The datamining approach was compared to experiments where samples were incubated at a single temperature as well as an experiment where the temperatures were changed during incubation. To demonstrate the method, we selected three laboratory tests considered representative: potassium, phosphate, and lactate dehydrogenase. The datamining approach showed results similar to the reference experiment. Furthermore, our results show that the analytes phosphate and potassium were not stable after short storage at a lower temperature.

Comparison and optimization of various moving patient-based real-time quality control procedures for serum sodium

Background

Patient‐based real‐time quality control (PBRTQC) is a valuable tool for monitoring the performance of testing processes. We aimed to compare and optimize various PBRTQC procedures for serum sodium.

Methods

In a computer simulation, artificial errors were added to 680,000 real patients’ results. The characteristics of error detection of various algorithms—moving average, moving median, moving SD and moving proportion of normal results including different control limits (CLs)—were assessed on their ability to detect critical errors early.

Results

The moving average and moving median were sensitive to system error, and the moving SD tended to detect random error. P_3SD (moving proportion of normal results, CLs based on mean and SD of proportion of normal results) demonstrated excellent performance for both system error and random error. The increase of block sizes (N) leads to the delay of error detection and the decrease of false rejection, except for QC procedures with minimum and maximum as CLs. CLs calculation with “0.1% false alarm rate” had more effective performance than that set false alarm to zero (minimum and maximum as CLs). The impact of truncation on QC performance depended on truncation limits, algorithms and the types of error. The significant improvement in QC performance due to truncation was only found in moving SD.

Conclusion

“P_3SD,N = 50, without truncation” and “moving SD, N = 25, set 0.1% false alarm as CLs and set 1% outliers exclusion as truncation limits” were recommended as the optimized procedures for serum sodium to monitor system error and random error, respectively.

Moving average quality control: principles, practical application and future perspectives

Moving average quality control (MA QC) was described decades ago as an analytical quality control instrument. Although a potentially valuable tool, it is struggling to meet expectations due to its complexity and need for evidence-based guidance. For this review, relevant literature and the world wide web were examined in order to (i) explain the basic concepts and current understanding of MA QC, (ii) discuss moving average (MA) optimization methods, (iii) gain insight into practical aspects related to applying MA in daily practice and (iv) describe future prospects to enable more widespread acceptance and application of MA QC. Each of the MA QC optimization methods currently available has their own advantages and disadvantages. Recently developed simulation methods provide realistic error detecting properties for MA QC and are available for laboratories. Operational MA management issues have been identified that allow developers of MA software to upgrade their packages to support optimal MA QC application and guide laboratories on MA management issues, such as MA alarm workup. The new insights into MA QC characteristics and operational issues, together with supporting online tools, may promote more widespread acceptance and application of MA QC.

Plasma creatinine medians from patients partitioned by gender and age used as a tool for assessment of analytical stability at different concentrations

Background Monthly medians of patient results are useful in assessment of analytical quality in medical laboratories. Separate medians by gender makes it possible to generate two independent estimates of contemporaneous errors. However, for plasma creatinine, reference intervals (RIs) are different by gender and also higher over 70 years of age. Methods Daily, weekly and monthly patient medians were calculated from the raw data of plasma creatinine concentrations for males between 18 and 70 years, males >70 years, females between 18 and 70 years and females >70 years. Results The medians of the four groups were all closely associated, with similar patterns. The mean of percentage bias from each group defined the best estimate of bias. The maximum half-range (%) of the bias evaluations provided an estimate of the uncertainty comparable to the analytical performance specifications: thus, bias estimates could be classified as optimum, desirable or minimum quality. Conclusions Medians by gender and age are useful in assessment of analytical stability for plasma creatinine concentration ranging from 60 to 90 μmol/L. The daily medians are valuable in rapid detection of large systematic errors, the weekly medians in detecting minor systematic errors and monthly medians in assessment of long-term analytical stability.

Failure of internal quality control in detecting significant reagent lot shift in serum creatinine measurement

Background

Internal quality control (IQC) in clinical laboratories is carried out to monitor analytical stability. Usually, the satisfactory results of the IQC ensure the acceptability of the examination results. Here, we reported that patients' creatinine results are unreliable, although the internal quality control is satisfactory.

Methods

Creatinine levels were analyzed from two quality control materials and twenty patients' specimens using two different lots of reagents. Lot‐to‐lot comparison was performed. The daily median values of serum creatinine levels of patients were calculated from the test results recorded in our laboratory information system.

Results

Although IQC was consistent, serum creatinine concentrations were higher using lot B (median: 153 μmol/L; interquartile range: 122‐522 μmol/L) than using lot A (median: 133 μmol/L; interquartile range: 76‐508 μmol/L) for 20 patients (P = .001). The Deming linear regression showed a best fit of y = 0.9394 × x + 45.66. R² = .8919, and mean percentage difference between two lots was 34%. The new lot was considered unacceptable. Likewise, the median serum creatinine level from the 360 patients using lot B was 102 μmol/L, which was significantly higher than the daily medians of patients using lot A (median: 66 μmol/L; range: 61‐70 μmol/L) in the previous month.

Conclusion

The variations in creatinine concentrations proved to be due to different lots of reagents. However, IQC materials tested using both lots of reagent exhibited minimal variation. Therefore, IQC alone is insufficient for assessing laboratory analytical results. This finding prompts us to be vigilant in potential pitfall of interpreting test results based on satisfactory IQC alone.

Predicting the Reasons of Customer Complaints: A First Step Toward Anticipating Quality Issues of In Vitro Diagnostics Assays with Machine Learning

Background

Vendors in the health care industry produce diagnostic systems that, through a secured connection, allow them to monitor performance almost in real time. However, challenges exist in analyzing and interpreting large volumes of noisy quality control (QC) data. As a result, some QC shifts may not be detected early enough by the vendor, but lead a customer to complain.

Objective

The aim of this study was to hypothesize that a more proactive response could be designed by utilizing the collected QC data more efficiently. Our aim is therefore to help prevent customer complaints by predicting them based on the QC data collected by in vitro diagnostic systems.

Methods

QC data from five select in vitro diagnostic assays were combined with the corresponding database of customer complaints over a period of 90 days. A subset of these data over the last 45 days was also analyzed to assess how the length of the training period affects predictions. We defined a set of features used to train two classifiers, one based on decision trees and the other based on adaptive boosting, and assessed model performance by cross-validation.

Results

The cross-validations showed classification error rates close to zero for some assays with adaptive boosting when predicting the potential cause of customer complaints. Performance was improved by shortening the training period when the volume of complaints increased. Denoising filters that reduced the number of categories to predict further improved performance, as their application simplified the prediction problem.

Conclusions

This novel approach to predicting customer complaints based on QC data may allow the diagnostic industry, the expected end user of our approach, to proactively identify potential product quality issues and fix these before receiving customer complaints. This represents a new step in the direction of using big data toward product quality improvement.

Gender-partitioned patient medians of serum albumin requested by general practitioners for the assessment of analytical stability

BACKGROUND

Recently, the use of separate gender-partitioned patient medians of serum sodium has revealed potential for monitoring analytical stability within the optimum analytical performance specifications for laboratory medicine. The serum albumin concentration depends on whether a patient is sitting or recumbent during phlebotomy. We therefore investigated only examinations requested by general practitioners (GPs) to provide data from sitting patients.

METHODS

Weekly and monthly patient medians of serum albumin requested by GP for both male and female patients were calculated from the raw data obtained from three analysers in the hospital laboratory on examination of samples from those >18 years. The half-range of medians were applied as an estimate of the maximum bias. Further, the ratios between the two medians were calculated (females/males).

RESULTS

The medians for male and female patients were closely related despite considerable variation due to the current analytical variation. This relationship was confirmed by the calculated half-range for the monthly ratio between the genders of 0.44%, which surpasses the optimum analytical performance specification for bias of serum albumin (0.72%). The weekly ratio had a half-range of 1.83%, which surpasses the minimum analytical performance specifications of 2.15%.

CONCLUSIONS

Monthly gender-partitioned patient medians of serum albumin are useful for monitoring of long-term analytical stability, where the gender medians are two independent estimates of changes in (delta) bias: only results requested by GP are of value in this application to ensure that all patients are sitting during phlebotomy.

Separate patient serum sodium medians from males and females provide independent information on analytical bias

BACKGROUND

During monitoring of monthly medians of results from patients undertaken to assess analytical stability in routine laboratory performance, the medians for serum sodium for male and female patients were found to be significantly related.

METHODS

Daily, weekly and monthly patient medians of serum sodium for both male and female patients were calculated from results obtained on samples from the population >18 years on three analysers in the hospital laboratory. The half-range of medians was applied as an estimate of the maximum bias. Further, the ratios between the two medians were calculated.

RESULTS

The medians of both genders demonstrated dispersions over time, but they were closely connected in like patterns, which were confirmed by the half-range of the ratios of medians for males and females that varied from 0.36% for daily, 0.14% for weekly and 0.036% for monthly ratios over all instruments.

CONCLUSIONS

The tight relationship between the gender medians for serum sodium is only possible when raw laboratory data are used for calculation. The two patient medians can be used to confirm both and are useful as independent estimates of analytical bias during constant calibration periods. In contrast to the gender combined median, the estimate of analytical bias can be confirmed further by calculation of the ratios of medians for males and females.

Stronger Together: Aggregated Z-values of Traditional Quality Control Measurements and Patient Medians Improve Detection of Biases

BACKGROUND

In clinical chemistry, quality control (QC) often relies on measurements of control samples, but limitations, such as a lack of commutability, compromise the ability of such measurements to detect out-of-control situations. Medians of patient results have also been used for QC purposes, but it may be difficult to distinguish changes observed in the patient population from analytical errors. This study aims to combine traditional control measurements and patient medians for facilitating detection of biases.

METHODS

The software package “rSimLab” was developed to simulate measurements of 5 analytes. Internal QC measurements and patient medians were assessed for detecting impermissible biases. Various control rules combined these parameters. A Westgard-like algorithm was evaluated and new rules that aggregate Z-values of QC parameters were proposed.

RESULTS

Mathematical approximations estimated the required sample size for calculating meaningful patient medians. The appropriate number was highly dependent on the ratio of the spread of sample values to their center. Instead of applying a threshold to each QC parameter separately like the Westgard algorithm, the proposed aggregation of Z-values averaged these parameters. This behavior was found beneficial, as a bias could affect QC parameters unequally, resulting in differences between their Z-transformed values. In our simulations, control rules tended to outperform the simple QC parameters they combined. The inclusion of patient medians substantially improved bias detection for some analytes.

CONCLUSIONS

Patient result medians can supplement traditional QC, and aggregations of Z-values are novel and beneficial tools for QC strategies to detect biases.