Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data

Lot-to-Lot Variance in Immunoassays-Causes, Consequences, and Solutions

Immunoassays, which have gained popularity in clinical practice and modern biomedical research, play an increasingly important role in quantifying various analytes in biological samples. Despite their high sensitivity and specificity, as well as their ability to analyze multiple samples in a single run, immunoassays are plagued by the problem of lot-to-lot variance (LTLV). LTLV negatively affects assay accuracy, precision, and specificity, leading to considerable uncertainty in reported results. Therefore, maintaining consistency in technical performance over time presents a challenge in reproducing immunoassays. In this article, we share our two-decade-long experience and delve into the reasons for and locations of LTLV, as well as explore methods to mitigate its effects. Our investigation identifies potential contributing factors, including quality fluctuation in critical raw materials and deviations in manufacturing processes. These findings offer valuable insights to developers and researchers working with immunoassays, emphasizing the importance of considering lot-to-lot variance in assay development and application.

Difference- and regression-based approaches for detection of bias

OBJECTIVE

This simulation study was undertaken to assess the statistical performance of six commonly used rejection criteria for bias detection.

METHODS

The false rejection rate (i.e. rejection in the absence of simulated bias) and the probability of bias detection were assessed for the following: difference in measurements for individual sample pair, the mean of the paired differences, t-statistics (paired t-test), slope < 0.9 or > 1.1, intercept > 50% of the lower limit of measurement range, and coefficient of determination (R²) > 0.95. The linear regressions evaluated were ordinary least squares, weighted least squares and Passing-Bablok regressions. A bias detection rate of < 50% and false rejection rates of >10% are considered unacceptable for the purpose of this study.

RESULTS

Rejection criteria based on regression slope, intercept and paired difference (10%) for individual samples have high false rejection rates and/ or low probability of bias detection. T-statistics (α = 0.05) performed best in low range ratio (lowest-to-highest concentration in measurement range) and low imprecision scenarios. Mean difference (10%) performed better in all other range ratio and imprecision scenarios. Combining mean difference and paired-t test improves the power of bias detection but carries higher false rejection rates.

CONCLUSIONS

This study provided objective evidence on commonly used rejection criteria to guide laboratory on the experimental design and statistical assessment for bias detection during method evaluation or reagent lot verification.

Lot-to-lot variation and verification

Lot-to-lot verification is an integral component for monitoring the long-term stability of a measurement procedure. The practice is challenged by the resource requirements as well as uncertainty surrounding experimental design and statistical analysis that is optimal for individual laboratories, although guidance is becoming increasingly available. Collaborative verification efforts as well as application of patient-based monitoring are likely to further improve identification of any differences in performance in a relatively timely manner. Appropriate follow up actions of failed lot-to-lot verification is required and must balance potential disruptions to clinical services provided by the laboratory. Manufacturers need to increase transparency surrounding release criteria and work closer with laboratory professionals to ensure acceptable reagent lots are released to end users. A tripartite collaboration between regulatory bodies, manufacturers, and laboratory medicine professional bodies is key to developing a balanced system where regulatory, manufacturing, and clinical requirements of laboratory testing are met, to minimize differences between reagent lots and ensure patient safety. Clinical Chemistry and Laboratory Medicine has served as a fertile platform for advancing the discussion and practice of lot-to-lot verification in the past 60 years and will continue to be an advocate of this important topic for many more years to come.

An approach for determining allowable between reagent lot variation

Clinicians trust medical laboratories to provide reliable results on which they rely for clinical decisions. Laboratories fulfil their responsibility for accurate and consistent results by utilizing an arsenal of approaches, ranging from validation and verification experiments to daily quality control procedures. All these procedures verify, on different moments, that the results of a certain examination procedure have analytical performance characteristics (APC) that meet analytical performance specifications (APS) set for a particular intended use. The APC can in part be determined by estimating the measurement uncertainty component under conditions of within-laboratory precision (uRw), which comprises all components influencing the measurement uncertainty of random sources. To maintain the adequacy of their measurement procedures, laboratories need to distinguish aspects that are manageable vs. those that are not. One of the aspects that may influence uRw is the momentary significant bias caused by shifts in reagent and/or calibrator lots, which, when accepted or unnoticed, become a factor of the APC. In this paper, we postulate a model for allocating a part of allowable uRw to between-reagent lot variation, based on the need for long-term consistency of the measurement variability for that specific measurand. The allocation manages the ratio between short-term and long-term variation and indicates laboratories when to reject or correct certain variations due to reagent lots.

A study of the moving rate of positive results for use in a patient-based real-time quality control program on a procalcitonin point-of-care testing analyzer

Objective

To establish an applicable and highly sensitive patient‐based real‐time quality control (PBRTQC) program based on a data model constructed with patients’ results of a procalcitonin point‐of‐care testing (POCT) analyzer.

Methods

Patients’ results were retrospectively collected within one year. The Excel software was used to establish quality control (QC) programs of the moving average (MA) and the moving rate of positive results (MR). A Monte Carlo simulation was used to introduce positive and negative biases between 0.01 and 1 ng/ml at random points of the testing data set. Different parameters were used to detect the biases, and the detection efficiency was expressed using the median number of patient samples affected until error detection (MNPed). After comparing the MNPeds of different programs, MA and MR programs with appropriate parameters were selected, and validation plots were generated using MNPeds and maximum number of the patient samples affected (MAX). β curves were generated using the power function of the programs, the performances were compared with that of the conventional QC program.

Results

Neither the conventional QC nor MA program was sensitive to small bias, While MR program can detect the minimum positive bias of 0.06 ng/ml and negative of 0.4 ng/ml at an average daily run size of 10 specimens, with FRs < 1.0%, βs < 1%.

Conclusion

The MR program, which is more sensitive to small biases than conventional QC and MA programs, with low FR and β. As such, it can be used as a PBRTQC program with high performance.

UPLC-ESI-MS/MS profiling of the underground parts of common Iris species in relation to their anti-virulence activities against Staphylococcusaureus

ETHNOPHARMACOLOGICAL RELEVANCE

The use of plant extracts and their phytochemicals as candidates for targeting the microbial resistance inhibition is increasingly focused in last decades. In Mongolian traditional medicine, Irises were long used for the treatment of bacterial infections. Irises have been used since the Ancient Egyptians.

AIM OF THE STUDY

Chemical composition and virulence inhibition potential of both polar (PF) and non-polar fractions (NPF) of three common Iris species (I. confusa, I. pseudacorus and I. germanica) were explored.

MATERIAL AND METHODS

Secondary metabolites profiling was characterized by the UPLC-HRMS/MS technique. Multi-variate data analysis was performed using Metaboanalyst 3.0. Anti-virulence inhibitory activity was evaluated via anti-haemolytic assay and Quantitative biofilm inhibition assay.

RESULTS

I. pseudacorus PF exhibited the most potent effect against S. aureus haemolytic activity. All the tested fractions from all species, except I. pseudacorus NPF, have no significant inhibition on the biofilm formation of methicillin resistant and sensitive (MRSA and MSSA) S. aureus. I. pseudacorus NPF showed potent biofilm inhibitory potential of 71.4 and 85.8% against biofilm formation of MRSA and MSSA, respectively. Metabolite profiling of the investigated species revealed ninety and forty-five metabolites detected in the PFs and NPFs, respectively. Nigricin-type, tectorigenin-type isoflavonids and xanthones allowed the discrimination of I. pseudacorus PF from the other species, highlighting the importance of those metabolites in exerting its promising activity. On the other hand, triterpene acids, iridals, triacylglycerols and ceramides represented the metabolites detected in highest abundance in I. pseudacorus NPF.

CONCLUSIONS

This is the sole map represents the secondary metabolites profiling of the PFs and NPFs of common Iris species correlating them with the potent explored Staphylococcus aureus anti-virulence activity.

Centralization of multisite reagent lot-to-lot validation for Ortho Clinical Vitros chemistry instruments

OBJECTIVE

Reagent lot-to-lot comparisons are recommended by accreditation bodies to ensure that the performance of each reagent lot meets acceptable standards for quality patient results. The general approach is comprised of performing quality control (QC) and patient comparison between the old and new reagent lots and evaluating against a pre-defined criteria. Reagent lot comparison practices are often variable despite using the same instrument across different laboratories. This is costly, time consuming, and can lead to variability in acceptance criteria. While Clinical & Laboratory Standards Institute (CLSI) has a recommended guideline for reagent lot validation, it is often difficult to execute for small and rural laboratories due to limited resources. Defining the analytes required for detailed validation is important to allocate appropriate resources to ensure quality patient results. The goal of this study was to develop a standardized approach to reagent lot validation and optimize lab resources on Vitros chemistry instruments.

DESIGN AND METHOD

This study consists of a retrospective and prospective analysis of reagent lot changes in dry slide chemistry analyzers (Ortho Clinical Diagnostics Vitros). Two years of retrospective reagent lot comparison data was obtained at a single site. A prospective study was conducted by assessing aliquots of 10 patient sample pools at 9 sites with Vitros analyzers.

RESULTS

Of the 19 chemistry analytes evaluated, albumin, sodium, and total protein showed significant differences between reagent lots and also exceeded the pre-defined acceptance criteria.

CONCLUSION

For these analytes, our recommendations are to perform a comprehensive lot validation with QC and patient samples. A simple lot validation with a reflex approach comprised of initially assaying QC can be adapted for the more stable analytes to allow achieving quality patient result in a resource constraint rural environment.