APS calculator: a data-driven tool for setting outcome-based analytical performance specifications for measurement uncertainty using specific clinical requirements and population data

OBJECTIVES

According to ISO 15189:2022, analytical performance specifications (APS) should relate to intended clinical use and impact on patient care. Therefore, we aimed to develop a web application for laboratory professionals to calculate APS based on a simulation of the impact of measurement uncertainty (MU) on the outcome using the chosen decision limits, agreement thresholds, and data of the population of interest.

METHODS

We developed the "APS Calculator" allowing users to upload and select data of concern, specify decision limits and agreement thresholds, and conduct simulations to determine APS for MU. The simulation involved categorizing original measurand concentrations, generating measured (simulated) results by introducing different degrees of MU, and recategorizing measured concentrations based on clinical decision limits and acceptable clinical misclassification rates. The agreements between original and simulated result categories were assessed, and values that met or exceeded user-specified agreement thresholds that set goals for the between-category agreement were considered acceptable. The application generates contour plots of agreement rates and corresponding MU values. We tested the application using National Health and Nutrition Examination Survey data, with decision limits from relevant guidelines.

RESULTS

We determined APS for MU of six measurands (blood total hemoglobin, plasma fasting glucose, serum total and high-density lipoprotein cholesterol, triglycerides, and total folate) to demonstrate the potential of the application to generate APS.

CONCLUSIONS

The developed data-driven web application offers a flexible tool for laboratory professionals to calculate APS for MU using their chosen decision limits and agreement thresholds, and the data of the population of interest.

ISO 15189 is a sufficient instrument to guarantee high-quality manufacture of laboratory developed tests for in-house-use conform requirements of the European In-Vitro-Diagnostics Regulation

The EU In-Vitro Diagnostic Device Regulation (IVDR) aims for transparent risk-and purpose-based validation of diagnostic devices, traceability of results to uniquely identified devices, and post-market surveillance. The IVDR regulates design, manufacture and putting into use of devices, but not medical services using these devices. In the absence of suitable commercial devices, the laboratory can resort to laboratory-developed tests (LDT) for in-house use. Documentary obligations (IVDR Art 5.5), the performance and safety specifications of ANNEX I, and development and manufacture under an ISO 15189-equivalent quality system apply. LDTs serve specific clinical needs, often for low volume niche applications, or correspond to the translational phase of new tests and treatments, often extremely relevant for patient care. As some commercial tests may disappear with the IVDR roll-out, many will require urgent LDT replacement. The workload will also depend on which modifications to commercial tests turns them into an LDT, and on how national legislators and competent authorities (CA) will handle new competences and responsibilities. We discuss appropriate interpretation of ISO 15189 to cover IVDR requirements. Selected cases illustrate LDT implementation covering medical needs with commensurate management of risk emanating from intended use and/or design of devices. Unintended collateral damage of the IVDR comprises loss of non-profitable niche applications, increases of costs and wasted resources, and migration of innovative research to more cost-efficient environments. Taking into account local specifics, the legislative framework should reduce the burden on and associated opportunity costs for the health care system, by making diligent use of existing frameworks.

Validation and verification of examination procedures in medical laboratories: opinion of the EFLM Working Group Accreditation and ISO/CEN standards (WG-A/ISO) on dealing with ISO 15189:2012 demands for method verification and validation

This paper reflects the opinion of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group Accreditation and ISO/CEN standards (WG-A/ISO). It aims to provide guidance for drawing up local/national documents about validation and verification of laboratory methods. We demonstrate how risk evaluation can be used to optimize laboratory policies to meet intended use requirements as well as requirements of standards. This is translated in a number of recommendations on how to introduce risk evaluation in various stages of the implementation of new methods ultimately covering the whole process cycle.

Documenting metrological traceability as intended by ISO 15189:2012: A consensus statement about the practice of the implementation and auditing of this norm element

ISO15189:2012 requires medical laboratories to document metrological traceability of their results. While the ISO17511:2003 standard on metrological traceability in laboratory medicine requires the use of the highest available level in the traceability chain, it recognizes that for many measurands there is no reference above the manufacturer’s selected measurement procedure and the manufacturer’s working calibrator. Some immunoassays, although they intend to measure the same quantity and may even refer to the same reference material, unfortunately produce different results because of differences in analytical selectivity as manufacturers select different epitopes and antibodies for the same analyte. In other cases, the cause is the use of reference materials, which are not commutable. The uncertainty associated with the result is another important aspect in metrological traceability implementation. As the measurement uncertainty on the clinical samples is influenced by the uncertainty of all steps higher in the traceability chain, laboratories should be provided with adequate and appropriate information on the uncertainty of the value assignment to the commercial calibrators that they use. Although the between-lot variation in value assignment will manifest itself as part of the long-term imprecision as estimated by the end-user, information on worst-case to be expected lot-lot variation has to be communicated to the end-user by the IVD provider. When laboratories use ancillary equipment that potentially could have a critical contribution to the reported results, such equipment needs verification of its proper calibration and criticality to the result uncertainty could be assessed by an approach based on risk analysis, which is a key element of ISO15189:2012 anyway. This paper discusses how the requirement for metrological traceability as stated in ISO15189 should be met by the medical laboratory and how this should be assessed by accreditation bodies.

Flexible scope for ISO 15189 accreditation: a guidance prepared by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group Accreditation and ISO/CEN standards (WG-A/ISO)

The recent revision of ISO15189 has further strengthened its position as the standard for accreditation for medical laboratories. Both for laboratories and their customers it is important that the scope of such accreditation is clear. Therefore the European co-operation for accreditation (EA) demands that the national bodies responsible for accreditation describe the scope of every laboratory accreditation in a way that leaves no room for doubt about the range of competence of the particular laboratories. According to EA recommendations scopes may be fixed, mentioning every single test that is part of the accreditation, or flexible, mentioning all combinations of medical field, examination type and materials for which the laboratory is competent. Up to now national accreditation bodies perpetuate use of fixed scopes, partly by inertia, partly out of fear that a too flexible scope may lead to over-valuation of the competence of laboratories, most countries only use fixed scopes. The EA however promotes use of flexible scopes, since this allows for more readily innovation, which contributes to quality in laboratory medicine. In this position paper, the Working Group Accreditation and ISO/CEN Standards belonging to the Quality and Regulation Committee of the EFLM recommends using an approach that has led to successful introduction of the flexible scope for ISO15189 accreditation as intended in EA-4/17 in The Netherlands. The approach is risk-based, discipline and competence-based, and focuses on defining a uniform terminology transferable across the borders of scientific disciplines, laboratories and countries.

Circadian and concentration profile of cathepsin S in sera from healthy subjects and asthmatic patients

Cathepsin S (CS) has been proposed to be associated with asthma pathogenesis but its exact role is not established. In order to understand this proposed association our objective was to follow the 24-h concentration pattern of CS in sera from apparently healthy subjects and from steroid-independent and steroid-dependent asthmatics before and after one weeks' treatment with methylprednisolone (MP) and cyclosporin A (CsA), respectively. Blood samples were collected every 4 h over a 24-h period. Statistical evaluation of data for time effect was performed by one way ANOVA and least-squares fit of 24-h cosine. Little or no significant change of CS concentrations with time over a 24-h period was observed in healthy and asthmatic sera. CS concentrations were significantly lower in steroid-independent asthmatics compared to controls while there was no difference between healthy subjects and steroid-dependent asthmatics. After one week of therapy MP decreased CS concentrations while CsA had no effect. Our data suggest the involvement of CS in asthma pathogenesis and the potential use of CS levels as an additional biological parameter for monitoring the extent of disease and response to therapy.

Serum cystatin C, a potent inhibitor of cysteine proteinases, is elevated in asthmatic patients

The effect of asthma pathogenesis on serum cystatin C, a potent inhibitor of cysteine proteinases and a newly proposed marker of the renal function, has not been yet determined. The objectives were to determine the 24-h pattern of cystatin C and creatinine concentrations in sera of asthmatic patients in order to test whether their concentrations might reflect circadian rhythms, the disease severity and the effect of therapy. Serum concentrations of cystatin C and creatinine were determined in steroid-independent and steroid-dependent asthmatics before and after 1 week of treatment with methylprednisolone and cyclosporin A, respectively. Samples were collected every 4 h during a 24-h period. Little or no significant effects of time on cystatin C and creatinine concentrations over a 24-h period were observed in healthy and asthmatic sera. However, significantly higher cystatin C concentrations were found in asthmatic patients compared to controls which suggests its role in the pathogenesis of asthma. Methylprednisolone increased and cyclosporin A decreased serum cystatin C concentrations after 1 week of therapy. Additionally these results support the need for the evaluation of cystatin C as a marker of glomerular filtration rate determination in asthma.