External quality assessment of point-of-care methods: model for combined assessment of method bias and single-participant performance by the use of native patient samples and noncommutable control materials

Analytical performance specifications and quality assurance of point-of-care testing in primary healthcare

Point-of-care testing (POCT) is the fastest-growing segment of laboratory medicine. This review focuses on the essential aspects of setting analytical performance specifications (APS) and performing quality assurance for POCT in primary healthcare. In-vitro diagnostic medical devices for POCT are typically small and easy to operate. Users often have little to no laboratory experience and may not necessarily see the value of conducting quality assurance on their devices. Therefore, training, guidance, and motivation should be integral parts of the total quality management system, as they are vital for managing errors and ensuring reliable results. It is common to believe that the analytical quality of POCT should be comparable to that of laboratory testing, and as a result, APS should be the same. This paper challenges this concept. The APS for POCT can often be less stringent compared to those used in a central laboratory because the requester is closer to both the analytical and clinical situation. Point-of-care instruments should be selected based on clinical needs, the required analytical quality and user-friendliness in the intended usage setting.Quality assurance should include both internal quality control (IQC) and external quality assessment (EQA). It is recommended that IQC protocols should be dependent on the complexity of the POCT device. A scoring system to determine how frequent IQC should be analyzed in primary healthcare on different types of POCT devices has been suggested. The main challenge in EQA for POCT involves using suitable control materials that reflect instrument performance on patient samples. Obtaining commutable control materials for POCT is difficult since the matrix often is whole blood. An essential aspect of EQA for POCT is that feedback reports should be easily interpretable. Users should receive advice from the EQA organizer regarding the root causes of deviating results. Quality assurance for POCT is not an easy task and presents numerous challenges. However, there is evidence that quality assurance improves the quality of POCT measurements and, consequently, can enhance patient outcomes.

External quality assurance in the era of standardization

Metrology in clinical chemistry aims to ensure the equivalence of measurement results from different in-vitro diagnostic measurement devices (IVD MD) for use in healthcare. The metrological traceability of measurement results to higher-order references is the cornerstone to achieving equivalent results. However, other fundamentals are also needed, including the commutability of reference materials and external quality assessment (EQA) materials for monitoring the equivalence of measurement results at the end-user level. This manuscript summarizes the findings and opinions expressed at the Joint Community for Traceability in Laboratory Medicine (JCTLM) workshop held on December 4-5, 2023. The workshop explored the relationship between EQA/proficiency testing and metrological traceability to higher-order references. EQA monitors the equivalence of measurement results from end-user IVD MDs. The workshop discussed the role and challenges of using EQA to improve and maintain the equivalence of measurement results. It also elucidated current developments in establishing the clinical suitability of laboratory results expressed as analytical performance specifications (APS).

Establishing an External Quality Assessment (EQA) Program for Urinalysis in Medical Laboratories of Thailand

Thailand Association of Clinical Biochemists (TACB) introduced External Quality Assurance schemes (EQAs) for urinalysis (UA) using urine strips in medical laboratories of Thailand. The few available External Quality Assessment (EQA) programs on urinary microalbumin rarely include an evaluation of clinical cases. The aim of the present study was to assess a descriptive analysis of biochemical urinalysis including urine microalbumin in the Thailand laboratory practice. From January 2021 to December 2021, four surveys were organized. EQA urine samples were distributed to the participants by mail. The participants measured the UA of 2 samples quarterly and returned the results together with the information about their instruments and suggestion for the performance of the laboratory report quarterly. Moreover, summary of the situation of each laboratory performance was feedbacked by online system. Fifty-eight laboratories participated in the survey. The EQA panels included positive and negative samples. The analytical results for passed parameters of urine chemical test range from 79.3-100%. All special tests; microalbumin, creatinine, and beta-HCG showed correct result from 85.1-96.1%. The overall accuracy, specificity, and sensitivity were 92.6, 85.7, and 75,4%, respectively. The major issues were observed: the low sensitivity for the detection of low-concentration samples and the incapacity of several methods to detect the positive sample. The assessment is needed to continuously evaluate the improvement proficiency of laboratories in Thailand.

ICSH guidance for INR and D-dimer testing using point of care testing in primary care

This guideline has been written on behalf of the International Council for Standardisation in Haematology (ICSH) and focuses on two point of care haematology tests used within primary care, namely International Normalised Ratio (INR) and D-dimer. Primary care covers out of hospital settings and can include General Practice (GP), Pharmacy and other non-hospital settings (although these guidelines would also be applicable to hospital out-patient settings). The recommendations are based on published data in peer reviewed literature and expert opinion; they should supplement regional requirements, regulations or standards.

How Can Point-of-Care Technologies Support In-Hospital Diabetes Care?

People with diabetes admitted to hospital are at risk of diabetes related complications including hypoglycaemia and diabetic ketoacidosis. Point-of-care (POC) tests undertaken at the patient bedside, for glucose, ketones, and other analytes, are a key component of monitoring people with diabetes, to ensure safety. POC tests implemented with a quality framework are critical to ensuring accuracy and veracity of results and preventing erroneous clinical decision making. POC results can be used for self-management of glucose levels in those well-enough and/or by healthcare professionals to identify unsafe levels. Connectivity of POC results to electronic health records further offers the possibility of utilising these results proactively to identify patients 'at risk' in real-time and for audit purposes. In this article, the key considerations when implementing POC tests for diabetes in-patient management are reviewed and potential to drive improvements using networked glucose and ketone measurements are discussed. In summary, new advances in POC technology should allow people with diabetes and the teams looking after them whilst in hospital to integrate to provide safe and effective care.

Using three external quality assurance schemes to achieve equivalent international normalized ratio results in primary and secondary healthcare

OBJECTIVES

Accurate prothrombin time international normalized ratio (INR) results are essential for safe anticoagulation treatment. Patients are treated both in primary and secondary healthcare, therefore equivalence of INR results from point-of-care (POC) and hospital measurement procedures (MPs) are important. It is not possible to evaluate this equivalence in traditional external quality assessment (EQA). The aim of this paper is to describe a special quality assurance system consisting of three different EQA schemes to monitor the harmonization of INR results in Norway.

METHODS

The EQA scheme for hospital laboratories uses commutable control materials and evaluates participant performance and the equivalence of hospital MPs. The EQA scheme for primary healthcare laboratories uses non-commutable control materials and evaluates participant performance. A third EQA scheme for selected primary healthcare laboratories uses native patient split samples and evaluates the equivalence between POC and hospital MPs.

RESULTS

The relationship between the three EQA schemes is presented. The split sample EQA scheme provides a link between the hospital scheme and the scheme for primary healthcare. Results from 2017 to 2022 are presented for all three schemes. When aberrant EQA results occur Noklus takes actions to be able to have a sustainable equivalence between INR results.

CONCLUSIONS

All three EQA schemes are important for monitoring the harmonization of INR results in Norway. This quality assurance system, including help and guidance of the participants, will reduce the risk of harm to patients due to non-equivalence of results from different MPs.

Harmonization activities of Noklus – a quality improvement organization for point-of-care laboratory examinations

Noklus is a non-profit quality improvement organization that focuses to improve all elements in the total testing process. The aim is to ensure that all medical laboratory examinations are ordered, performed and interpreted correctly and in accordance with the patients’ needs for investigation, treatment and follow-up. For 25 years, Noklus has focused on point-of-care (POC) testing in primary healthcare laboratories and has more than 3100 voluntary participants. The Noklus quality system uses different tools to obtain harmonization and improvement: (1) external quality assessment for the pre-examination, examination and postexamination phase to monitor the harmonization process and to identify areas that need improvement and harmonization, (2) manufacturer-independent evaluations of the analytical quality and user-friendliness of POC instruments and (3) close interactions and follow-up of the participants through site visits, courses, training and guidance. Noklus also recommends which tests that should be performed in the different facilities like general practitioner offices, nursing homes, home care, etc. About 400 courses with more than 6000 delegates are organized annually. In 2017, more than 21,000 e-learning programs were completed.

Commutability and traceability in EQA programs

OBJECTIVES

The concept of commutability of samples has focused laboratories on the importance of traceability. However, the critical role of External Quality Assurance (EQA) in achieving the primary role of traceability (i.e. facilitating comparable patient results in different laboratories) has largely been lost. The aim of this paper is to review the role of EQA in achieving traceable/commutable results.

DESIGN AND METHODS

The role of commutability and traceability in EQA and Internal Quality Control (IQC) are discussed. Examples of commutable EQA samples are given to highlight the problem of assuming EQA material does not behave like patient samples.

RESULTS

We provide the conventional traceability chain (top down) and the role of EQA in a "bottom up" model using conventional EQA samples.

CONCLUSIONS

The quest for commutable samples has compromised the value of EQA without an understanding that some EQA materials are commutable for some measurands. EQA plays a key role in performance improvement, but laboratories need to understand the importance of using a range of values appropriate to the assay to identify areas of quality need. Traceability and EQA using conventional samples are not mutually exclusive concepts.

Essential aspects of external quality assurance for point-of-care testing

External quality assurance (EQA) or proficiency testing for point-of-care (POC) testing is in principle similar to EQA for larger hospital laboratories, but the participants are different. The participants are usually health care personnel with little or no knowledge of laboratory medicine. The implication of this is that the EQA provider has to a) convince the participants that participation in EQA schemes are important, b) be able to circulate materials with reasonable time intervals, c) produce feedback reports that are understandable, and d) offer help and guidance to the participants when needed. It is also important that EQA for POC testing e) address the pre-examination, the examination and the post-examination processes, and f) that schemes for measurement procedures using interval or ordinal scale are offered. The aim of the present paper is to highlight important issues of these essential aspects of EQA for POC testing.

Impact of HbA1c Testing at Point of Care on Diabetes Management

Diabetes is a highly prevalent disease also implicated in the development of several other serious complications like cardiovascular or renal disease. HbA1c testing is a vital step for effective diabetes management, however, given the low compliance to testing frequency and, commonly, a subsequent delay in the corresponding treatment modification, HbA1c at the point of care (POC) offers an opportunity for improvement of diabetes care. In this review, based on data from 1999 to 2016, we summarize the evidence supporting a further implementation of HbA1c testing at POC, discuss its limitations and propose recommendations for further development.

Effect of Participating in a Quality Improvement System over Time for Point-of-Care C-Reactive Protein, Glucose, and Hemoglobin Testing

BACKGROUND

Users of point-of-care testing (POCT) in Norway participate in a quality improvement system that includes education and guidance in safe laboratory management along with participation in external quality assurance schemes (EQAS).

The aim of this study was to identify the effect on the analytical performance of POCT C-reactive protein (CRP), glucose, and hemoglobin (Hb) with the use of a quality improvement system over time and to identify which factors are associated with good performance.

METHODS

Participants' results from 19 EQAS for CRP, glucose, and Hb from 2006 to 2015 along with information on the instruments used and different practice characteristics were analyzed. Logistic regression analysis was used to evaluate the factors associated with good laboratory performance. An instrument evaluation and comparison for CRP determination was performed by using commutable EQA material.

RESULTS

The mean number of participants in each EQAS was 2134, 2357, and 2271 for CRP, glucose, and Hb, respectively. The percentage of good participant performances increased gradually whereas that of poor performances decreased with participation in a quality improvement system over 9 years for all 3 analytes. Independent factors associated with good performance were type of instrument, the number of times performing EQA, performing internal QC weekly, performing 10 or more tests weekly, and having laboratory-qualified personnel perform the tests. Considering CRP instrument performance, Afinion and QuikRead exhibited the lowest systematic deviation.

CONCLUSIONS

The analytical quality of CRP, glucose, and Hb testing is improved by systematic participation in a quality improvement system over time.

Postmarket Surveillance of Point-of-Care Glucose Meters through Analysis of Electronic Medical Records

BACKGROUND

The electronic medical record (EMR) holds a promising source of data for active postmarket surveillance of diagnostic accuracy, particularly for point-of-care (POC) devices. Through a comparison with prospective bedside and laboratory accuracy studies, we demonstrate the validity of active surveillance via an EMR data mining method [Data Mining EMRs to Evaluate Coincident Testing (DETECT)], comparing POC glucose results to near-in-time central laboratory glucose results.

METHODS

The Roche ACCU-CHEK Inform II® POC glucose meter was evaluated in a laboratory validation study (n = 73), a prospective bedside intensive care unit (ICU) study (n = 124), and with DETECT (n = 852–27 503). For DETECT, the EMR was queried for POC and central laboratory glucose results with filtering based on of bedside collection timestamps, central laboratory time delays, patient location, time period, absence of repeat testing, and presence of peripheral lines.

RESULTS

DETECT and the bedside ICU study produced similar estimates of average bias (4.5 vs 5.0 mg/dL) and relative random error (6.3% vs 5.6%), with overlapping CIs. For glucose <100 mg/dL, the laboratory validation study estimated a lower relative random error of 3.6%. POC average bias correlated with central laboratory turnaround times, consistent with 4.8 mg · dL−1 · h−1 glycolysis. After glycolysis adjustment, average bias was estimated by the bedside ICU study at −0.4 mg/dL (CI, −1.6 to 0.9) and DETECT at −0.7 (CI, −1.3 to 0.2), and percentage POC results occurring outside Clinical Laboratory Standards Institute quality goals were 2.4% and 4.8%, respectively.

CONCLUSIONS

This study validates DETECT for estimating POC glucose meter accuracy compared with a prospective bedside ICU study and establishes it as a reliable postmarket surveillance methodology.

The Importance of Reagent Lot Registration in External Quality Assurance/Proficiency Testing Schemes

BACKGROUND

Providers of external quality assurance (EQA)/proficiency testing schemes have traditionally focused on evaluation of measurement procedures and participant performance and little attention has been given to reagent lot variation. The aim of the present study was to show the importance of reagent lot registration and evaluation in EQA schemes.

METHODS

Results from the Noklus (Norwegian Quality Improvement of Primary Care Laboratories) urine albumin/creatinine ratio (ACR) and prothrombin time international normalized ratio (INR) point-of-care EQA schemes from 2009-2015 were used as examples in this study.

RESULTS

The between-participant CV for Afinion ACR increased from 6%-7% to 11% in 3 consecutive surveys. This increase was caused by differences between albumin reagent lots that were also observed when fresh urine samples were used. For the INR scheme, the CoaguChek INR results increased with the production date of the reagent lots, with reagent lot medians increasing from 2.0 to 2.5 INR and from 2.7 to 3.3 INR (from the oldest to the newest reagent lot) for 2 control levels, respectively. These differences in lot medians were not observed when native patient samples were used.

CONCLUSIONS

Presenting results from different reagent lots in EQA feedback reports can give helpful information to the participants that may explain their deviant EQA results. Information regarding whether the reagent lot differences found in the schemes can affect patient samples is important and should be communicated to the participants as well as to the manufacturers. EQA providers should consider registering and evaluating results from reagent lots.

Bias in clinical chemistry

Clinical chemistry uses automated measurement techniques and medical knowledge in the interest of patients and healthy subjects. Automation has reduced repeatability and day-to-day variation considerably. Bias has been reduced to a lesser extent by reference measurement systems. It is vital to minimize clinically important bias, in particular bias within conglomerates of laboratories that measure samples from the same patients. Small and variable bias components will over time show random error properties and conventional random-error based methods for calculating measurement uncertainty can then be applied. The present overview of bias presents the general principles of error and uncertainty concepts, terminology and analysis, and suggests methods to minimize bias and measurement uncertainty in the interest of healthcare.

Analytical accuracy and precision of two novel Point-of-Care systems for INR determination

BACKGROUND

Portable point-of-care (POC) instruments for determination of the whole blood prothrombin time (PT) have been available for the last three decades. Recently, two novel POC instruments for PT and International Normalized Ratio (INR) determination in whole blood have been manufactured. The purpose of this study was to compare INR values obtained with the novel instruments (microINR® and ProTime InRhythm™) to the INR determined with the international standard for thromboplastin rTF/09.

MATERIALS AND METHODS

In 60 patients treated with vitamin K-antagonists, venous whole blood was analysed with four different types of POC instruments including the novel ones. In the same patients, citrated plasma was analysed with the international standard rTF/09 and the manual tilt tube technique for clotting time determination. We assessed the bias of the INR read from the POC instruments relative to the international standard. To study the imprecision of the two novel POC instruments, duplicate INR determinations were performed.

RESULTS

The results obtained with the two novel POC instruments were positively correlated with those of the international standard rTF/09. However, there was a significant bias between INR read from the novel instruments and the INR determined with rTF/09 (P < 0.001). The mean bias was -13.7% (MicroINR) and -9.3% (InRhythm). The imprecision coefficient of variation in venous blood was 5.0% and 5.1%, respectively.

CONCLUSION

The imprecision of the two novel instruments is acceptable with respect to the average within-subject variation of the INR. The accuracy of the systems is borderline and should be improved by the manufacturers.

Measurements for 8 common analytes in native sera identify inadequate standardization among 6 routine laboratory assays

BACKGROUND

External quality assessment (EQA) with commutable samples is essential for assessing the quality of assays performed by laboratories, particularly when the emphasis is on their standardization status and interchangeability of results.

METHODS

We used a panel of 20 fresh-frozen single-donation serum samples to assess assays for the measurement of creatinine, glucose, phosphate, uric acid, total cholesterol, HDL cholesterol, LDL cholesterol, and triglycerides. The commercial random access platforms included: Abbott Architect, Beckman Coulter AU, Ortho Vitros, Roche Cobas, Siemens Advia, and Thermo Scientific Konelab. The assessment was done at the peer group level and by comparison against the all-method trimmed mean or reference method values, where available. The considered quality indicators were intraassay imprecision, combined imprecision (including sample-matrix interference), bias, and total error. Fail/pass decisions were based on limits reflecting state-of-the-art performance, but also limits related to biological variation.

RESULTS

Most assays showed excellent peer performance attributes, except for HDL- and LDL cholesterol. Cases in which individual assays had biases exceeding the used limits were the Siemens Advia creatinine (-4.2%), Ortho Vitros phosphate (8.9%), Beckman Coulter AU triglycerides (5.4%), and Thermo Scientific Konelab uric acid (6.4%), which lead to considerable interassay discrepancies. Additionally, large laboratory effects were observed that caused interlaboratory differences of >30%.

CONCLUSIONS

The design of the EQA study was well suited for monitoring different quality attributes of assays performed in daily laboratory practice. There is a need for improvement, even for simple clinical chemistry analytes. In particular, the interchangeability of results remains jeopardized both by assay standardization issues and individual laboratory effects.