Do the New FDA Guidance Documents Help Improving Performance of Blood Glucose Monitoring Systems Compared With ISO 15197?

Impact of Two Different Reference Measurement Procedures on Apparent System Accuracy of 18 CE-Marked Current-Generation Blood Glucose Monitoring Systems

BACKGROUND

Measurement accuracy has been assessed for many different blood glucose monitoring systems (BGMS) over the years by different study groups. However, the choice of the comparison measurement procedure may impact the apparent level of accuracy found in such studies.

MATERIALS AND METHODS

Measurement accuracy of 18 different BGMS was assessed in a setting based on ISO 15197 using two different comparison methods in parallel: a glucose oxidase (GOD)-based and a hexokinase (HK)-based method. Accuracy limits of ISO 15197 were applied, and additional analyses were performed, including bias, linear regression, and mean absolute relative difference (MARD) to assess the impact of possible differences between comparison methods on the apparent level of accuracy.

RESULTS

While ≈80% of BGMS met the accuracy criteria of ISO 15197 when compared with the respective manufacturers' reference measurement procedure, only two-thirds did so against both comparison methods. The mean relative bias ranged from -6.6% to +5.7% for the analysis against the GOD-based method and from -11.1% to +1.3% for the analysis against the HK-based method, whereas MARD results ranged from 3.7% to 9.8% and from 2.3% to 10.5%, respectively. Results of regression analysis showed slopes between 0.85 and 1.08 (GOD-based method) and between 0.81 and 1.01 (HK-based method).

CONCLUSIONS

The results of this study indicate that there are systematic differences between the reference measurement procedures used for BGMS calibration as well as for system accuracy assessment. Because of the potential impact on therapy of patients with diabetes resulting from these differences, further steps toward harmonization of the measurement procedures' results are important.

Technologies for Diabetes Self-Monitoring: A Scoping Review and Assessment Using the REASSURED Criteria

BACKGROUND

Self-management is an important pillar for diabetes control and to achieve it, glucose self-monitoring devices are needed. Currently, there exist several different devices in the market and many others are being developed. However, whether these devices are suitable to be used in resource constrained settings is yet to be evaluated.

AIMS

To assess existing glucose monitoring tools and also those in development against the REASSURED which have been previously used to evaluate diagnostic tools for communicable diseases.

METHODS

We conducted a scoping review by searching PubMed for peer-review articles published in either English, Spanish or Portuguese in the last 5 years. We selected papers including information about devices used for self-monitoring and tested on humans with diabetes; then, the REASSURED criteria were used to assess them.

RESULTS

We found a total of 7 continuous glucose monitoring device groups, 6 non-continuous, and 6 devices in development. Accuracy varied between devices and most of them were either invasive or minimally invasive. Little to no evidence is published around robustness, affordability and delivery to those in need. However, when reviewing publicly available prices, none of the devices would be affordable for people living in low- and middle-income countries.

CONCLUSIONS

Available devices cannot be considered adapted for use in self-monitoring in resource constraints settings. Further studies should aim to develop less-invasive devices that do not require a large set of components. Additionally, we suggest some improvement in the REASSURED criteria such as the inclusion of patient-important outcomes to increase its appropriateness to assess non-communicable diseases devices.

Do We Need the Replacement of YSI 2300? A View from the Clinical Laboratory

In this issue of Journal of Diabetes Science and Technology, Baumstark et al. evaluated the analytical performance of a bench-top laboratory glucose analyzer (SUPER-GL) intended for replacement for the YSI2300-STAT analyzer, that served for several decades as a comparator method in clinical and analytical studies of blood glucose monitoring systems (BGMS). The authors concluded that the SUPER-GL's overall performance is comparable to that of YSI2300-STAT, and has the potential to be a candidate comparator analyzer. However, the question is if we need to recommend as a "comparator method," a specific device, that measure glucose using the same analytical method with most BGMS. In this analysis we present our point of view hoping to generate a discussion on the necessity for such a replacement.

Glucose biosensors in clinical practice: principles, limits and perspectives of currently used devices

The demand of glucose monitoring devices and even of updated guidelines for the management of diabetic patients is dramatically increasing due to the progressive rise in the prevalence of diabetes mellitus and the need to prevent its complications. Even though the introduction of the first glucose sensor occurred decades ago, important advances both from the technological and clinical point of view have contributed to a substantial improvement in quality healthcare. This review aims to bring together purely technological and clinical aspects of interest in the field of glucose devices by proposing a roadmap in glucose monitoring and management of patients with diabetes. Also, it prospects other biological fluids to be examined as further options in diabetes care, and suggests, throughout the technology innovation process, future directions to improve the follow-up, treatment, and clinical outcomes of patients.

Evaluation of Precision and Bias Specifications Required to Achieve the 2018 FDA Guidance Criteria for Glucose Meter Performance Using Simulation Models

BACKGROUND

The objective of this study was to estimate the combinations of total bias and total imprecision required for devices to meet the Food and Drug Administration (FDA) specifications using Monte Carlo simulation rather than collection and analysis of experimental data.

METHODS

A model Gaussian distribution of true-glucose values was altered by adding bias and imprecision to create measured-glucose values affected by analytic error. The fraction of measured-glucose values that met the 2018 FDA criteria for blood glucose monitoring system (BGMS) or self-monitoring blood glucose (SMBG) devices was determined as a function of bias and imprecision.

RESULTS

The BGMS model determined that a maximum total imprecision of 6% was required with no bias, and with a total bias of +10 mg/dL the total imprecision allowed was reduced to 5% to achieve the 95% FDA performance expectation: 95% of results ≥75 mg/dL within ±12% and 95% of results <75 mg/dL within ±12 mg/dL. The SMBG model determined that a maximum total imprecision of 6% was required at no bias, and with a total bias of +10 mg/dL the total imprecision allowed was reduced to 4% to achieve the 98% FDA expectation: 98% of results ±75 mg/dL within ±15% and 98% of results <75 mg/dL within ±15 mg/dL.

CONCLUSIONS

The 2018 FDA guidance criteria require strict conditions for glucose meter clinical trials to achieve <10 mg/dL total bias and total imprecision of <5%. Total imprecision and bias values assessed in models in this study represent the cumulative imprecision and bias errors for the glucose meters, the reference method, and preanalytic processes.

Accuracy and stability of an arterial sensor for glucose monitoring in a porcine model using glucose clamp technique

Intravascular glucose sensors have the potential to improve and facilitate glycemic control in critically ill patients and might overcome measurement delay and accuracy issues. This study investigated the accuracy and stability of a biosensor for arterial glucose monitoring tested in a hypo- and hyperglycemic clamp experiment in pigs. 12 sensors were tested over 5 consecutive days in 6 different pigs. Samples of sensor and reference measurement pairs were obtained every 15 minutes. 1337 pairs of glucose values (range 37–458 mg/dl) were available for analysis. The systems met ISO 15197:2013 criteria in 99.2% in total, 100% for glucose <100 mg/dl (n = 414) and 98.8% for glucose ≥100 mg/dl (n = 923). The mean absolute relative difference (MARD) during the entire glycemic range of all sensors was 4.3%. The MARDs within the hypoglycemic (<70 mg/dl), euglycemic (≥70–180 mg/dl) and hyperglycemic glucose ranges (≥180 mg/dl) were 6.1%, 3.6% and 4.7%, respectively. Sensors indicated comparable performance on all days investigated (day 1, 3 and 5). None of the systems showed premature failures. In a porcine model, the performance of the biosensor revealed a promising performance. The transfer of these results into a human setting is the logical next step.

Analysis of "Capillary and Venous Blood Glucose Accuracy in Blood Glucose Meters Versus Reference Standards: The Impact of Study Design on Accuracy Evaluations"

The article by Macleod and coauthors outlines an accuracy study of two blood glucose monitoring systems (BGMSs) against glucose-oxidase- and hexokinase-based assays showing how the reference/comparison method and inappropriate sample comparisons can affect accuracy conclusions. The dangers of independent institutions producing inappropriate conclusions when the methodology used for product regulatory or registration accuracy requirements is not according to best practice are stressed. The authors highlight several important aspects of a multistep accuracy evaluation protocol. However, it is essential that anyone undertaking or reporting BGMS accuracy studies should have a clear understanding of each and every protocol point, best practice, and how each can influence accuracy conclusions. Claims against regulatory accuracy criteria should be made only if the detailed specified protocol design and analysis is followed.

Capillary and Venous Blood Glucose Accuracy in Blood Glucose Meters Versus Reference Standards: The Impact of Study Design on Accuracy Evaluations

BACKGROUND

Anecdotal blood glucose assessments conducted by health care professionals (HCPs) in the field have highlighted differences in results when methodology used is not according to best practices for measuring blood glucose. This study assessed the impact on accuracy of blood glucose measurements when methodology deviates from the recommended study design and recommended reference instrument.

METHODS

Adults with type 1 or type 2 diabetes provided capillary and venous blood samples for accuracy assessments using OneTouch® Verio® (Verio) and OneTouch® Ultra 2® (Ultra) blood glucose meters (BGM) and two different reference instruments.

RESULTS

Increases in mean bias were observed when comparing capillary to venous samples tested on the BGMs and the recommended reference instrument. Mean bias was even greater when a hospital blood glucose analyzer was used to measure venous plasma glucose. Increases in mean bias observed for Ultra BGM when testing venous blood on the meter compared to the recommended reference instrument was likely due to the interfering effects of low oxygen levels in the venous blood sample. Conversely, Verio meters, which are insensitive to low oxygen levels, showed little difference from baseline when testing venous blood on the meter compared to results from the same venous sample measured on a reference instrument.

CONCLUSIONS

Deviations from the best practice study design of comparing capillary blood glucose results tested on the blood glucose meter with the manufacturer's stated reference instrument will affect accuracy of blood glucose measurements.

Impact of Glucose Measuring Systems and Sample Type on Diagnosis Rates of Diabetes Mellitus

INTRODUCTION

The use of glucose point-of-care testing (POCT) devices for the diagnosis of diabetes mellitus (DM) is an ongoing controversy. In patient management, glucose concentrations are determined by POCT and core laboratory glucose methods, and the values are commonly compared even though the samples collected are different, namely, capillary whole blood and venous plasma. In individual patients it is difficult to distinguish between factors that can influence the results, such as sample type and measuring procedure. In this study, glucose concentrations obtained using POCT and core laboratory instruments were assessed to duplicate typical scenarios experienced in healthcare. Corresponding diagnosis rates of impaired glucose tolerance (IGT) and DM based on fixed, method-independent cutoffs were compared.

METHODS

Glucose concentration was measured by the 2-h oral glucose tolerance test (OGTT) in samples collected from an inpatient cohort and a cohort from the general population. Two POCT methods, namely, a handheld unit-use glucometer and a small bench-top analyzer with batch reagents, and two core laboratory procedures were used to measure glucose concentrations. The sample types were whole blood and plasma samples collected from venous and capillary blood. The glycated hemoglobin level in whole blood was also determined.

RESULTS

A total of 231 subjects were included in the study. The 2-h OGTT glucose concentrations in the capillary whole blood samples showed a positive bias of 0.8 mmol/L compared to those obtained using core laboratory plasma glucose methods, leading to increased rates of diabetes diagnosis. This bias decreased to 0.2 mmol/L when venous blood was used in the tests. A change in the method used by the core laboratory introduced a negative bias of 0.5 mmol/L and, consequently, a lower diagnosis rates.

DISCUSSION AND CONCLUSION

Venous blood samples measured at the point-of-care are the most suitable sample type for the measurement of the glucose concentration in the 2-h OGTT. The investigated unit-use POCT method is suitable for the diagnosis of IGT and DM when venous blood samples are collected. Importantly, changes in measurement procedures can introduce a bias and affect diagnosis rates, thereby emphasizing the need for further harmonization of glucose methods. A plain language summary is available for this article.

Self-measurement of Blood Glucose and Continuous Glucose Monitoring - Is There Only One Future?

Monitoring glycaemic control in patients with diabetes has evolved dramatically over the past decades. The introduction of easy-to-use systems for self-monitoring of blood glucose (SMBG) utilising capillary blood samples has resulted in the availability of a wide range of systems, providing different measurement quality. Systems for continuous glucose monitoring (CGM) – used mainly in patients with type 1 diabetes (T1D) – were made possible by the development of glucose sensors that measure glucose levels in the interstitial fluid (ISF) in the subcutaneous tissue of the skin. CGM readings might not correspond exactly to SMBG measurement results taken at the same time, especially during rapid changes in either blood glucose or ISF glucose levels. The mean absolute relative difference is the most popular method used for characterising the measurement performance of CGM systems. Unlike the International Organization for Standardization 15197:2013 criteria for SMBG systems, no accuracy standards for CGM systems exist. Measurement quality of CGM systems can vary based on several factors, limiting their safety and effective use in managing diabetes. Patients have to be trained adequately to make safe and efficient use of CGM systems (like with SMBG systems). Also, systems for CGM must be evaluated in terms of patient safety and the ability to provide accurate measurements regardless of the fluctuation of glucose levels. As new technological advancements in glucose monitoring are essential for improved management options of diabetes, such as automated insulin dosing systems, there is a need for a critical view of all such developments. It is likely that both, SMBG and CGM systems, will play important future roles in the treatment of diabetes.

Investigation of the Accuracy of 18 Marketed Blood Glucose Monitors

OBJECTIVE

Cleared blood glucose monitors (BGMs) for personal use may not always deliver levels of accuracy currently specified by international and U.S. regulatory bodies. This study's objective was to assess the accuracy of 18 such systems cleared by the U.S. Food and Drug Administration representing approximately 90% of commercially available systems used from 2013 to 2015.

RESEARCH DESIGN AND METHODS

A total of 1,035 subjects were recruited to have a capillary blood glucose (BG) level measured on six different systems and a reference capillary sample prepared for plasma testing at a reference laboratory. Products were obtained from consumer outlets and tested in three triple-blinded studies. Each of the three participating clinical sites tested a different set of six systems for each of the three studies in a round-robin. In each study, on average, a BGM was tested on 115 subjects. A compliant BG result was defined as within 15% of a reference plasma value (for BG ≥100 mg/dL [5.55 mmol/L]) or within 15 mg/dL (0.83 mmol/L) (for BG <100 mg/dL [5.55 mmol/L]). The proportion of compliant readings in each study was compared against a predetermined accuracy standard similar to, but more lenient than, current regulatory standards. Other metrics of accuracy included the overall compliance proportion; the proportion of extreme outlier readings differing from the reference value by >20%; modified Bland-Altman analysis including average bias, coefficient of variation, and 95% limits of agreement; and proportion of readings with no clinical risk as determined by the Surveillance Error Grid.

RESULTS

The different accuracy metrics produced almost identical BGM rankings. Six of the 18 systems met the predetermined accuracy standard in all three studies, 5 systems met it in two studies, and 3 met it in one study. Four BGMs did not meet the accuracy standard in any of the three studies.

CONCLUSIONS

Cleared BGMs do not always meet the level of analytical accuracy currently required for regulatory clearance. This information could assist patients, professionals, and payers in choosing products and regulators in evaluating postclearance performance.