The Quantitative Relationship Between ISO 15197 Accuracy Criteria and Mean Absolute Relative Difference (MARD) in the Evaluation of Analytical Performance of Self-Monitoring of Blood Glucose (SMBG) Systems

Flash glucose monitoring system in gestational diabetes: a study of accuracy and usability

PURPOSE

Studies of flash glucose monitoring systems (FGMSs) in pregnancy are insufficient, especially in gestational diabetes (GD). Our aim was to evaluate Freestyle Libre's usability and accuracy (compared to self-monitoring of blood glucose [SMBG]) for GD patients in real-life conditions.

METHODS

This is a prospective study with pregnant women diagnosed with GD (n = 24 for the usability analysis; n = 19 for the accuracy analysis). The study duration was up to 28 days (lifetime of two sensors). Participants executed a minimum of four daily FGMS readings obtained immediately after capillary SMBG. Analytical accuracy was assessed with mean absolute relative difference (MARD) and mean absolute difference (MAD); clinical accuracy was assessed with Surveillance Error Grid (SEG). Usability was evaluated with a user acceptability questionnaire.

RESULTS

The mean pregestational BMI was 25.21 ± 5.15 kg/m2 (mean ± SD), the mean gestational age was 30.31 ± 2.02 weeks, and the mean glucose values were 76.63 ± 7.49 mg/dL. A total of 1339 SMBG-FGMS pairs of values were obtained. Analytical accuracy was good with an overall MARD of 14.07% and an in-target MARD of 13.79%. The number of SMBG-FMGS pairs for above-target values was low (122 of 1339) with an associated MARD of 17.95%. Clinical accuracy of the FGMS was demonstrated, with 94.4% of values in the no-risk or slight, lower risk zones of the SEG. FGMS accuracy was unaffected by pregestational BMI or gestational age. The user acceptability questionnaire showed high levels of satisfaction, with 95.8-100% preferring FGMS to SMBG. No unexpected or severe adverse effects occurred.

CONCLUSION

FGMS showed good performance in GD regarding accuracy and usability. Larger studies are needed to corroborate our results, verify the analytical accuracy of above-target values as this glucose range might lead to initiation or adjustment of pharmacological therapy, and ultimately establish definitive recommendations regarding prescription of FGMS for GD patients.

Post-Market Surveillance Assessment of the Clinical Accuracy of a Blood Glucose Monitoring System with an Improved Algorithm for Enhanced Product Performance

BACKGROUND

On-going manufacturer-led post-market surveillance (PMS), assessing the clinical accuracy of blood glucose monitoring (BGM) systems, is critical to substantiate the performance of such products for people with diabetes.

MATERIALS AND METHODS

Batches of Verio test-strip product were randomly and routinely selected over the period from launch of an improved-algorithm product to reporting date and sent to 3 clinic sites for clinician-led accuracy assessment. Accuracy is reported as per recently adopted FDA guidance for BGM systems, EN ISO 15197:2015 and MARD/MAD (Mean absolute relative difference/Mean absolute difference).

RESULTS

Thirty-three individual test-strip batches were evaluated corresponding to 506 unique donors. Accuracy performance - FDA: 98.9% of values within ±15% of comparator; ISO: 99.0% within ±15 mg/dL or ±15% at <100 mg/dL (<5.55 mmol/L) or ≥100 mg/dL (≥5.55 mmol/L) glucose, respectively. Overall MARD was 4.19% with a MARD range of 3.54%-5.73% across all test strip batches.

CONCLUSIONS

This post-market surveillance program demonstrates the new BGM system consistently meets measures of clinical accuracy specified by regulators. This program supports a growing demand by regulators for real-world evidence demonstrating consistent in-market product efficacy as opposed to the current largely passive approach that relies on assessment of reports filed by device users.

The Devil Is in the Details: Use, Limitations, and Implementation of Continuous Glucose Monitoring in the Inpatient Setting

Until recently, continuous glucose monitoring (CGM) systems were reserved for use in the outpatient setting or for investigational purposes in hospitalized patients. However, during the coronavirus disease 2019 pandemic, use of CGM in the inpatient setting has grown rapidly. This review outlines important details related to the accuracy, limitations, and implementation of, as well as necessary staff education for, inpatient CGM use and offers a glimpse into the future of CGM in the inpatient setting.

Mean Absolute Relative Difference of Blood Glucose Monitoring Systems and Relationship to ISO 15197

BACKGROUND

The analytical quality of a blood glucose monitoring system (BGMS) is often assessed according to the requirements described in the international standard ISO 15197. However, the mean absolute relative difference (MARD) is sometimes used as well. This analysis aims at providing empirical data from BGMS evaluation studies conducted according to ISO 15197 and at providing an estimation of how MARD and percentage of measurement results within ISO accuracy limits are related.

METHODS

Results of 77 system accuracy evaluations conducted according to ISO 15197 were used to calculate MARD between BGMS and a laboratory comparison method's results (glucose oxidase or hexokinase method). Additionally, bias and 95%-limits of agreement (LoA) using the Bland and Altman method were calculated.

RESULTS

MARD results ranged from 2.3% to 20.5%. The lowest MARD of a test strip lot that showed <95% of results within ISO limits was 6.1%. The distribution of MARD results shows that only 3.6% of test strip lots with a MARD equal to or below 7% showed <95% of results within ISO limits (2.2% of all test strip lots). Bias of test strip lots that showed ≥95% of results within the limits ranged from -10.3% to +7.4%. The half-width of the 95%-LoA of test strip lots that showed ≥95% of results within the limits ranged from 4.8% to 24.0%.

CONCLUSION

There is a threshold MARD that may allow an estimate whether ISO 15197 requirements are fulfilled, but this statement cannot be made with certainty.

Evaluation of the Abbot FreeStyle Optium Neo H blood glucose meter in the hyperbaric oxygen environment

INTRODUCTION

This study investigated the effects of hyperbaric oxygen treatment (HBOT) on the accuracy and reliability of point-of-care fingertip capillary blood glucose values in euglycaemic non-diabetic participants compared against venous serum blood glucose samples processed in an accredited pathology laboratory.

METHOD

Ten non-diabetic hyperbaric staff members (age 35-55 years) underwent a standard 243 kPa HBOT exposure for 95 minutes. Blood glucose levels were measured via (i) finger-prick capillary test using the FreeStyle Optium™ Neo H glucometer and (ii) venous serum test using the Cobas 6000 laboratory analyser. Samples were taken at (T1) 0 minutes (pre-HBOT), (T2) 25 minutes, and (T3) 55 minutes into HBOT.

RESULTS

All participants were euglycaemic at T1 (BGL 3.8-5.4 mmol·L⁻¹). The highest venous serum value was 5.90 mmol·L⁻¹ at T3 and the highest capillary value was 6.30 mmol·L⁻¹ at T1. Post hoc tests showed a statistically significant difference between the mean capillary result pre-dive (T1) and readings at T2 (P = 0.001) and T3 (P < 0.001) while differences between T2 and T3 capillary results were not statistically significant, illustrating the effect of HBOT on capillary beds. Differences in venous values across the time points were not significant.

CONCLUSION

Venous serum glucose samples processed in an accredited laboratory may be more consistently accurate, but capillary point-of-care testing avoids delays in sample processing and provides glucose data that are of clinical relevance. The FreeStyle Optium™ Neo H glucometer is safe to use and provides a reliable measurement of blood glucose in the HBOT environment.

Continuous glucose monitoring for hypoglycaemia in children: Perspectives in 2020

Hypoglycaemia in children is a major risk factor for adverse neurodevelopment with rates as high as 50% in hyperinsulinaemic hypoglycaemia (HH). A key part of management relies upon timely identification and treatment of hypoglycaemia. The current standard of care for glucose monitoring is by infrequent fingerprick plasma glucose testing but this carries a high risk of missed hypoglycaemia identification. High-frequency Continuous Glucose Monitoring (CGM) offers an attractive alternative for glucose trend monitoring and glycaemic phenotyping but its utility remains largely unestablished in disorders of hypoglycaemia. Attempts to determine accuracy through correlation with plasma glucose measurements using conventional methods such as Mean Absolute Relative Difference (MARD) overestimate accuracy at hypoglycaemia. The inaccuracy of CGM in true hypoglycaemia is amplified by calibration algorithms that prioritize hyperglycaemia over hypoglycaemia with minimal objective evidence of efficacy in HH. Conversely, alternative algorithm design has significant potential for predicting hypoglycaemia to prevent neuroglycopaenia and consequent brain dysfunction in childhood disorders. Delays in the detection of hypoglycaemia, alarm fatigue, device calibration and current high cost are all barriers to the wider adoption of CGM in disorders of hypoglycaemia. However, machine learning, artificial intelligence and other computer-generated algorithms now offer significant potential for further improvement in CGM device technology and widespread application in childhood hypoglycaemia.

Measures of Accuracy for Continuous Glucose Monitoring and Blood Glucose Monitoring Devices

Currently, patients with diabetes may choose between two major types of system for glucose measurement: blood glucose monitoring (BGM) systems measuring glucose within capillary blood and continuous glucose monitoring (CGM) systems measuring glucose within interstitial fluid. Although BGM and CGM systems offer different functionality, both types of system are intended to help users achieve improved glucose control. Another area in which BGM and CGM systems differ is measurement accuracy. In the literature, BGM system accuracy is assessed mainly according to ISO 15197:2013 accuracy requirements, whereas CGM accuracy has hitherto mainly been assessed by MARD, although often results from additional analyses such as bias analysis or error grid analysis are provided. The intention of this review is to provide a comparison of different approaches used to determine the accuracy of BGM and CGM systems and factors that should be considered when using these different measures of accuracy to make comparisons between the analytical performance (ie, accuracy) of BGM and CGM systems. In addition, real-world implications of accuracy and its relevance are discussed.

Blood glucose meters and test strips: global market and challenges to access in low-resource settings

Blood glucose meters and test strips for self-monitoring of blood glucose (SMBG) are often inaccessible to, and infrequently used by, people with diabetes in countries with limited resources for health care. Supplies for measuring blood glucose can also be scarce in health facilities, despite being needed in a myriad of clinical settings at all levels of the health system. Numerous studies and international guidelines emphasise the value of SMBG in diabetes care, particularly in people with type 1 diabetes. In this Review, we assess global access to blood glucose meters and test strips, collating published information on cost, availability, system accuracy, competitive bidding, technological trends, and non-financial barriers. We also provide new information on global market share data and prices, taxes and tariffs, and product availability. Blood glucose meters and test strips should be viewed similarly to essential medicines, with issues of access prioritised by relevant international agencies. Efforts are needed to reduce tariffs and taxes and to create unified global system accuracy requirements and accountable post-marketing evaluations. Preferential pricing arrangements, pooled procurement, and best-purchasing practices could help to lower direct costs. SMBG supplies should also be included in national health insurance schemes. Enhanced diabetes education of health professionals and patients is crucial to ensure effective use of SMBG. Finally, as technology advances for people who can afford new interstitial fluid glucose monitoring systems, blood glucose meters and test strips must remain available and become more affordable in low-resource settings.

Head-to-head comparison of two continuous glucose monitoring systems on a cardio-surgical ICU

In critical illness hypo-and hyperglycemia have a negative influence on patient outcome. Continuous glucose monitoring (CGM) could help in early detection of hypo-and hyperglycemia. A requirement for these new methods is an acceptable accuracy and precision in clinical practice. In this pilot study we prospectively evaluated the accuracy and precision of two CGM sensors (subcutaneous sensor: Sentrino®, Medtronic and intravasal sensor: Glucoclear®, Edwards) in 20 patients on a cardio-surgical ICU in a head to head comparison. CGM data were recorded for up to 48 h and values were compared with blood-gas-analysis (BGA) values, analysed with Bland-Altman-plots and color-coded surveillance error-grids. Shown are means ± standard deviations. In total 270/255 intravasal/subcutaneous pairs with BGA-values were analysed. The average runtime of the sensors was 28.4 ± 6.4 h. Correlation with BGA values yielded a correlation coefficient of 0.76 (subcutaneous sensor) and 0.92 (intravasal sensor). The Bland Altman Plots revealed an accuracy of 2.5 mg/dl, and a precision of + 43.0 mg/dl to - 38.0 mg/dl (subcutaneous sensor) and an accuracy of - 6.0 mg/dl, and a precision of + 12.4 mg/dl to - 24.4 mg/dl (intravasal sensor). No severe hypoglycemic event, defined as BG level below 40 mg/dl, occurred during treatment. Both sensors showed good accuracy in comparison to the BGA values, however they differ regarding precision, which in case of the subcutaneous sensor is considerable high.

Self-measurement of Blood Glucose and Continuous Glucose Monitoring - Is There Only One Future?

Monitoring glycaemic control in patients with diabetes has evolved dramatically over the past decades. The introduction of easy-to-use systems for self-monitoring of blood glucose (SMBG) utilising capillary blood samples has resulted in the availability of a wide range of systems, providing different measurement quality. Systems for continuous glucose monitoring (CGM) – used mainly in patients with type 1 diabetes (T1D) – were made possible by the development of glucose sensors that measure glucose levels in the interstitial fluid (ISF) in the subcutaneous tissue of the skin. CGM readings might not correspond exactly to SMBG measurement results taken at the same time, especially during rapid changes in either blood glucose or ISF glucose levels. The mean absolute relative difference is the most popular method used for characterising the measurement performance of CGM systems. Unlike the International Organization for Standardization 15197:2013 criteria for SMBG systems, no accuracy standards for CGM systems exist. Measurement quality of CGM systems can vary based on several factors, limiting their safety and effective use in managing diabetes. Patients have to be trained adequately to make safe and efficient use of CGM systems (like with SMBG systems). Also, systems for CGM must be evaluated in terms of patient safety and the ability to provide accurate measurements regardless of the fluctuation of glucose levels. As new technological advancements in glucose monitoring are essential for improved management options of diabetes, such as automated insulin dosing systems, there is a need for a critical view of all such developments. It is likely that both, SMBG and CGM systems, will play important future roles in the treatment of diabetes.

Seven-Year Clinical Surveillance Program Demonstrates Consistent MARD Accuracy Performance of a Blood Glucose Test Strip

BACKGROUND

MARD (mean absolute relative difference) is increasingly used to describe performance of glucose monitoring systems, providing a single-value quantitative measure of accuracy and allowing comparisons between different monitoring systems. This study reports MARDs for the OneTouch Verio® glucose meter clinical data set of 80 258 data points (671 individual batches) gathered as part of a 7.5-year self-surveillance program Methods: Test strips were routinely sampled from randomly selected manufacturer's production batches and sent to one of 3 clinic sites for clinical accuracy assessment using fresh capillary blood from patients with diabetes, using both the meter system and standard laboratory reference instrument.

RESULTS

Evaluation of the distribution of strip batch MARD yielded a mean value of 5.05% (range: 3.68-6.43% at ±1.96 standard deviations from mean). The overall MARD for all clinic data points (N = 80 258) was also 5.05%, while a mean bias of 1.28 was recorded. MARD by glucose level was found to be consistent, yielding a maximum value of 4.81% at higher glucose (≥100 mg/dL) and a mean absolute difference (MAD) of 5.60 mg/dL at low glucose (<100 mg/dL). MARD by year of manufacture varied from 4.67-5.42% indicating consistent accuracy performance over the surveillance period.

CONCLUSIONS

This 7.5-year surveillance program showed that this meter system exhibits consistently low MARD by batch, glucose level and year, indicating close agreement with established reference methods whilste exhibiting lower MARD values than continuous glucose monitoring (CGM) systems and providing users with confidence in the performance when transitioning to each new strip batch.

Continuous Glucose Monitoring (CGM) or Blood Glucose Monitoring (BGM): Interactions and Implications

At the 2017 10th annual International Conference on Advanced Technologies and Treatments for Diabetes (ATTD) in Paris, France, four speakers presented their perspectives on the roles of continuous glucose monitoring (CGM) and of blood glucose monitoring (BGM) in patient management within one symposium. These presentations included discussions of the differences in the accuracy of CGM and BGM, a clinical perspective on the physiological reasons behind differences in CGM and BGM values, and an overview of the impact of variations in device accuracy on patients with diabetes. Subsequently a short summary of these presentations is given, highlighting the value of good accuracy of BGM or CGM systems and the ongoing need for standardization. The important role of both BGM and CGM in patient management was a theme across all presentations.

Clinical Impact of Blood Glucose Monitoring Accuracy: An In-Silico Study

BACKGROUND

Patients with diabetes rely on blood glucose (BG) monitoring devices to manage their condition. As some self-monitoring devices are becoming more and more accurate, it becomes critical to understand the relationship between system accuracy and clinical outcomes, and the potential benefits of analytical accuracy.

METHODS

We conducted a 30-day in-silico study in type 1 diabetes mellitus (T1DM) patients using continuous subcutaneous insulin infusion (CSII) therapy and a variety of BG meters, using the FDA-approved University of Virginia (UVA)/Padova Type 1 Simulator. We used simulated meter models derived from the published characteristics of 43 commercial meters. By controlling random events in each parallel run, we isolated the differences in clinical performance that are directly associated with the meter characteristics.

RESULTS

A meter's systematic bias has a significant and inverse effect on HbA1c ( P < .01), while also affecting the number of severe hypoglycemia events. On the other hand, error, defined as the fraction of measurements beyond 5% of the true value, is a predictor of severe hypoglycemia events ( P < .01), but in the absence of bias has a nonsignificant effect on average glycemia (HbA1c). Both bias and error have significant effects on total daily insulin (TDI) and the number of necessary glucose measurements per day ( P < .01). Furthermore, these relationships can be accurately modeled using linear regression on meter bias and error.

CONCLUSIONS

Two components of meter accuracy, bias and error, clearly affect clinical outcomes. While error has little effect on HbA1c, it tends to increase episodes of severe hypoglycemia. Meter bias has significant effects on all considered metrics: a positive systemic bias will reduce HbA1c, but increase the number of severe hypoglycemia attacks, TDI use, and number of fingersticks per day.