Performance Assessment of Three Continuous Glucose Monitoring Systems in Adults With Type 1 Diabetes

Validating the Use of Continuous Glucose Monitors With Nondiabetic Recreational Runners

PURPOSE

Continuous glucose monitors (CGMs) are becoming increasingly popular among endurance athletes despite unconfirmed accuracy. We assessed the concurrent validity of the FreeStyle Libre 2 worn on 2 different sites at rest, during steady-state running, and postprandial.

METHODS

Thirteen nondiabetic, well-trained recreational runners (age = 40 [8] y, maximal aerobic oxygen consumption = 46.1 [6.4] mL·kg-1·min-1) wore a CGM on the upper arm and chest while treadmill running for 30, 60, and 90 minutes at intensities corresponding to 50%, 60%, and 70% of maximal aerobic oxygen consumption, respectively. Glucose was measured by manually scanning CGMs and obtaining a finger-prick capillary blood glucose sample. Mean absolute relative difference, time in range, and continuous glucose Clarke error grid analysis were used to compare paired CGM and blood glucose readings.

RESULTS

Across all intensities of steady-state running, we found a mean absolute relative difference of 13.8 (10.9) for the arm and 11.4 (9.0) for the chest. The coefficient of variation exceeded 70%. Approximately 47% of arm and 50% of chest paired glucose measurements had an absolute difference ≤10%. Continuous glucose Clarke error grid analysis indicated 99.8% (arm) and 99.6% (chest) CGM data fell in clinically acceptable zones A and B. Time-in-range analysis showed reduced accuracy at lower glucose levels. However, CGMs accurately detected trends in mean glucose readings over time.

CONCLUSIONS

CGMs are not valid for point glucose monitoring but appear to be valid for monitoring glucose trends during steady-state exercise. Accuracy is similar for arm and chest. Further research is needed to determine whether CGMs can detect important events such as hypoglycemia during exercise.

Importance of FDA-Integrated Continuous Glucose Monitors to Ensure Accuracy of Continuous Glucose Monitoring

Continuous glucose monitoring (CGM) has been shown to improve glycemic control and self-monitoring, as well as to reduce the risk of hypoglycemia. Integrated CGM (iCGM) FDA-cleared systems with published performance data are established nonadjunctive and accurate CGM tools that can directly inform decision-making in the treatment of diabetes (i.e., insulin dosing). Studies have assessed accuracy and safety data of CGMs that were eventually cleared for iCGM by the FDA and that informed the recommendation for their nonadjunctive use. Subsequent robust clinical trials and real-world studies demonstrated clinical effectiveness with improvements in a range of patient outcomes. In recent years, a number of non-iCGM-approved CGM devices have entered the market outside the United States worldwide. Some of these non-iCGM-approved CGM devices require additional user verification of blood glucose levels to be performed for making treatment decisions, termed adjunctive. Moreover, in many non-iCGM-approved CGM devices, accuracy studies published in peer-reviewed journals are scarce or have many limitations. Consequently, non-iCGM-approved CGM devices cannot be automatically perceived as having the same performance or quality standards than those approved for iCGM by the FDA. As a result, although these devices tend to cost less than iCGMs that carry FDA clearance and could therefore be attractive from the point of view of a health care payer, it must be emphasized that evaluation of costs should not be limited to the device (such as the usability preference that patients have for nonadjunctive sensors compared to adjunctive sensors) but to the wider value of the total benefit that the product provides to the patient.

Benefits of Usability Evaluation in the Development Process of Diabetes Technologies Using the Example of a Continuous Glucose Monitoring System Prototype

BACKGROUND

Usability engineering analyzes the interaction between the intended users and a device. Its implementation is mandatory for manufacturers to obtain regulatory approval for the European market. The aim of this evaluation was assessing the role of usability testing in the development process. For this purpose, a continuous glucose monitoring (CGM) device under development was investigated to determine whether it could be used safely and effectively by the intended users.

METHODS

Conduct of the usability testing was based on the international standard IEC 62366-1. Medical device use of CGM-experienced and non-experienced users (n = 15 each) was observed without initial training in use scenarios containing 18 tasks. The success rate of task completion was determined and the System Usability Scale (SUS) score was calculated from a questionnaire. A prototype of the FiberSense CGM System (EyeSense GmbH, Großostheim, Germany), comprising of a single-use sensor and a reusable detector, was investigated.

RESULTS

Most use errors made by both user groups were related to ease of handling of the reusable detectors. The SUS scores achieved in this study were below the pre-defined SUS score acceptance criterion of ≥68. The most frequently mentioned reason for use errors was an incomprehensible and non-chronological instructions for use (IFU).

CONCLUSIONS

The evaluation provides valuable insights on how to improve usability of the prototype device and demonstrates the value of conducting structured usability testing prior to product finalization. The results reflected areas for improvement of the user interface, mainly by restructuring the IFU, provision of an additional leaflet, and device training prior to use.

Clinical Performance Evaluation of Continuous Glucose Monitoring Systems: A Scoping Review and Recommendations for Reporting

The use of different approaches for design and results presentation of studies for the clinical performance evaluation of continuous glucose monitoring (CGM) systems has long been recognized as a major challenge in comparing their results. However, a comprehensive characterization of the variability in study designs is currently unavailable. This article presents a scoping review of clinical CGM performance evaluations published between 2002 and 2022. Specifically, this review quantifies the prevalence of numerous options associated with various aspects of study design, including subject population, comparator (reference) method selection, testing procedures, and statistical accuracy evaluation. We found that there is a large variability in nearly all of those aspects and, in particular, in the characteristics of the comparator measurements. Furthermore, these characteristics as well as other crucial aspects of study design are often not reported in sufficient detail to allow an informed interpretation of study results. We therefore provide recommendations for reporting the general study design, CGM system use, comparator measurement approach, testing procedures, and data analysis/statistical performance evaluation. Additionally, this review aims to serve as a foundation for the development of a standardized CGM performance evaluation procedure, thereby supporting the goals and objectives of the Working Group on CGM established by the Scientific Division of the International Federation of Clinical Chemistry and Laboratory Medicine.