Basal Glucose Control in Type 1 Diabetes Using Deep Reinforcement Learning: An In Silico Validation

Continuous Glucose, Insulin and Lifestyle Data Augmentation in Artificial Pancreas Using Adaptive Generative and Discriminative Models

Artificial pancreas requires data from multiple sources for accurate insulin dose estimation. These include data from continuous glucose sensors, past insulin dosage information, meal quantity and time and physical activity data. The effectiveness of closed-loop diabetes management systems might be hampered by the absence of these data caused by device error or lack of compliance by patients. In this study, we demonstrate the effect of output sequence length-driven generative and discriminative model selection in high quality data generation and augmentation. This novel generative adversarial network (GAN) based architecture automatically selects the generator and discriminator architecture based on the desired output sequence length. The proposed model is able to generate glucose, physical activity, meal information data for individual patients. The discriminative scores for Ohio T1DM (2018) dataset were 0.17 ±0.03 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, Heart Rate, Steps) and for Ohio T1D (2020) dataset was 0.16 ±0.02 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, acceleration). A mixture of generated and real data was used to test predictive scores for glucose forecasting models. The best RMSE and MARD achieved for OhioT1DM patients were 17.19 ±3.22 and 7.14 ±1.76 for PH=30 min with CGM, CHO, Insulin, heartrate and steps as inputs. Similarly, the RMSE and MARD for real+synthetic data were 15.63 ±2.57 and 5.86 ±1.69 respectively. Compared to existing generative models, we demonstrate that sequence length based architecture selection leads to better synthetic data generation for multiple output sequences (CGM, CHO, Insulin) and forecasting accuracy.

End-to-end offline reinforcement learning for glycemia control

The development of closed-loop systems for glycemia control in type I diabetes relies heavily on simulated patients. Improving the performances and adaptability of these close-loops raises the risk of over-fitting the simulator. This may have dire consequences, especially in unusual cases which were not faithfully - if at all - captured by the simulator. To address this, we propose to use model-free offline RL agents, trained on real patient data, to perform the glycemia control. To further improve the performances, we propose an end-to-end personalization pipeline, which leverages offline-policy evaluation methods to remove altogether the need of a simulator, while still enabling an estimation of clinically relevant metrics for diabetes.

Impartial feature selection using multi-agent reinforcement learning for adverse glycemic event prediction

We developed an attention model to predict future adverse glycemic events 30 min in advance based on the observation of past glycemic values over a 35 min period. The proposed model effectively encodes insulin administration and meal intake time using Time2Vec (T2V) for glucose prediction. The proposed impartial feature selection algorithm is designed to distribute rewards proportionally according to agent contributions. Agent contributions are calculated by a step-by-step negation of updated agents. Thus, the proposed feature selection algorithm optimizes features from electronic medical records to improve performance. For evaluation, we collected continuous glucose monitoring data from 102 patients with type 2 diabetes admitted to Cheonan Hospital, Soonchunhyang University. Using our proposed model, we achieved F1-scores of 89.0%, 60.6%, and 89.8% for normoglycemia, hypoglycemia, and hyperglycemia, respectively.

An Extensive-Form Game Paradigm for Visual Field Testing via Deep Reinforcement Learning

Glaucoma is the leading cause of irreversible but preventable blindness worldwide, and visual field testing is an important tool for its diagnosis and monitoring. Testing using standard visual field thresholding procedures is time-consuming, and prolonged test duration leads to patient fatigue and decreased test reliability. Different visual field testing algorithms have been developed to shorten testing time while maintaining accuracy. However, the performance of these algorithms depends heavily on prior knowledge and manually crafted rules that determine the intensity of each light stimulus as well as the termination criteria, which is suboptimal. We leverage deep reinforcement learning to find improved decision strategies for visual field testing. In our proposed algorithms, multiple intelligent agents are employed to interact with the patient in an extensive-form game fashion, with each agent controlling the test on one of the testing locations in the patient's visual field. Through training, each agent learns an optimized policy that determines the intensities of light stimuli and the termination criteria, which minimizes the error in sensitivity estimation and test duration at the same time. In simulation experiments, we compare the performance of our algorithms against baseline visual field testing algorithms and show that our algorithms achieve a better trade-off between estimation accuracy and test duration. By retaining testing accuracy with reduced test duration, our algorithms improve test reliability, clinic efficiency, and patient satisfaction, and translationally affect clinical outcomes.

The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review

BACKGROUND

Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine.

OBJECTIVE

We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies.

METHODS

We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility.

RESULTS

After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant.

CONCLUSIONS

The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

A personalized multitasking framework for real-time prediction of blood glucose levels in type 1 diabetes patients

Real-time prediction of blood glucose levels (BGLs) in individuals with type 1 diabetes (T1D) presents considerable challenges. Accordingly, we present a personalized multitasking framework aimed to forecast blood glucose levels in patients. The patient data was initially categorized according to gender and age and subsequently utilized as input for a modified GRU network model, creating five prediction sub-models. The model hyperparameters were optimized and tuned after introducing the decay factor and incorporating the TCN network and attention mechanism into the GRU model. This step was undertaken to improve the capability of feature extraction. The Ohio T1DM clinical dataset was used to train and evaluate the performance of the proposed model. The metrics, including Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Clark Error Grid Analysis (EGA), were used to evaluate the performance. The results showed that the average RMSE and the MAE of the proposed model were 16.896 and 9.978 mg/dL, respectively, over the prediction horizon (PH) of 30 minutes. The average RMSE and the MAE were 28.881 and 19.347 mg/dL, respectively, over the PH of 60 min. The proposed model demonstrated excellent prediction accuracy. In addition, the EGA analysis showed that the proposed model accurately predicted 30-minute and 60-minute PH within zones A and B, demonstrating that the framework is clinically feasible. The proposed personalized multitask prediction model in this study offers robust assistance for clinical decision-making, playing a pivotal role in improving the outcomes of individuals with diabetes.

Artificial Intelligence and Machine Learning for Improving Glycemic Control in Diabetes: Best Practices, Pitfalls, and Opportunities

OBJECTIVE

Artificial intelligence and machine learning are transforming many fields including medicine. In diabetes, robust biosensing technologies and automated insulin delivery therapies have created a substantial opportunity to improve health. While the number of manuscripts addressing the topic of applying machine learning to diabetes has grown in recent years, there has been a lack of consistency in the methods, metrics, and data used to train and evaluate these algorithms. This manuscript provides consensus guidelines for machine learning practitioners in the field of diabetes, including best practice recommended approaches and warnings about pitfalls to avoid.

METHODS

Algorithmic approaches are reviewed and benefits of different algorithms are discussed including importance of clinical accuracy, explainability, interpretability, and personalization. We review the most common features used in machine learning applications in diabetes glucose control and provide an open-source library of functions for calculating features, as well as a framework for specifying data sets using data sheets. A review of current data sets available for training algorithms is provided as well as an online repository of data sources.

SIGNIFICANCE

These consensus guidelines are designed to improve performance and translatability of new machine learning algorithms developed in the field of diabetes for engineers and data scientists.

Deep reinforcement learning-based control of chemo-drug dose in cancer treatment

BACKGROUND AND OBJECTIVE

Advancement in the treatment of cancer, as a leading cause of death worldwide, has promoted several research activities in various related fields. The development of effective treatment regimens with optimal drug dose administration using a mathematical modeling framework has received extensive research attention during the last decades. However, most of the control techniques presented for cancer chemotherapy are mainly model-based approaches. The available model-free techniques based on Reinforcement Learning (RL), commonly discretize the problem states and variables, which other than demanding expert supervision, cannot model the real-world conditions accurately. The more recent Deep Reinforcement Learning (DRL) methods, which enable modeling the problem in its original continuous space, are rarely applied in cancer chemotherapy.

METHODS

In this paper, we propose an effective and robust DRL-based, model-free method for the closed-loop control of cancer chemotherapy drug dosing. A nonlinear pharmacological cancer model is used for simulating the patient and capturing the cancer dynamics. In contrast to previous work, the state variables and control action are modeled in their original infinite spaces to avoid expert-guided discretization and provide a more realistic solution. The DRL network is trained to automatically adjust the drug dose based on the monitored states of the patient. The proposed method provides an adaptive control technique to respond to the special conditions and diagnosis measurements of different categories of patients.

RESULTS AND CONCLUSIONS

The performance of the proposed DRL-based controller is evaluated by numerical analysis of different diverse simulated patients. Comparison to the state-of-the-art RL-based method, which uses discretized state and action spaces, shows the superiority of the approach in the process and duration of cancer chemotherapy treatment. In the majority of the studied cases, the proposed model decreases the medication period and the total amount of administrated drug, while increasing the rate of reduction in tumor cells.

Evaluating Deep Q-Learning Algorithms for Controlling Blood Glucose in In Silico Type 1 Diabetes

Patients with type 1 diabetes must continually decide how much insulin to inject before each meal to maintain blood glucose levels within a healthy range. Recent research has worked on a solution for this burden, showing the potential of reinforcement learning as an emerging approach for the task of controlling blood glucose levels. In this paper, we test and evaluate several deep Q-learning algorithms for automated and personalized blood glucose regulation in an in silico type 1 diabetes patient with the goal of estimating and delivering proper insulin doses. The proposed algorithms are model-free approaches with no prior information about the patient. We used the Hovorka model with meal variation and carbohydrate counting errors to simulate the patient included in this work. Our experiments compare different deep Q-learning extensions showing promising results controlling blood glucose levels, with some of the proposed algorithms outperforming standard baseline treatment.

GluGAN: Generating Personalized Glucose Time Series Using Generative Adversarial Networks

Time series data generated by continuous glucose monitoring sensors offer unparalleled opportunities for developing data-driven approaches, especially deep learning-based models, in diabetes management. Although these approaches have achieved state-of-the-art performance in various fields such as glucose prediction in type 1 diabetes (T1D), challenges remain in the acquisition of large-scale individual data for personalized modeling due to the elevated cost of clinical trials and data privacy regulations. In this work, we introduce GluGAN, a framework specifically designed for generating personalized glucose time series based on generative adversarial networks (GANs). Employing recurrent neural network (RNN) modules, the proposed framework uses a combination of unsupervised and supervised training to learn temporal dynamics in latent spaces. Aiming to assess the quality of synthetic data, we apply clinical metrics, distance scores, and discriminative and predictive scores computed by post-hoc RNNs in evaluation. Across three clinical datasets with 47 T1D subjects (including one publicly available and two proprietary datasets), GluGAN achieved better performance for all the considered metrics when compared with four baseline GAN models. The performance of data augmentation is evaluated by three machine learning-based glucose predictors. Using the training sets augmented by GluGAN significantly reduced the root mean square error for the predictors over 30 and 60-minute horizons. The results suggest that GluGAN is an effective method in generating high-quality synthetic glucose time series and has the potential to be used for evaluating the effectiveness of automated insulin delivery algorithms and as a digital twin to substitute for pre-clinical trials.

A practical approach based on learning-based model predictive control with minimal prior knowledge of patients for artificial pancreas

BACKGROUND AND OBJECTIVES

Complete identification of the glucose dynamics for a patient generally requires prior clinical procedures and several measurements for the patient. However, these steps may not be always feasible. To address this limitation, we propose a practical approach integrating learning-based model predictive control (MPC), adaptive basal and bolus injections, and suspension with minimal requirements of prior knowledge of the patient.

METHODS

The glucose dynamic system matrices were periodically updated using only input values, without any pretrained models. The optimal insulin dose was calculated based on a learning-based MPC algorithm. Meal detection and estimation modules were also introduced. The basal and bolus insulin injections were fine-tuned using the performance of glucose control from the previous day. To validate the proposed method, evaluations with 20 virtual patients from a type 1 diabetes metabolic simulator were employed.

RESULTS

Time-in-range (TIR) and time-below-range (TBR) were 90.8% (84.1% - 95.6%) and 0.3% (0% - 0.8%), as represented by the median, first (Q1), and third quartiles (Q3), respectively, when meal intakes were fully announced. When one out of three meal intake announcements was missing, TIR and TBR were 85.2% (75.0% - 88.9%) and 0.9% (0.4% - 1.1%), respectively.

CONCLUSIONS

The proposed approach obviates the need for prior tests from patients and shows effective regulation of blood glucose levels. From the perspective of practical implementation in clinical environments, to deal with minimal prior information of the patient, our study demonstrates how essential clinical knowledge and learning-based modules can be integrated into a control framework for an artificial pancreas.

Offline Deep Reinforcement Learning and Off-Policy Evaluation for Personalized Basal Insulin Control in Type 1 Diabetes

Recent advancements in hybrid closed-loop systems, also known as the artificial pancreas (AP), have been shown to optimize glucose control and reduce the self-management burdens for people living with type 1 diabetes (T1D). AP systems can adjust the basal infusion rates of insulin pumps, facilitated by real-time communication with continuous glucose monitoring. Deep reinforcement learning (DRL) has introduced new paradigms of basal insulin control algorithms. However, all the existing DRL-based AP controllers require extensive random online interactions between the agent and environment. While this can be validated in T1D simulators, it becomes impractical in real-world clinical settings. To this end, we propose an offline DRL framework that can develop and validate models for basal insulin control entirely offline. It comprises a DRL model based on the twin delayed deep deterministic policy gradient and behavior cloning, as well as off-policy evaluation (OPE) using fitted Q evaluation. We evaluated the proposed framework on an in silico dataset generated by the UVA/Padova T1D simulator, and the OhioT1DM dataset, a real clinical dataset. The performance on the in silico dataset shows that the offline DRL algorithm significantly increased time in range while reducing time below range and time above range for both adult and adolescent groups. Then, we used the OPE to estimate model performance on the clinical dataset, where a notable increase in policy values was observed for each subject. The results demonstrate that the proposed framework is a viable and safe method for improving personalized basal insulin control in T1D.

Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes

The widespread adoption of effective hybrid closed loop systems would represent an important milestone of care for people living with type 1 diabetes (T1D). These devices typically utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a healthy range. Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in these devices. Previous approaches have been shown to reduce patient risk and improve time spent in the target range when compared to classical control algorithms, but are prone to instability in the learning process, often resulting in the selection of unsafe actions. This work presents an evaluation of offline RL for developing effective dosing policies without the need for potentially dangerous patient interaction during training. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of the 30 virtual patients available within the FDA-approved UVA/Padova glucose dynamics simulator. When trained on less than a tenth of the total training samples required by online RL to achieve stable performance, this work shows that offline RL can significantly increase time in the healthy blood glucose range from 61.6±0.3% to 65.3±0.5% when compared to the strongest state-of-art baseline (p<0.001). This is achieved without any associated increase in low blood glucose events. Offline RL is also shown to be able to correct for common and challenging control scenarios such as incorrect bolus dosing, irregular meal timings and compression errors. The code for this work is available at: https://github.com/hemerson1/offline-glucose.

Adaptive Personalized Prior-Knowledge-Informed Model Predictive Control for Type 1 Diabetes

This work considers the problem of adaptive prior-informed model predictive control (MPC) formulations that explicitly incorporate prior knowledge in the model development and is robust to missing data in the output measurements. The proposed prediction model is based on a latent variables model to extract glycemic dynamics from highly-correlated data and incorporates prior knowledge of exponential stability to improve the prediction ability. Missing data structures are formulated to enable model predictions when output measurements are missing for short periods of time. Based on the latent variables model, the MPC strategy and adaptive rules are developed to automatically tune the aggressiveness of the MPC. The adaptive prior-knowledge-informed MPC is evaluated with computer simulations for the control of blood glucose concentrations in people with Type 1 diabetes (T1D) using simulated virtual patients. Due to the variability among people with T1D, the hyperparameters of the prior-knowledge-informed model are personalized to individual subjects. The percentage of time spent in the target range is 76.48% when there are no missing data and 76.52% when there are missing data episodes lasting up to 30 mins (6 samples). Incorporating the adaptive rules further improves the percentage of time in target range to 84.58% and 84.88% for cases with no missing data and missing data, respectively. The proposed adaptive prior-informed MPC formulation provides robust, effective, and safe regulation of glucose concentration in T1D despite disturbances and missing measurements.

Personalized Blood Glucose Prediction for Type 1 Diabetes Using Evidential Deep Learning and Meta-Learning

The availability of large amounts of data from continuous glucose monitoring (CGM), together with the latest advances in deep learning techniques, have opened the door to a new paradigm of algorithm design for personalized blood glucose (BG) prediction in type 1 diabetes (T1D) with superior performance. However, there are several challenges that prevent the widespread implementation of deep learning algorithms in actual clinical settings, including unclear prediction confidence and limited training data for new T1D subjects. To this end, we propose a novel deep learning framework, Fast-adaptive and Confident Neural Network (FCNN), to meet these clinical challenges. In particular, an attention-based recurrent neural network is used to learn representations from CGM input and forward a weighted sum of hidden states to an evidential output layer, aiming to compute personalized BG predictions with theoretically supported model confidence. The model-agnostic meta-learning is employed to enable fast adaptation for a new T1D subject with limited training data. The proposed framework has been validated on three clinical datasets. In particular, for a dataset including 12 subjects with T1D, FCNN achieved a root mean square error of 18.64±2.60 mg/dL and 31.07±3.62 mg/dL for 30 and 60-minute prediction horizons, respectively, which outperformed all the considered baseline methods with significant improvements. These results indicate that FCNN is a viable and effective approach for predicting BG levels in T1D. The well-trained models can be implemented in smartphone apps to improve glycemic control by enabling proactive actions through real-time glucose alerts.

Review: Machine learning in precision pharmacotherapy of type 2 diabetes-A promising future or a glimpse of hope?

Precision pharmacotherapy of diabetes requires judicious selection of the optimal therapeutic agent for individual patients. Artificial intelligence (AI), a swiftly expanding discipline, holds substantial potential to transform current practices in diabetes diagnosis and management. This manuscript provides a comprehensive review of contemporary research investigating drug responses in patient subgroups, stratified via either supervised or unsupervised machine learning approaches. The prevalent algorithmic workflow for investigating drug responses using machine learning involves cohort selection, data processing, predictor selection, development and validation of machine learning methods, subgroup allocation, and subsequent analysis of drug response. Despite the promising feature, current research does not yet provide sufficient evidence to implement machine learning algorithms into routine clinical practice, due to a lack of simplicity, validation, or demonstrated efficacy. Nevertheless, we anticipate that the evolving evidence base will increasingly substantiate the role of machine learning in molding precision pharmacotherapy for diabetes.

Intelligent Control with Artificial Neural Networks for Automated Insulin Delivery Systems

Type 1 diabetes mellitus is a disease that affects millions of people around the world. Recent progress in embedded devices has allowed the development of artificial pancreas that can pump insulin subcutaneously to automatically regulate blood glucose levels in diabetic patients. In this work, a Lyapunov-based intelligent controller using artificial neural networks is proposed for application in automated insulin delivery systems. The adoption of an adaptive radial basis function network within the control scheme allows regulation of blood glucose levels without the need for a dynamic model of the system. The proposed model-free approach does not require the patient to inform when they are going to have a meal and is able to deal with inter- and intrapatient variability. To ensure safe operating conditions, the stability of the control law is rigorously addressed through a Lyapunov-like analysis. In silico analysis using virtual patients are provided to demonstrate the effectiveness of the proposed control scheme, showing its ability to maintain normoglycemia in patients with type 1 diabetes mellitus. Three different scenarios were considered: one long- and two short-term simulation studies. In the short-term analyses, 20 virtual patients were simulated for a period of 7 days, with and without prior basal therapy, while in the long-term simulation, 1 virtual patient was assessed over 63 days. The results show that the proposed approach was able to guarantee a time in the range above 95% for the target glycemia in all scenarios studied, which is in fact well above the desirable 70%. Even in the long-term analysis, the intelligent control scheme was able to keep blood glucose metrics within clinical care standards: mean blood glucose of 119.59 mg/dL with standard deviation of 32.02 mg/dL and coefficient of variation of 26.78%, all below the respective reference values.

A Reinforcement Learning Based System for Blood Glucose Control without Carbohydrate Estimation in Type 1 Diabetes: In Silico Validation

Type 1 Diabetes (T1D) is a chronic autoimmune disease, which requires the use of exogenous insulin for glucose regulation. In current hybrid closed-loop systems, meal entry is manual which adds cognitive burden to the persons living with T1D. In this study, we proposed a control system based on Proximal Policy Optimisation (PPO) that controls both basal and bolus insulin infusion and only requires meal announcement, thus eliminating the need for carbohydrate estimation. We evaluated the system on a challenging meal scenario, using an open-source simulator based on the UVA/Padova 2008 model and achieved a mean Time in Range value of 65% for the adult subject cohort, while maintaining a moderate hypoglycemic and hyperglycemic risk profile. The approach shows promise and welcomes further research towards the translation to a real-life artificial pancreas. Clinical relevance- This was an in-silico analysis towards the development of an autonomous artificial pancreas system for glucose control. The proposed system show promise in eliminating the need for estimating the carbohydrate content in meals.

Enhancing self-management in type 1 diabetes with wearables and deep learning

People living with type 1 diabetes (T1D) require lifelong self-management to maintain glucose levels in a safe range. Failure to do so can lead to adverse glycemic events with short and long-term complications. Continuous glucose monitoring (CGM) is widely used in T1D self-management for real-time glucose measurements, while smartphone apps are adopted as basic electronic diaries, data visualization tools, and simple decision support tools for insulin dosing. Applying a mixed effects logistic regression analysis to the outcomes of a six-week longitudinal study in 12 T1D adults using CGM and a clinically validated wearable sensor wristband (NCT ID: NCT03643692), we identified several significant associations between physiological measurements and hypo- and hyperglycemic events measured an hour later. We proceeded to develop a new smartphone-based platform, ARISES (Adaptive, Real-time, and Intelligent System to Enhance Self-care), with an embedded deep learning algorithm utilizing multi-modal data from CGM, daily entries of meal and bolus insulin, and the sensor wristband to predict glucose levels and hypo- and hyperglycemia. For a 60-minute prediction horizon, the proposed algorithm achieved the average root mean square error (RMSE) of 35.28 ± 5.77 mg/dL with the Matthews correlation coefficients for detecting hypoglycemia and hyperglycemia of 0.56 ± 0.07 and 0.70 ± 0.05, respectively. The use of wristband data significantly reduced the RMSE by 2.25 mg/dL (p < 0.01). The well-trained model is implemented on the ARISES app to provide real-time decision support. These results indicate that the ARISES has great potential to mitigate the risk of severe complications and enhance self-management for people with T1D.

Generation of post-meal insulin correction boluses in type 1 diabetes simulation models for in-silico clinical trials: More realistic scenarios obtained using a decision tree approach

BACKGROUND AND OBJECTIVE

In type 1 diabetes (T1D) research, in-silico clinical trials (ISCTs) notably facilitate the design/testing of new therapies. Published simulation tools embed mathematical models of blood glucose (BG) and insulin dynamics, continuous glucose monitoring (CGM) sensors, and insulin treatments, but lack a realistic description of some aspects of patient lifestyle impacting on glucose control. Specifically, to effectively simulate insulin correction boluses, required to treat post-meal hyperglycemia (BG > 180 mg/dL), the timing of the bolus may be influenced by subjects' behavioral attitudes. In this work, we develop an easily interpretable model of the variability of correction bolus timing observed in real data, and embed it into a popular simulation tool for ISCTs.

METHODS

Using data collected in 196 adults with T1D monitored in free-living conditions, we trained a decision tree (DT) model to classify whether a correction bolus is injected in a future time window, based on predictors collected back in time, related to CGM data, previous insulin boluses and subject's characteristics. The performance was compared to that of a logistic regression classifier with LASSO regularization (LC), trained on the same dataset. After validation, the DT was embedded within a popular T1D simulation tool and an ISCT was performed to compare the simulated correction boluses against those observed in a subset of data not used for model training.

RESULTS

The DT provided better classification performance (accuracy: 0.792, sensitivity: 0.430, specificity: 0.878, precision: 0.455) than the LC and presented good interpretability. The most predictive features were related to CGM (and its temporal variations), time since the last insulin bolus, and time of the day. The correction boluses simulated by the DT, after implementation in the simulation tool, showed a good agreement with real-world data.

CONCLUSIONS

The DT developed in this work represents a simple set of rules to mimic the same timing of correction boluses observed on real data. The inclusion of the model in simulation tools allows investigators to perform ISCTs that more realistically represent the patient behavior in taking correction boluses and the post-prandial BG response. In the future, more complex models can be investigated.

Automatic Artificial Pancreas Systems Using an Intelligent Multiple-Model PID Strategy

In this paper, an individualized intelligent multiple-model technique is proposed to design automatic artificial pancreas (AP) systems for the glycemic regulation of type 1 diabetic patients. At first, using the multiple-model concept, the insulin-glucose regulatory system is mathematically identified by constructing some local models. In this step, trade-offs between the number of local models and the complexity of the overall closed-loop system are made by defining and solving a bi-objective optimization problem. Then, optimal AP systems are designed by tuning a bank of proportionalintegralderivative (PID) controllers via the genetic algorithm (GA). A fuzzy gain scheduling strategy is employed to determine the participation percentages of the PID controllers in the control action. Finally, two safety mechanisms, called insulin on board (IOB) constraint and pump shut-off, are installed in the AP systems to enhance their performance. To assess the proposed AP systems, in silico experiments are performed on virtual patients of the UVA/Padova metabolic simulator. The obtained results reveal that the proposed intelligent multiple-model methodology leads to AP systems with limited hyperglycemia and no severe hypoglycemia.

Deep Learning for Diabetes: A Systematic Review

Diabetes is a chronic metabolic disorder that affects an estimated 463 million people worldwide. Aiming to improve the treatment of people with diabetes, digital health has been widely adopted in recent years and generated a huge amount of data that could be used for further management of this chronic disease. Taking advantage of this, approaches that use artificial intelligence and specifically deep learning, an emerging type of machine learning, have been widely adopted with promising results. In this paper, we present a comprehensive review of the applications of deep learning within the field of diabetes. We conducted a systematic literature search and identified three main areas that use this approach: diagnosis of diabetes, glucose management, and diagnosis of diabetes-related complications. The search resulted in the selection of 40 original research articles, of which we have summarized the key information about the employed learning models, development process, main outcomes, and baseline methods for performance evaluation. Among the analyzed literature, it is to be noted that various deep learning techniques and frameworks have achieved state-of-the-art performance in many diabetes-related tasks by outperforming conventional machine learning approaches. Meanwhile, we identify some limitations in the current literature, such as a lack of data availability and model interpretability. The rapid developments in deep learning and the increase in available data offer the possibility to meet these challenges in the near future and allow the widespread deployment of this technology in clinical settings.

Reinforcement Learning in Neurocritical and Neurosurgical Care: Principles and Possible Applications

Dynamic decision-making was essential in the clinical care of surgical patients. Reinforcement learning (RL) algorithm is a computational method to find sequential optimal decisions among multiple suboptimal options. This review is aimed at introducing RL's basic concepts, including three basic components: the state, the action, and the reward. Most medical studies using reinforcement learning methods were trained on a fixed observational dataset. This paper also reviews the literature of existing practical applications using reinforcement learning methods, which can be further categorized as a statistical RL study and a computational RL study. The review proposes several potential aspects where reinforcement learning can be applied in neurocritical and neurosurgical care. These include sequential treatment strategies of intracranial tumors and traumatic brain injury and intraoperative endoscope motion control. Several limitations of reinforcement learning are representations of basic components, the positivity violation, and validation methods.

Machine Learning Techniques for Hypoglycemia Prediction: Trends and Challenges

(1) Background: the use of machine learning techniques for the purpose of anticipating hypoglycemia has increased considerably in the past few years. Hypoglycemia is the drop in blood glucose below critical levels in diabetic patients. This may cause loss of cognitive ability, seizures, and in extreme cases, death. In almost half of all the severe cases, hypoglycemia arrives unannounced and is essentially asymptomatic. The inability of a diabetic patient to anticipate and intervene the occurrence of a hypoglycemic event often results in crisis. Hence, the prediction of hypoglycemia is a vital step in improving the life quality of a diabetic patient. The objective of this paper is to review work performed in the domain of hypoglycemia prediction by using machine learning and also to explore the latest trends and challenges that the researchers face in this area; (2) Methods: literature obtained from PubMed and Google Scholar was reviewed. Manuscripts from the last five years were searched for this purpose. A total of 903 papers were initially selected of which 57 papers were eventually shortlisted for detailed review; (3) Results: a thorough dissection of the shortlisted manuscripts provided an interesting split between the works based on two categories: hypoglycemia prediction and hypoglycemia detection. The entire review was carried out keeping this categorical distinction in perspective while providing a thorough overview of the machine learning approaches used to anticipate hypoglycemia, the type of training data, and the prediction horizon.

An Insulin Bolus Advisor for Type 1 Diabetes Using Deep Reinforcement Learning

(1) Background: People living with type 1 diabetes (T1D) require self-management to maintain blood glucose (BG) levels in a therapeutic range through the delivery of exogenous insulin. However, due to the various variability, uncertainty and complex glucose dynamics, optimizing the doses of insulin delivery to minimize the risk of hyperglycemia and hypoglycemia is still an open problem. (2) Methods: In this work, we propose a novel insulin bolus advisor which uses deep reinforcement learning (DRL) and continuous glucose monitoring to optimize insulin dosing at mealtime. In particular, an actor-critic model based on deep deterministic policy gradient is designed to compute mealtime insulin doses. The proposed system architecture uses a two-step learning framework, in which a population model is first obtained and then personalized by subject-specific data. Prioritized memory replay is adopted to accelerate the training process in clinical practice. To validate the algorithm, we employ a customized version of the FDA-accepted UVA/Padova T1D simulator to perform in silico trials on 10 adult subjects and 10 adolescent subjects. (3) Results: Compared to a standard bolus calculator as the baseline, the DRL insulin bolus advisor significantly improved the average percentage time in target range (70-180 mg/dL) from 74.1%±8.4% to 80.9%±6.9% (p<0.01) and 54.9%±12.4% to 61.6%±14.1% (p<0.01) in the the adult and adolescent cohorts, respectively, while reducing hypoglycemia. (4) Conclusions: The proposed algorithm has the potential to improve mealtime bolus insulin delivery in people with T1D and is a feasible candidate for future clinical validation.