Machine Learning Approach to Decision Making for Insulin Initiation in Japanese Patients With Type 2 Diabetes (JDDM 58): Model Development and Validation Study

Background

Applications of machine learning for the early detection of diseases for which a clear-cut diagnostic gold standard exists have been evaluated. However, little is known about the usefulness of machine learning approaches in the decision-making process for decisions such as insulin initiation by diabetes specialists for which no absolute standards exist in clinical settings.

Objective

The objectives of this study were to examine the ability of machine learning models to predict insulin initiation by specialists and whether the machine learning approach could support decision making by general physicians for insulin initiation in patients with type 2 diabetes.

Methods

Data from patients prescribed hypoglycemic agents from December 2009 to March 2015 were extracted from diabetes specialists’ registries, resulting in a sample size of 4860 patients who had received initial monotherapy with either insulin (n=293) or noninsulin (n=4567). Neural network output was insulin initiation ranging from 0 to 1 with a cutoff of >0.5 for the dichotomous classification. Accuracy, recall, and area under the receiver operating characteristic curve (AUC) were calculated to compare the ability of machine learning models to make decisions regarding insulin initiation to the decision-making ability of logistic regression and general physicians. By comparing the decision-making ability of machine learning and logistic regression to that of general physicians, 7 cases were chosen based on patient information as the gold standard based on the agreement of 8 of the 9 specialists.

Results

The AUCs, accuracy, and recall of logistic regression were higher than those of machine learning (AUCs of 0.89-0.90 for logistic regression versus 0.67-0.74 for machine learning). When the examination was limited to cases receiving insulin, discrimination by machine learning was similar to that of logistic regression analysis (recall of 0.05-0.68 for logistic regression versus 0.11-0.52 for machine learning). Accuracies of logistic regression, a machine learning model (downsampling ratio of 1:8), and general physicians were 0.80, 0.70, and 0.66, respectively, for 43 randomly selected cases. For the 7 gold standard cases, the accuracies of logistic regression and the machine learning model were 1.00 and 0.86, respectively, with a downsampling ratio of 1:8, which were higher than the accuracy of general physicians (ie, 0.43).

Conclusions

Although we found no superior performance of machine learning over logistic regression, machine learning had higher accuracy in prediction of insulin initiation than general physicians, defined by diabetes specialists’ choice of the gold standard. Further study is needed before the use of machine learning–based decision support systems for insulin initiation can be incorporated into clinical practice.

Comparison of baseline characteristics and clinical course in Japanese patients with type 2 diabetes among whom different types of oral hypoglycemic agents were chosen by diabetes specialists as initial monotherapy (JDDM 42)

Little is known about the relationships between patient factors and the antihyperglycemic agents that have been prescribed as initial therapy by diabetes specialists for patients with type 2 diabetes. Moreover, there has been little clarification of the subsequent usage patterns and related factors that influenced the continuation or discontinuation of the drug or the addition of another drug. To provide information on these issues, we evaluated the clinical characteristics of Japanese patients with type 2 diabetes for whom different types of oral hypoglycemic agents (i.e., either sulfonylureas, biguanides, or DPP-4 inhibitors (DPP-4Is)) were chosen as initial monotherapy by diabetes specialists and evaluated subsequent usage patterns.Prescription data on 3 different antidiabetic agents from December 2009 to March 2015 from diabetes specialists' patient registries were used to identify variables at baseline related to initial prescriptions; also, the addition of another hypoglycemic drug or discontinuation of the initial therapy was evaluated 1 year after the initial prescription. Analyzed were data on 2666 patients who received initial monotherapy with either a sulfonylurea (305 patients), biguanide (951 patients), or DPP-4I (1410 patients). Patients administered sulfonylureas were older, had a lower body mass index (BMI), longer duration of diabetes, and worse glycemic control than recipients of biguanides. Use of biguanides was related to younger age, short duration of diabetes, and obesity but was negatively associated with poor glycemic control. Older age but neither obesity nor poor glycemic control was associated with DPP-4Is. In all 3 groups a high HbA1c value was related to adding another hypoglycemic agent to the initial therapy. Moreover, adding another drug to a DPP-4I was related to a younger age and higher BMI.Patients' age, duration of diabetes, obesity, and glycemic control at baseline influenced the choice of hypoglycemic agents. Selection of a biguanide differs greatly from that of a sulfonylurea or DPP-4I with regard to age and obesity.