Optimal treatment recommendations for diabetes patients using the Markov decision process along with the South Korean electronic health records

Methods in dynamic treatment regimens using observational healthcare data: A systematic review

We present a systematic review of methods used to estimate Dynamic Treatment Regimens (DTR) using observational healthcare data and provide a brief summary of their strengths and weaknesses, evaluation metrics, and suitable research problem settings. We considered all observational studies identified in PubMed or EMBASE between January 1950 until January 2022, including only studies that evaluated medical treatments or interventions as exposure and/or outcome in patients and where DTRs were estimated. 83 studies met our inclusion criteria; 44.6% estimating DTR utilizing reinforcement learning, 18.1% utilizing counterfactual-based models, 12.1% utilizing classification-based methods, and 9.6% utilized g-methods. Among the studies analyzed, 28.9% aimed to replicate human expert DTRs, while 71.1% aimed to refine and improve existing DTRs. Approximately two-thirds of studies (65.1%) reported the assumptions required for their applied methods, such as exchangeability, positivity, consistency, and Markov property. Most of the studies (83.1%) estimated DTRs with more than two treatment options; 50.6% mentioned time-varying confounders, only a few estimated conditional average treatment effects (7.2%). Most (85.5%) validated their methods, with 32.5% using expected outcomes (e.g., survival rates), 26.5% employing simulated data, and 25.3% conducting direct comparisons with observational data.

A drug mix and dose decision algorithm for individualized type 2 diabetes management

Pharmacotherapy guidelines for type 2 diabetes (T2D) emphasize patient-centered care, but applying this approach effectively in outpatient practice remains challenging. Data-driven treatment optimization approaches could enhance individualized T2D management, but current approaches cannot account for drug-specific and dose-dependent variations in safety and efficacy. We developed and evaluated an AI Drug mix and dose Advisor (AIDA) for glycemic management, using electronic medical records from 107,854 T2D patients in the SingHealth Diabetes Registry. Given a patient's medical profile, AIDA leverages a predict-then-optimize approach to identify the minimal drug mix and dose changes required to optimize glycemic control, subject to clinical knowledge-based guidelines. On unseen data from large internal, external, and temporal validation sets, AIDA recommendations were estimated to improve post-visit glycated hemoglobin (HbA1c) by an average of 0.40-0.68% over standard of care (P < 0.0001). In qualitative evaluations on 60 diverse cases by a panel of three endocrinologists, AIDA recommendations were mostly rated as reasonable and precise. Finally, AIDA's ability to account for drug-dose specifics offered several advantages over competing methods, including greater consistency with practice preferences and clinical guidelines for practical but effective options, indication-based treatments, and renal dosing. As AIDA provides drug-dose recommendations to improve outcomes for individual T2D patients, it could be used for clinical decision support at point-of-care, especially in resource-limited settings.

Identifying Patients at Risk for Alcohol-Exposed Pregnancies: The Importance of Addressing Multiple Risk Factors

BACKGROUND

The increasing prevalence of fetal alcohol spectrum disorders is a critical public health issue. Two behaviors, consuming alcohol and using less effective pregnancy prevention, may result in alcohol-exposed pregnancies (AEPs) in individuals who can become pregnant. In the context of alcohol screening and brief intervention (SBI) services, cutoff scores on widely used alcohol risk assessments (eg, Alcohol Use Disorders Identification Test, U.S. version [USAUDIT]) may fail to identify individuals whose relatively low alcohol consumption may still put them at risk for an AEP due to their pregnancy prevention method.

METHODS

To identify this gap in alcohol SBI service delivery, we examined data from 2 reproductive healthcare systems implementing alcohol SBI, to explore the prevalence of individuals who met both of the following risk conditions: reported any alcohol use on the USAUDIT and a pregnancy prevention method less than 88% effective. Electronic health records for individuals aged 18 to 49 presenting for preventive care in 2021 were analyzed.

RESULTS

Of 11 567 screened, 7638 reported some alcohol use, but screened at a lower-risk level and were not flagged to receive an alcohol-focused brief intervention (BI). Of these, 1477 were using a method of pregnancy prevention that was less than 88% effective. In addition, 118 of the 1676 who screened positive on the USAUDIT were using less effective contraception and did not receive a BI. In summary, the number of individuals at risk of an AEP who did not receive an alcohol BI was 1595 (13.8%) of the total patients screened for at-risk alcohol use.

CONCLUSIONS

There is a need for system modifications to assess multiple behaviors simultaneously and alert providers when a combination of behaviors increases a specific health risk, such as an AEP. Tailored alcohol BIs that include the risks/benefits of various pregnancy prevention methods to reduce AEPs provide opportunities to enhance the reach of standard alcohol SBI services.

Data-driven meal events detection using blood glucose response patterns

BACKGROUND

In the Diabetes domain, events such as meals and exercises play an important role in the disease management. For that, many studies focus on automatic meal detection, specially as part of the so-called artificial [Formula: see text]-cell systems. Meals are associated to blood glucose (BG) variations, however such variations are not peculiar to meals, it mostly comes as a combination of external factors. Thus, general approaches such as the ones focused on glucose signal rate of change are not enough to detect personalized influence of such factors. By using a data-driven individualized approach for meal detection, our method is able to fit real data, detecting personalized meal responses even when such external factors are implicitly present.

METHODS

The method is split into model training and selection. In the training phase, we start observing meal responses for each individual, and identifying personalized patterns. Occurrences of such patterns are searched over the BG signal, evaluating the similarity of each pattern to each possible signal subsequence. The most similar occurrences are then selected as possible meal event candidates. For that, we include steps for excluding less relevant neighbors per pattern, and grouping close occurrences in time globally. Each candidate is represented by a set of time and response signal related qualitative variables. These variables are used as input features for different binary classifiers in order to learn to classify a candidate as MEAL or NON-MEAL. In the model selection phase, we compare all trained classifiers to select the one that performs better with the data of each individual.

RESULTS

The results show that the method is able to detect daily meals, providing a result with a balanced proportion between detected meals and false alarms. The analysis on multiple patients indicate that the approach achieves good outcomes when there is enough reliable training data, as this is reflected on the testing results.

CONCLUSIONS

The approach aims at personalizing the meal detection task by relying solely on data. The premise is that a model trained with data that contains the implicit influence of external factors is able to recognize the nuances of the individual that generated the data. Besides, the approach can also be used to improve data quality by detecting meals, opening opportunities to possible applications such as detecting and reminding users of missing or wrongly informed meal events.

Diabetes medication recommendation system using patient similarity analytics

Type-2 diabetes mellitus (T2DM) is a medical condition in which oral medications avail to patients to curb their hyperglycaemia after failed dietary therapy. However, individual responses to the prescribed pharmacotherapy may differ due to their clinical profiles, comorbidities, lifestyles and medical adherence. One approach is to identify similar patients within the same community to predict their likely response to the prescribed diabetes medications. This study aims to present an evidence-based diabetes medication recommendation system (DMRS) underpinned by patient similarity analytics. The DMRS was developed using 10-year electronic health records of 54,933 adult patients with T2DM from six primary care clinics in Singapore. Multiple clinical variables including patient demographics, comorbidities, laboratory test results, existing medications, and trajectory patterns of haemoglobin A1c (HbA1c) were used to identify similar patients. The DMRS was evaluated on four groups of patients with comorbidities such as hyperlipidaemia (HLD) and hypertension (HTN). Recommendations were assessed using hit ratio which represents the percentage of patients with at least one recommended sets of medication matches exactly the diabetes prescriptions in both the type and dosage. Recall, precision, and mean reciprocal ranking of the recommendation against the diabetes prescriptions in the EHR records were also computed. Evaluation against the EHR prescriptions revealed that the DMRS recommendations can achieve hit ratio of 81% for diabetes patients with no comorbidity, 84% for those with HLD, 78% for those with HTN, and 75% for those with both HLD and HTN. By considering patients' clinical profiles and their trajectory patterns of HbA1c, the DMRS can provide an individualized recommendation that resembles the actual prescribed medication and dosage. Such a system is useful as a shared decision-making tool to assist clinicians in selecting the appropriate medications for patients with T2DM.

An interpretable RL framework for pre-deployment modeling in ICU hypotension management

Computational methods from reinforcement learning have shown promise in inferring treatment strategies for hypotension management and other clinical decision-making challenges. Unfortunately, the resulting models are often difficult for clinicians to interpret, making clinical inspection and validation of these computationally derived strategies challenging in advance of deployment. In this work, we develop a general framework for identifying succinct sets of clinical contexts in which clinicians make very different treatment choices, tracing the effects of those choices, and inferring a set of recommendations for those specific contexts. By focusing on these few key decision points, our framework produces succinct, interpretable treatment strategies that can each be easily visualized and verified by clinical experts. This interrogation process allows clinicians to leverage the model’s use of historical data in tandem with their own expertise to determine which recommendations are worth investigating further e.g. at the bedside. We demonstrate the value of this approach via application to hypotension management in the ICU, an area with critical implications for patient outcomes that lacks data-driven individualized treatment strategies; that said, our framework has broad implications on how to use computational methods to assist with decision-making challenges on a wide range of clinical domains.

A Promising Approach to Optimizing Sequential Treatment Decisions for Depression: Markov Decision Process

The most appropriate next step in depression treatment after the initial treatment fails is unclear. This study explores the suitability of the Markov decision process for optimizing sequential treatment decisions for depression. We conducted a formal comparison of a Markov decision process approach and mainstream state-transition models as used in health economic decision analysis to clarify differences in the model structure. We performed two reviews: the first to identify existing applications of the Markov decision process in the field of healthcare and the second to identify existing health economic models for depression. We then illustrated the application of a Markov decision process by reformulating an existing health economic model. This provided input for discussing the suitability of a Markov decision process for solving sequential treatment decisions in depression. The Markov decision process and state-transition models differed in terms of flexibility in modeling actions and rewards. In all, 23 applications of a Markov decision process within the context of somatic disease were included, 16 of which concerned sequential treatment decisions. Most existing health economic models relating to depression have a state-transition structure. The example application replicated the health economic model and enabled additional capacity to make dynamic comparisons of more interventions over time than was possible with traditional state-transition models. Markov decision processes have been successfully applied to address sequential treatment-decision problems, although the results have been published mostly in economics journals that are not related to healthcare. One advantage of a Markov decision process compared with state-transition models is that it allows extended action space: the possibility of making dynamic comparisons of different treatments over time. Within the context of depression, although existing state-transition models are too basic to evaluate sequential treatment decisions, the assumptions of a Markov decision process could be satisfied. The Markov decision process could therefore serve as a powerful model for optimizing sequential treatment in depression. This would require a sufficiently elaborate state-transition model at the cohort or patient level.

Precision Medicine for Hypertension Patients with Type 2 Diabetes via Reinforcement Learning

Precision medicine is a new approach to understanding health and disease based on patient-specific data such as medical diagnoses; clinical phenotype; biologic investigations such as laboratory studies and imaging; and environmental, demographic, and lifestyle factors. The importance of machine learning techniques in healthcare has expanded quickly in the last decade owing to the rising availability of vast multi-modality data and developed computational models and algorithms. Reinforcement learning is an appealing method for developing efficient policies in various healthcare areas where the decision-making process is typically defined by a long period or a sequential process. In our research, we leverage the power of reinforcement learning and electronic health records of South Koreans to dynamically recommend treatment prescriptions, which are personalized based on patient information of hypertension. Our proposed reinforcement learning-based treatment recommendation system decides whether to use mono, dual, or triple therapy according to the state of the hypertension patients. We evaluated the performance of our personalized treatment recommendation model by lowering the occurrence of hypertension-related complications and blood pressure levels of patients who followed our model’s recommendation. With our findings, we believe that our proposed hypertension treatment recommendation model could assist doctors in prescribing appropriate antihypertensive medications.