Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting

An Appraisal of the Quality of Development and Reporting of Predictive Models in Neurosurgery: A Systematic Review

BACKGROUND AND OBJECTIVES

Significant evidence has indicated that the reporting quality of novel predictive models is poor because of confounding by small data sets, inappropriate statistical analyses, and a lack of validation and reproducibility. The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement was developed to increase the generalizability of predictive models. This study evaluated the quality of predictive models reported in neurosurgical literature through their compliance with the TRIPOD guidelines.

METHODS

Articles reporting prediction models published in the top 5 neurosurgery journals by SCImago Journal Rank-2 (Neurosurgery, Journal of Neurosurgery, Journal of Neurosurgery: Spine, Journal of NeuroInterventional Surgery, and Journal of Neurology, Neurosurgery, and Psychiatry) between January 1st, 2018, and January 1st, 2023, were identified through a PubMed search strategy that combined terms related to machine learning and prediction modeling. These original research articles were analyzed against the TRIPOD criteria.

RESULTS

A total of 110 articles were assessed with the TRIPOD checklist. The median compliance was 57.4% (IQR: 50.0%-66.7%). Models using machine learning-based models exhibited lower compliance on average compared with conventional learning-based models (57.1%, 50.0%-66.7% vs 68.1%, 50.2%-68.1%, P = .472). Among the TRIPOD criteria, the lowest compliance was observed in blinding the assessment of predictors and outcomes (n = 7, 12.7% and n = 10, 16.9%, respectively), including an informative title (n = 17, 15.6%) and reporting model performance measures such as confidence intervals (n = 27, 24.8%). Few studies provided sufficient information to allow for the external validation of results (n = 26, 25.7%).

CONCLUSION

Published predictive models in neurosurgery commonly fall short of meeting the established guidelines laid out by TRIPOD for optimal development, validation, and reporting. This lack of compliance may represent the minor extent to which these models have been subjected to external validation or adopted into routine clinical practice in neurosurgery.

Development, validation and visualization of a web-based nomogram for predicting risk of new-onset diabetes after percutaneous coronary intervention

Simple and practical tools for screening high-risk new-onset diabetes after percutaneous coronary intervention (PCI) (NODAP) are urgently needed to improve post-PCI prognosis. We aimed to evaluate the risk factors for NODAP and develop an online prediction tool using conventional variables based on a multicenter database. China evidence-based Chinese medicine database consisted of 249, 987 patients from 4 hospitals in mainland China. Patients ≥ 18 years with implanted coronary stents for acute coronary syndromes and did not have diabetes before PCI were enrolled in this study. According to the occurrence of new-onset diabetes mellitus after PCI, the patients were divided into NODAP and Non-NODAP. After least absolute shrinkage and selection operator regression and logistic regression, the model features were selected and then the nomogram was developed and plotted. Model performance was evaluated by the receiver operating characteristic curve, calibration curve, Hosmer-Lemeshow test and decision curve analysis. The nomogram was also externally validated at a different hospital. Subsequently, we developed an online visualization tool and a corresponding risk stratification system to predict the risk of developing NODAP after PCI based on the model. A total of 2698 patients after PCI (1255 NODAP and 1443 non-NODAP) were included in the final analysis based on the multicenter database. Five predictors were identified after screening: fasting plasma glucose, low-density lipoprotein cholesterol, hypertension, family history of diabetes and use of diuretics. And then we developed a web-based nomogram ( https://mr.cscps.com.cn/wscoringtool/index.html ) incorporating the above conventional factors for predicting patients at high risk for NODAP. The nomogram showed good discrimination, calibration and clinical utility and could accurately stratify patients into different NODAP risks. We developed a simple and practical web-based nomogram based on multicenter database to screen for NODAP risk, which can assist clinicians in accurately identifying patients at high risk of NODAP and developing post-PCI management strategies to improved patient prognosis.

Practical guide to building machine learning-based clinical prediction models using imbalanced datasets

Clinical prediction models often aim to predict rare, high-risk events, but building such models requires robust understanding of imbalance datasets and their unique study design considerations. This practical guide highlights foundational prediction model principles for surgeon-data scientists and readers who encounter clinical prediction models, from feature engineering and algorithm selection strategies to model evaluation and design techniques specific to imbalanced datasets. We walk through a clinical example using readable code to highlight important considerations and common pitfalls in developing machine learning-based prediction models. We hope this practical guide facilitates developing and critically appraising robust clinical prediction models for the surgical community.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models

OBJECTIVES

To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique.

STUDY DESIGN AND SETTING

We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed.

RESULTS

This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples.

CONCLUSION

We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

Cardiometabolic Disease Staging and Major Adverse Cardiovascular Event Prediction in 2 Prospective Cohorts

BACKGROUND

Cardiometabolic risk prediction models that incorporate metabolic syndrome traits to predict cardiovascular outcomes may help identify high-risk populations early in the progression of cardiometabolic disease.

OBJECTIVES

The purpose of this study was to examine whether a modified cardiometabolic disease staging (CMDS) system, a validated diabetes prediction model, predicts major adverse cardiovascular events (MACE).

METHODS

We developed a predictive model using data accessible in clinical practice [fasting glucose, blood pressure, body mass index, cholesterol, triglycerides, smoking status, diabetes status, hypertension medication use] from the REGARDS (REasons for Geographic And Racial Differences in Stroke) study to predict MACE [cardiovascular death, nonfatal myocardial infarction, and/or nonfatal stroke]. Predictive performance was assessed using receiver operating characteristic curves, mean squared errors, misclassification, and area under the curve (AUC) statistics.

RESULTS

Among 20,234 REGARDS participants with no history of stroke or myocardial infarction (mean age 64 ± 9.3 years, 58% female, 41% non-Hispanic Black, and 18% diabetes), 2,695 developed incident MACE (13.3%) during a median 10-year follow-up. The CMDS development model in REGARDS for MACE had an AUC of 0.721. Our CMDS model performed similarly to both the ACC/AHA 10-year risk estimate (AUC 0.721 vs 0.716) and the Framingham risk score (AUC 0.673).

CONCLUSIONS

The CMDS predicted the onset of MACE with good predictive ability and performed similarly or better than 2 commonly known cardiovascular disease prediction risk tools. These data underscore the importance of insulin resistance as a cardiovascular disease risk factor and that CMDS can be used to identify individuals at high risk for progression to cardiovascular disease.

Development of a prediction equation to estimate lower-limb arterial occlusion pressure with a thigh sphygmomanometer

INTRODUCTION

Previous investigators have developed prediction equations to estimate arterial occlusion pressure (AOP) for blood flow restriction (BFR) exercise. Most equations have not been validated and are designed for use with expensive cuff systems. Thus, their implementation is limited for practitioners.

PURPOSE

To develop and validate an equation to predict AOP in the lower limbs when applying an 18 cm wide thigh sphygmomanometer (SPHYG18cm).

METHODS

Healthy adults (n = 143) underwent measures of thigh circumference (TC), skinfold thickness (ST), and estimated muscle cross-sectional area (CSA) along with brachial and femoral systolic (SBP) and diastolic (DBP) blood pressure. Lower-limb AOP was assessed in a seated position at the posterior tibial artery (Doppler ultrasound) using a SPHYG18cm. Hierarchical linear regression models were used to determine predictors of AOP. The best set of predictors was used to construct a prediction equation to estimate AOP. Performance of the equation was evaluated and internally validated using bootstrap resampling.

RESULTS

Models containing measures of either TC or thigh composition (ST and CSA) paired with brachial blood pressures explained the most variability in AOP (54%) with brachial SBP accounting for majority of explained variability. A prediction equation including TC, brachial SBP, and age showed good predictability (R2 = 0.54, RMSE = 7.18 mmHg) and excellent calibration. Mean difference between observed and predicted values was 0.0 mmHg and 95% Limits of Agreement were ± 18.35 mmHg. Internal validation revealed small differences between apparent and optimism adjusted performance measures, suggesting good generalizability.

CONCLUSION

This prediction equation for use with a SPHYG18cm provided a valid way to estimate lower-limb AOP without expensive equipment.

Pros and cons of live kidney donation in prediabetics: A critical review and way forward

There is shortage of organs, including kidneys, worldwide. Along with deceased kidney transplantation, there is a significant rise in live kidney donation. The prevalence of prediabetes (PD), including impaired fasting glucose and impaired glucose tolerance, is on the rise across the globe. Transplant teams frequently come across prediabetic kidney donors for evaluation. Prediabetics are at risk of diabetes, chronic kidney disease, cardiovascular events, stroke, neuropathy, retinopathy, dementia, depression and nonalcoholic liver disease along with increased risk of all-cause mortality. Unfortunately, most of the studies done in prediabetic kidney donors are retrospective in nature and have a short follow up period. There is lack of prospective long-term studies to know about the real risk of complications after donation. Furthermore, there are variations in recommendations from various guidelines across the globe for donations in prediabetics, leading to more confusion among clinicians. This increases the responsibility of transplant teams to take appropriate decisions in the best interest of both donors and recipients. This review focuses on pathophysiological changes of PD in kidneys, potential complications of PD, other risk factors for development of type 2 diabetes, a review of guidelines for kidney donation, the potential role of diabetes risk score and calculator in kidney donors and the way forward for the evaluation and selection of prediabetic kidney donors.

Remnant cholesterol, iron status and diabetes mellitus: a dose-response relationship and mediation analysis

BACKGROUND

Remnant cholesterol (RC) is recognized as a risk factor for diabetes mellitus (DM). Although iron status has been shown to be associated with cholesterol metabolism and DM, the association between RC, iron status, and DM remains unclear. We examined the relationship between RC and iron status and investigated the role of iron status in the association between RC and DM.

METHODS

A total of 7308 patients were enrolled from the China Health and Nutrition Survey. RC was calculated as total cholesterol minus low-density lipoprotein cholesterol and high-density lipoprotein cholesterol. Iron status was assessed as serum ferritin (SF) and total body iron (TBI). DM was ascertained by self-reported physician diagnosis and/or antidiabetic drug use and/or fasting plasma glucose ≥ 126 mg/dL and/or glycated haemoglobin ≥ 6.5%. General linear models were used to evaluate the relationships between RC and iron status. Restricted cubic splines were used to assess the association between RC and DM. Mediation analysis was used to clarified the mediating role of iron status in the association between the RC and DM.

RESULTS

The average age of the participants was 50.6 (standard deviation = 15.1) years. Higher RC was significantly associated with increased SF (β = 73.14, SE = 3.75, 95% confidence interval [CI] 65.79-80.49) and TBI (β = 1.61, SE = 0.08, 95% CI 1.44-1.78). J-shape relationships were found in the association between RC levels with DM, as well as iron status with DM. Significant indirect effects of SF and TBI in the association between RC and DM were found, with the index mediated at 9.58% and 6.37%, respectively.

CONCLUSIONS

RC has a dose-response relationship with iron status. The association between RC and DM was mediated in part by iron status. Future studies are needed to confirm these findings and further clarify the underlying mechanism.

Validation of the Framingham Diabetes Risk Model Using Community-Based KoGES Data

BACKGROUND

An 8-year prediction of the Framingham Diabetes Risk Model (FDRM) was proposed, but the predictor has a gap with current clinical standards. Therefore, we evaluated the validity of the original FDRM in Korean population data, developed a modified FDRM by redefining the predictors based on current knowledge, and evaluated the internal and external validity.

METHODS

Using data from a community-based cohort in Korea (n = 5,409), we calculated the probability of diabetes through FDRM, and developed a modified FDRM based on modified definitions of hypertension (HTN) and diabetes. We also added clinical features related to diabetes to the predictive model. Model performance was evaluated and compared by area under the curve (AUC).

RESULTS

During the 8-year follow-up, the cumulative incidence of diabetes was 8.5%. The modified FDRM consisted of age, obesity, HTN, hypo-high-density lipoprotein cholesterol, elevated triglyceride, fasting glucose, and hemoglobin A1c. The expanded clinical model added γ-glutamyl transpeptidase to the modified FDRM. The FDRM showed an estimated AUC of 0.71, and the model's performance improved to an AUC of 0.82 after applying the redefined predictor. Adding clinical features (AUC = 0.83) to the modified FDRM further improved in discrimination, but this was not maintained in the validation data set. External validation was evaluated on population-based cohort data and both modified models performed well, with AUC above 0.82.

CONCLUSION

The performance of FDRM in the Korean population was found to be acceptable for predicting diabetes, but it was improved when corrected with redefined predictors. The validity of the modified model needs to be further evaluated.

Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review

BACKGROUND

A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation.

METHODS

PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789).

RESULTS

In 20,887 screened references, 79 articles (82.5% in 2017-2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development (n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5-52,000, median 21) and large-span sample size (range 80-3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as "recommended"; however, 281 and 187 were "not recommended" and "warning," respectively.

CONCLUSION

AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.

Mean age and body mass index at type 2 diabetes diagnosis: Pooled analysis of 56 health surveys across income groups and world regions

BACKGROUND

Screening for type 2 diabetes mellitus (T2DM) targets people aged 35+ years and those with overweight/obesity. With mounting evidence on young-onset T2DM and T2DM patients with lean phenotypes, it is worth revising the screening criteria to include younger and leaner adults. We quantified the mean age and body mass index (BMI; kg/m2 ) at T2DM diagnosis in 56 countries.

METHODS

Descriptive cross-sectional analysis of WHO STEPS surveys. We analysed adults (25-69 years) with new T2DM diagnosis (not necessarily T2DM onset) as per fasting plasma glucose ≥126 mg/dL measured during the survey. For people with new T2DM diagnosis, we summarized the mean age and the proportion of each five-year age group; also, we summarized the mean BMI and the proportion of mutually exclusive BMI categories.

RESULTS

There were 8695 new T2DM patients. Overall, the mean age at T2DM diagnosis was 45.1 years in men and 45.0 years in women; and the mean BMI at T2DM diagnosis was 25.2 in men and 26.9 in women. Overall, in men, 10.3% were 25-29 years and 8.5% were 30-34 years old; in women, 8.6% and 12.5% were 25-29 years and 30-34 years old, respectively. 48.5% of men and 37.3% of women were in the normal BMI category.

CONCLUSIONS

A non-negligible proportion of new T2DM patients were younger than 35 years. Many new T2DM patients were in the normal weight range. Guidelines for T2DM screening may consider revising the age and BMI criteria to incorporate young and lean adults.

Should insulin resistance (HOMA-IR), insulin secretion (HOMA-β), and visceral fat area be considered for improving the performance of diabetes risk prediction models

INTRODUCTION

Insulin resistance and defects in pancreatic beta cells are the two major pathophysiologic abnormalities that underlie type 2 diabetes. In addition, visceral fat area (VFA) is reported to be a stronger predictor for diabetes than body mass index (BMI). Here, we tested whether the performance of diabetes prediction models could be improved by adding HOMA-IR and HOMA-β and replacing BMI with VFA.

RESEARCH DESIGN AND METHODS

We developed five prediction models using data from a cohort study (5578 individuals, of whom 94.7% were male, and 943 had incident diabetes). We conducted a baseline model (model 1) including age, sex, BMI, smoking, dyslipidemia, hypertension, and HbA1c. Subsequently, we developed another four models: model 2, predictors in model 1 plus fasting plasma glucose (FPG); model 3, predictors in model 1 plus HOMA-IR and HOMA-β; model 4, predictors in model 1 plus FPG, HOMA-IR, and HOMA-β; model 5, replaced BMI with VFA in model 2. We assessed model discrimination and calibration for the first 10 years of follow-up.

RESULTS

The addition of FPG to model 1 obviously increased the value of the area under the receiver operating characteristic curve from 0.79 (95% CI 0.78, 0.81) to 0.84 (0.83, 0.85). Compared with model 1, model 2 also significantly improved the risk reclassification and discrimination, with a continuous net reclassification improvement index of 0.61 (0.56, 0.70) and an integrated discrimination improvement index of 0.09 (0.08, 0.10). Adding HOMA-IR and HOMA-β (models 3 and 4) or replacing BMI with VFA (model 5) did not further materially improve the performance.

CONCLUSIONS

This cohort study, primarily composed of male workers, suggests that a model with BMI, FPG, and HbA1c effectively identifies those at high diabetes risk. However, adding HOMA-IR, HOMA-β, or replacing BMI with VFA does not significantly improve the model. Further studies are needed to confirm our findings.

Predictive model for early detection of type 2 diabetes using patients' clinical symptoms, demographic features, and knowledge of diabetes

Background and Aims

With the global rise in type 2 diabetes, predictive modeling has become crucial for early detection, particularly in populations with low routine medical checkup profiles. This study aimed to develop a predictive model for type 2 diabetes using health check-up data focusing on clinical details, demographic features, biochemical markers, and diabetes knowledge.

Methods

Data from 444 Nigerian patients were collected and analysed. We used 80% of this data set for training, and the remaining 20% for testing. Multivariable penalized logistic regression was employed to predict the disease onset, incorporating waist-hip ratio (WHR), triglycerides (TG), catalase, and atherogenic indices of plasma (AIP).

Results

The predictive model demonstrated high accuracy, with an area under the curve of 99% (95% CI = 97%-100%) for the training set and 94% (95% CI = 89%-99%) for the test set. Notably, an increase in WHR (adjusted odds ratio [AOR] = 70.35; 95% CI = 10.04-493.1, p-value < 0.001) and elevated AIP (AOR = 4.55; 95% CI = 1.48-13.95, p-value = 0.008) levels were significantly associated with a higher risk of type 2 diabetes, while higher catalase levels (AOR = 0.33; 95% CI = 0.22-0.49, p < 0.001) correlated with a decreased risk. In contrast, TG levels (AOR = 1.04; 95% CI = 0.40-2.71, p-value = 0.94) were not associated with the disease.

Conclusion

This study emphasizes the importance of using distinct clinical and biochemical markers for early type 2 diabetes detection in Nigeria, reflecting global trends in diabetes modeling, and highlighting the need for context-specific methods. The development of a web application based on these results aims to facilitate the early identification of individuals at risk, potentially reducing health complications, and improving diabetes management strategies in diverse settings.

A nomogram model for predicting 5-year risk of prediabetes in Chinese adults

Early identification is crucial to effectively intervene in individuals at high risk of developing pre-diabetes. This study aimed to create a personalized nomogram to determine the 5-year risk of pre-diabetes among Chinese adults. This retrospective cohort study included 184,188 participants without prediabetes at baseline. Training cohorts (92,177) and validation cohorts (92,011) were randomly assigned (92,011). We compared five prediction models on the training cohorts: full cox proportional hazards model, stepwise cox proportional hazards model, multivariable fractional polynomials (MFP), machine learning, and least absolute shrinkage and selection operator (LASSO) models. At the same time, we validated the above five models on the validation set. And we chose the LASSO model as the final risk prediction model for prediabetes. We presented the model with a nomogram. The model's performance was evaluated in terms of its discriminative ability, clinical utility, and calibration using the area under the receiver operating characteristic (ROC) curve, decision curve analysis, and calibration analysis on the training cohorts. Simultaneously, we also evaluated the above nomogram on the validation set. The 5-year incidence of prediabetes was 10.70% and 10.69% in the training and validation cohort, respectively. We developed a simple nomogram that predicted the risk of prediabetes by using the parameters of age, body mass index (BMI), fasting plasma glucose (FBG), triglycerides (TG), systolic blood pressure (SBP), and serum creatinine (Scr). The nomogram's area under the receiver operating characteristic curve (AUC) was 0.7341 (95% CI 0.7290-0.7392) for the training cohort and 0.7336 (95% CI 0.7285-0.7387) for the validation cohort, indicating good discriminative ability. The calibration curve showed a perfect fit between the predicted prediabetes risk and the observed prediabetes risk. An analysis of the decision curve presented the clinical application of the nomogram, with alternative threshold probability spectrums being presented as well. A personalized prediabetes prediction nomogram was developed and validated among Chinese adults, identifying high-risk individuals. Doctors and others can easily and efficiently use our prediabetes prediction model when assessing prediabetes risk.

Development of a risk prediction model to predict the risk of hospitalization due to exacerbated asthma among adult asthma patients in a lower middle-income country

BACKGROUND

Asthma patients experience higher rates of hospitalizations due to exacerbations leaving a considerable clinical and economic burden on the healthcare system. The use of a simple, risk prediction tool offers a low-cost mechanism to identify these high-risk asthma patients for specialized care. The study aimed to develop and validate a risk prediction model to identify high-risk asthma patients for hospitalization due to exacerbations.

METHODS

Hospital-based, case-control study was carried out among 466 asthma patients aged ≥ 20 years recruited from four tertiary care hospitals in a district of Sri Lanka to identify risk factors for asthma-related hospitalizations. Patients (n = 116) hospitalized due to an exacerbation with respiratory rate > 30/min, pulse rate > 120 bpm, O2 saturation (on air) < 90% on admission, selected consecutively from medical wards; controls (n = 350;1:3 ratio) randomly selected from asthma/medical clinics. Data was collected via a pre-tested Interviewer-Administered Questionnaire (IAQ). Logistic Regression (LR) analyses were performed to develop the model with consensus from an expert panel. A second case-control study was carried out to assess the criterion validity of the new model recruiting 158 cases and 101 controls from the same hospitals. Data was collected using an IAQ based on the newly developed risk prediction model.

RESULTS

The developed model consisted of ten predictors with an Area Under the Curve (AUC) of 0.83 (95% CI: 0.78 to 0.88, P < 0.001), sensitivity 69.0%, specificity 86.1%, positive predictive value (PPV) 88.6%, negative predictive value (NPV) 63.9%. Positive and negative likelihood ratios were 4.9 and 0.3, respectively.

CONCLUSIONS

The newly developed model was proven valid to identify adult asthma patients who are at risk of hospitalization due to exacerbations. It is recommended as a simple, low-cost tool for identifying and prioritizing high-risk asthma patients for specialized care.

Genetic polymorphisms in the angiotensin converting enzyme, actinin 3 and paraoxonase 1 genes in women with diabetes and hypertension

Objective

To study associations between polymorphisms in the angiotensin converting enzyme (ACE I/D), actinin 3 (ACTN3 R577X) and paraoxonase 1 (PON1 T(-107)C) genes and chronic diseases (diabetes and hypertension) in women.

Materials and methods

Genomic DNA was extracted from saliva samples of 78 women between 18 and 59 years old used for genetic polymorphism screening. Biochemical data were collected from the medical records in Basic Health Units from Southern Brazil. Questionnaires about food consumption, physical activity level and socioeconomic status were applied.

Results

The XX genotype of ACTN3 was associated with low HDL levels and high triglycerides, total cholesterol and glucose levels. Additionally, high triglycerides and LDL levels were observed in carriers of the TT genotype of PON1, and lower total cholesterol levels were associated to the CC genotype. As expected, women with diabetes/hypertense had increased body weight, BMI (p = 0.02), waist circumference (p = 0.01), body fat percentage, blood pressure (p = 0.02), cholesterol, triglycerides (p = 0.02), and blood glucose (p = 0.01), when compared to the control group.

Conclusion

Both ACTN3 R577X and PON1 T(-107)C polymorphisms are associated with nutritional status and blood glucose and lipid levels in women with diabetes/hypertense. These results contribute to genetic knowledge about predisposition to obesity-related diseases.

Machine learning in precision diabetes care and cardiovascular risk prediction

Artificial intelligence and machine learning are driving a paradigm shift in medicine, promising data-driven, personalized solutions for managing diabetes and the excess cardiovascular risk it poses. In this comprehensive review of machine learning applications in the care of patients with diabetes at increased cardiovascular risk, we offer a broad overview of various data-driven methods and how they may be leveraged in developing predictive models for personalized care. We review existing as well as expected artificial intelligence solutions in the context of diagnosis, prognostication, phenotyping, and treatment of diabetes and its cardiovascular complications. In addition to discussing the key properties of such models that enable their successful application in complex risk prediction, we define challenges that arise from their misuse and the role of methodological standards in overcoming these limitations. We also identify key issues in equity and bias mitigation in healthcare and discuss how the current regulatory framework should ensure the efficacy and safety of medical artificial intelligence products in transforming cardiovascular care and outcomes in diabetes.

Poor handling of continuous predictors in clinical prediction models using logistic regression: a systematic review

BACKGROUND AND OBJECTIVES

When developing a clinical prediction model, assuming a linear relationship between the continuous predictors and outcome is not recommended. Incorrect specification of the functional form of continuous predictors could reduce predictive accuracy. We examine how continuous predictors are handled in studies developing a clinical prediction model.

METHODS

We searched PubMed for clinical prediction model studies developing a logistic regression model for a binary outcome, published between July 01, 2020, and July 30, 2020.

RESULTS

In total, 118 studies were included in the review (18 studies (15%) assessed the linearity assumption or used methods to handle nonlinearity, and 100 studies (85%) did not). Transformation and splines were commonly used to handle nonlinearity, used in 7 (n = 7/18, 39%) and 6 (n = 6/18, 33%) studies, respectively. Categorization was most often used method to handle continuous predictors (n = 67/118, 56.8%) where most studies used dichotomization (n = 40/67, 60%). Only ten models included nonlinear terms in the final model (n = 10/18, 56%).

CONCLUSION

Though widely recommended not to categorize continuous predictors or assume a linear relationship between outcome and continuous predictors, most studies categorize continuous predictors, few studies assess the linearity assumption, and even fewer use methodology to account for nonlinearity. Methodological guidance is provided to guide researchers on how to handle continuous predictors when developing a clinical prediction model.

Misuse of multinomial logistic regression in stroke related health research: A systematic review of methodology

Multinomial logistic regression (MLR) is often used to model the association between a nominal outcome variable and one or more covariates. The results of MLR are interpreted as relative risk ratios (RRR) and warrant a more coherent interpretation than ordinary logistic regression. Some authors compare the results of MLR to ordinal logistic regression (OLR), irrespective of the fact that these estimate different quantities. We aim to investigate the time trends in the use and misuse of MLR in studies including stroke patients, specifically the extent to which (1) the results are denoted as anything other than RRR, (2) comparisons are made of results with results of OLR and (3) results have been interpreted coherently. Secondarily, we examine the use of model validation techniques in studies with predictive aims. We searched EMBASE and PubMed for articles using MLR on populations of stroke patients. Identified studies were screened, and information pertaining to our aims was extracted. A total of 285 articles were identified through a systematic literature search, and 68 of these were included in the review. Of these, 60 articles (88%) did not denote exponentiated coefficients of MLR as relative risk ratios but rather some other measure. Additionally, 63 articles (93%) interpreted the results of MLR in a non-coherent manner. Two articles attempted to compare MLR results with those of OLR. Nine studies attempted to use MLR for predictive means, and three used relevant validation techniques. From these findings, it is clear that the interpretation of MLR is often suboptimal.

Data collaboration analysis in predicting diabetes from a small amount of health checkup data

Recent studies showed that machine learning models such as gradient-boosting decision tree (GBDT) can predict diabetes with high accuracy from big data. In this study, we asked whether highly accurate prediction of diabetes is possible even from small data by expanding the amount of data through data collaboration (DC) analysis, a modern framework for integrating and analyzing data accumulated at multiple institutions while ensuring confidentiality. To this end, we focused on data from two institutions: health checkup data of 1502 citizens accumulated in Tsukuba City and health history data of 1399 patients collected at the University of Tsukuba Hospital. When using only the health checkup data, the ROC-AUC and Recall for logistic regression (LR) were 0.858 ± 0.014 and 0.970 ± 0.019, respectively, while those for GBDT were 0.856 ± 0.014 and 0.983 ± 0.016, respectively. When using also the health history data through DC analysis, these values for LR improved to 0.875 ± 0.013 and 0.993 ± 0.009, respectively, while those for GBDT deteriorated because of the low compatibility with a method used for confidential data sharing (although DC analysis brought improvements). Even in a situation where health checkup data of only 324 citizens are available, the ROC-AUC and Recall for LR were 0.767 ± 0.025 and 0.867 ± 0.04, respectively, thanks to DC analysis, indicating an 11% and 12% improvement. Thus, we concluded that the answer to the above question was "Yes" for LR but "No" for GBDT for the data set tested in this study.

Prediction of complications in health economic models of type 2 diabetes: a review of methods used

AIM

Diabetes health economic (HE) models play important roles in decision making. For most HE models of diabetes 2 diabetes (T2D), the core model concerns the prediction of complications. However, reviews of HE models pay little attention to the incorporation of prediction models. The objective of the current review is to investigate how prediction models have been incorporated into HE models of T2D and to identify challenges and possible solutions.

METHODS

PubMed, Web of Science, Embase, and Cochrane were searched from January 1, 1997, to November 15, 2022, to identify published HE models for T2D. All models that participated in The Mount Hood Diabetes Simulation Modeling Database or previous challenges were manually searched. Data extraction was performed by two independent authors. Characteristics of HE models, their underlying prediction models, and methods of incorporating prediction models were investigated.

RESULTS

The scoping review identified 34 HE models, including a continuous-time object-oriented model (n = 1), discrete-time state transition models (n = 18), and discrete-time discrete event simulation models (n = 15). Published prediction models were often applied to simulate complication risks, such as the UKPDS (n = 20), Framingham (n = 7), BRAVO (n = 2), NDR (n = 2), and RECODe (n = 2). Four methods were identified to combine interdependent prediction models for different complications, including random order evaluation (n = 12), simultaneous evaluation (n = 4), the 'sunflower method' (n = 3), and pre-defined order (n = 1). The remaining studies did not consider interdependency or reported unclearly.

CONCLUSIONS

The methodology of integrating prediction models in HE models requires further attention, especially regarding how prediction models are selected, adjusted, and ordered.

Prevalence and predictive modeling of undiagnosed diabetes and impaired fasting glucose in Taiwan: a Taiwan Biobank study

INTRODUCTION

We investigated the prevalence of undiagnosed diabetes and impaired fasting glucose (IFG) in individuals without known diabetes in Taiwan and developed a risk prediction model for identifying undiagnosed diabetes and IFG.

RESEARCH DESIGN AND METHODS

Using data from a large population-based Taiwan Biobank study linked with the National Health Insurance Research Database, we estimated the standardized prevalence of undiagnosed diabetes and IFG between 2012 and 2020. We used the forward continuation ratio model with the Lasso penalty, modeling undiagnosed diabetes, IFG, and healthy reference group (individuals without diabetes or IFG) as three ordinal outcomes, to identify the risk factors and construct the prediction model. Two models were created: Model 1 predicts undiagnosed diabetes, IFG_110 (ie, fasting glucose between 110 mg/dL and 125 mg/dL), and the healthy reference group, while Model 2 predicts undiagnosed diabetes, IFG_100 (ie, fasting glucose between 100 mg/dL and 125 mg/dL), and the healthy reference group.

RESULTS

The standardized prevalence of undiagnosed diabetes for 2012-2014, 2015-2016, 2017-2018, and 2019-2020 was 1.11%, 0.99%, 1.16%, and 0.99%, respectively. For these periods, the standardized prevalence of IFG_110 and IFG_100 was 4.49%, 3.73%, 4.30%, and 4.66% and 21.0%, 18.26%, 20.16%, and 21.08%, respectively. Significant risk prediction factors were age, body mass index, waist to hip ratio, education level, personal monthly income, betel nut chewing, self-reported hypertension, and family history of diabetes. The area under the curve (AUC) for predicting undiagnosed diabetes in Models 1 and 2 was 80.39% and 77.87%, respectively. The AUC for predicting undiagnosed diabetes or IFG in Models 1 and 2 was 78.25% and 74.39%, respectively.

CONCLUSIONS

Our results showed the changes in the prevalence of undiagnosed diabetes and IFG. The identified risk factors and the prediction models could be helpful in identifying individuals with undiagnosed diabetes or individuals with a high risk of developing diabetes in Taiwan.

Development and validation of DNA methylation scores in two European cohorts augment 10-year risk prediction of type 2 diabetes

Type 2 diabetes mellitus (T2D) presents a major health and economic burden that could be alleviated with improved early prediction and intervention. While standard risk factors have shown good predictive performance, we show that the use of blood-based DNA methylation information leads to a significant improvement in the prediction of 10-year T2D incidence risk. Previous studies have been largely constrained by linear assumptions, the use of cytosine-guanine pairs one-at-a-time and binary outcomes. We present a flexible approach (via an R package, MethylPipeR) based on a range of linear and tree-ensemble models that incorporate time-to-event data for prediction. Using the Generation Scotland cohort (training set n_cases = 374, n_controls = 9,461; test set n_cases = 252, n_controls = 4,526) our best-performing model (area under the receiver operating characteristic curve (AUC) = 0.872, area under the precision-recall curve (PRAUC) = 0.302) showed notable improvement in 10-year onset prediction beyond standard risk factors (AUC = 0.839, precision-recall AUC = 0.227). Replication was observed in the German-based KORA study (n = 1,451, n_cases = 142, P = 1.6 × 10^-5).

External validation of the American prediction model for incident type 2 diabetes in the Iranian population

Background

The primary aim of the present study was to validate the REasons for Geographic and Racial Differences in Stroke (REGARDS) model for incident Type 2 diabetes (T2DM) in Iran.

Methods

Present study was a prospective cohort study on 1835 population aged ≥ 45 years from Tehran lipids and glucose study (TLGS).The predictors of REGARDS model based on Bayesian hierarchical techniques included age, sex, race, body mass index, systolic and diastolic blood pressures, triglycerides, high-density lipoprotein cholesterol, and fasting plasma glucose. For external validation, the area under the curve (AUC), sensitivity, specificity, Youden’s index, and positive and negative predictive values (PPV and NPV) were assessed.

Results

During the 10-year follow-up 15.3% experienced T2DM. The model showed acceptable discrimination (AUC (95%CI): 0.79 (0.76–0.82)), and good calibration. Based on the highest Youden’s index the suggested cut-point for the REGARDS probability would be ≥ 13% which yielded a sensitivity of 77.2%, specificity 66.8%, NPV 94.2%, and PPV 29.6%.

Conclusions

Our findings do support that the REGARDS model is a valid tool for incident T2DM in the Iranian population. Moreover, the probability value higher than the 13% cut-off point is stated to be significant for identifying those with incident T2DM.

The use of Chinese herbal medicines throughout the pregnancy life course and their safety profiles: a population-based cohort study

BACKGROUND

Chinese herbal medicines have been long used among pregnant populations in China. However, despite the high susceptibility of this population to drug exposure, it continued to remain unclear about how often they were used, to what extent they were used at different pregnancy stages, and whether their use was based on sound safety profiles, particularly when used in combination with pharmaceutical drugs.

OBJECTIVE

This descriptive cohort study aimed to systematically investigate the use of Chinese herbal medicines throughout pregnancy and their safety profiles.

STUDY DESIGN

A large medication use cohort was developed by linking a population-based pregnancy registry and a population-based pharmacy database, which documented all prescriptions at both outpatients and inpatients from conception to 7 days after delivery, including pharmaceutical drugs and processed Chinese herbal medicine formulas that were approved by the regulatory authority and prepared under the guidance of national quality standards. The prevalence of the use of Chinese herbal medicine formulas, prescription pattern, and combination use of pharmaceutical drugs throughout pregnancy were investigated. Multivariable log-binomial regression was performed to assess temporal trends and further explore the potential characteristics associated with the use of Chinese herbal medicines. Of note, 2 authors independently conducted a qualitative systematic review of patient package inserts of the top 100 Chinese herbal medicine formulas used to identify their safety profiles.

RESULTS

This study included 199,710 pregnancies; of those pregnancies, 131,235 (65.71%) used Chinese herbal medicine formulas, including 26.13% during pregnancy (corresponding to 14.00%, 8.91%, and 8.26% in the first, second, and third trimesters of pregnancy) and 55.63% after delivery. The peak uses of Chinese herbal medicines occurred between 5 and 10 weeks of gestation. The use of Chinese herbal medicines significantly increased over the years (from 63.28% in 2014 to 69.59% in 2018; adjusted relative risk, 1.11; 95% confidence interval, 1.10-1.13), which was particularly great during pregnancy (from 18.47% in 2014 to 32.46% in 2018; adjusted relative risk, 1.84; 95% confidence interval, 1.77-1.90). Our study observed 291,836 prescriptions involving 469 Chinese herbal medicine formulas, and the top 100 most used Chinese herbal medicines accounted for 98.28% of the total prescriptions. Of those, a third (33.39%) were dispensed at outpatient visits; 6.79% were external use, and 0.29% were administered intravenously. However, Chinese herbal medicines were very often prescribed in combination with pharmaceutical drugs (94.96% overall), involving 1175 pharmaceutical drugs with 1,667,459 prescriptions. The median of pharmaceutical drugs prescribed in combination with Chinese herbal medicines per pregnancy was 10 (interquartile range, 5-18). The systematic review of drug patient package inserts found that the 100 most frequently prescribed Chinese herbal medicines contained a total of 240 herb constituents (median, 4.5); 7.00% were explicitly indicated for pregnancy or postpartum conditions; 43.00% were reported with efficacy or safety data from randomized controlled trials. Information was lacking about whether the medications had any reproductive toxicity, were excreted in human milk, or crossed the placenta.

CONCLUSION

The use of Chinese herbal medicines was prevalent throughout pregnancy and increased over the years. The use of Chinese herbal medicines peaked in the first trimester of pregnancy and was very often used in combination with pharmaceutical drugs. However, their safety profiles were mostly unclear or incomplete, suggesting a strong need for postapproval surveillance for the use of Chinese herbal medicines during pregnancy.

Causal Model Building in the Context of Cardiac Rehabilitation: A Systematic Review

Randomization is an effective design option to prevent bias from confounding in the evaluation of the causal effect of interventions on outcomes. However, in some cases, randomization is not possible, making subsequent adjustment for confounders essential to obtain valid results. Several methods exist to adjust for confounding, with multivariable modeling being among the most widely used. The main challenge is to determine which variables should be included in the causal model and to specify appropriate functional relations for continuous variables in the model. While the statistical literature gives a variety of recommendations on how to build multivariable regression models in practice, this guidance is often unknown to applied researchers. We set out to investigate the current practice of explanatory regression modeling to control confounding in the field of cardiac rehabilitation, for which mainly non-randomized observational studies are available. In particular, we conducted a systematic methods review to identify and compare statistical methodology with respect to statistical model building in the context of the existing recent systematic review CROS-II, which evaluated the prognostic effect of cardiac rehabilitation. CROS-II identified 28 observational studies, which were published between 2004 and 2018. Our methods review revealed that 24 (86%) of the included studies used methods to adjust for confounding. Of these, 11 (46%) mentioned how the variables were selected and two studies (8%) considered functional forms for continuous variables. The use of background knowledge for variable selection was barely reported and data-driven variable selection methods were applied frequently. We conclude that in the majority of studies, the methods used to develop models to investigate the effect of cardiac rehabilitation on outcomes do not meet common criteria for appropriate statistical model building and that reporting often lacks precision.

Minimal reporting improvement after peer review in reports of COVID-19 prediction models: systematic review

OBJECTIVES

To assess improvement in the completeness of reporting coronavirus (COVID-19) prediction models after the peer review process.

STUDY DESIGN AND SETTING

Studies included in a living systematic review of COVID-19 prediction models, with both preprint and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) reporting guidelines between pre-print and published manuscripts.

RESULTS

Nineteen studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence among preprint versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from preprint to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14 pp) across all studies. No association was observed between the change in percentage adherence and preprint score, journal impact factor, or time between journal submission and acceptance.

CONCLUSIONS

The preprint reporting quality of COVID-19 prediction modeling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic.

Development and External Validation of a Multivariable Prediction Model to Identify Nondiabetic Hyperglycemia and Undiagnosed Type 2 Diabetes: Diabetes Risk Assessment in Dentistry Score (DDS)

The aim of this study was to develop and externally validate a score for use in dental settings to identify those at risk of undiagnosed nondiabetic hyperglycemia (NDH) or type 2 diabetes (T2D). The Studies of Health in Pomerania (SHIP) project comprises 2 representative population-based cohort studies conducted in northeast Germany. SHIP-TREND-0, 2008 to 2012 (the development data set) had 3,339 eligible participants, with 329 having undiagnosed NDH or T2D. Missing data were replaced using multiple imputation. Potential covariates were selected for inclusion in the model using backward elimination. Heuristic shrinkage was used to reduce overfitting, and the final model was adjusted for optimism. We report the full model and a simplified paper-based point-score system. External validation of the model and score employed an independent data set comprising 2,359 participants with 357 events. Predictive performance, discrimination, calibration, and clinical utility were assessed. The final model included age, sex, body mass index, smoking status, first-degree relative with diabetes, presence of a dental prosthesis, presence of mobile teeth, history of periodontal treatment, and probing pocket depths ≥5 mm as well as prespecified interaction terms. In SHIP-TREND-0, the model area under the curve (AUC) was 0.72 (95% confidence interval [CI] 0.69, 0.75), calibration in the large was -0.025. The point score AUC was 0.69 (95% CI 0.65, 0.72), with sensitivity of 77.0 (95% CI 76.8, 77.2), specificity of 51.5 (95% CI 51.4, 51.7), negative predictive value of 94.5 (95% CI 94.5, 94.6), and positive predictive value of 17.0 (95% CI 17.0, 17.1). External validation of the point score gave an AUC of 0.69 (95% CI 0.66, 0.71), sensitivity of 79.2 (95% CI 79.0, 79.4), specificity of 49.9 (95% CI 49.8, 50.00), negative predictive value 91.5 (95% CI 91.5, 91.6), and positive predictive value of 25.9 (95% CI 25.8, 26.0). A validated prediction model involving dental variables can identify NDH or undiagnosed T2DM. Further studies are required to validate the model for different European populations.

Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models

BACKGROUND AND OBJECTIVES

We sought to summarize the study design, modelling strategies, and performance measures reported in studies on clinical prediction models developed using machine learning techniques.

METHODS

We search PubMed for articles published between 01/01/2018 and 31/12/2019, describing the development or the development with external validation of a multivariable prediction model using any supervised machine learning technique. No restrictions were made based on study design, data source, or predicted patient-related health outcomes.

RESULTS

We included 152 studies, 58 (38.2% [95% CI 30.8-46.1]) were diagnostic and 94 (61.8% [95% CI 53.9-69.2]) prognostic studies. Most studies reported only the development of prediction models (n = 133, 87.5% [95% CI 81.3-91.8]), focused on binary outcomes (n = 131, 86.2% [95% CI 79.8-90.8), and did not report a sample size calculation (n = 125, 82.2% [95% CI 75.4-87.5]). The most common algorithms used were support vector machine (n = 86/522, 16.5% [95% CI 13.5-19.9]) and random forest (n = 73/522, 14% [95% CI 11.3-17.2]). Values for area under the Receiver Operating Characteristic curve ranged from 0.45 to 1.00. Calibration metrics were often missed (n = 494/522, 94.6% [95% CI 92.4-96.3]).

CONCLUSION

Our review revealed that focus is required on handling of missing values, methods for internal validation, and reporting of calibration to improve the methodological conduct of studies on machine learning-based prediction models.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO, CRD42019161764.

The reporting of prognostic prediction models for obstetric care was poor: a cross-sectional survey of 10-year publications

BACKGROUND

To investigate the reporting of prognostic prediction model studies in obstetric care through a cross-sectional survey design.

METHODS

PubMed was searched to identify prognostic prediction model studies in obstetric care published from January 2011 to December 2020. The quality of reporting was assessed by the TRIPOD checklist. The overall adherence by study and the adherence by item were calculated separately, and linear regression analysis was conducted to explore the association between overall adherence and prespecified study characteristics.

RESULTS

A total of 121 studies were included, while no study completely adhered to the TRIPOD. The results showed that the overall adherence was poor (median 46.4%), and no significant improvement was observed after the release of the TRIPOD (43.9 to 46.7%). Studies including both model development and external validation had higher reporting quality versus those including model development only (68.1% vs. 44.8%). Among the 37 items required by the TRIPOD, 10 items were reported adequately with an adherence rate over of 80%, and the remaining 27 items had an adherence rate ranging from 2.5 to 79.3%. In addition, 11 items had a report rate lower than 25.0% and even covered key methodological aspects, including blinding assessment of predictors (2.5%), methods for model-building procedures (4.5%) and predictor handling (13.5%), how to use the model (13.5%), and presentation of model performance (14.4%).

CONCLUSIONS

In a 10-year span, prognostic prediction studies in obstetric care continued to be poorly reported and did not improve even after the release of the TRIPOD checklist. Substantial efforts are warranted to improve the reporting of obstetric prognostic prediction models, particularly those that adhere to the TRIPOD checklist are highly desirable.

Time above range for predicting the development of type 2 diabetes

Aim

To investigate the prognostic value of time range metrics, as measured by continuous glucose monitoring, with respect to the development of type 2 diabetes (T2D).

Research design and methods

A total of 499 persons without diabetes from the general population were followed-up for 5 years. Time range metrics were measured at the start and medical records were checked over the period study.

Results

Twenty-two subjects (8.3 per 1,000 person-years) developed T2D. After adjusting for age, gender, family history of diabetes, body mass index and glycated hemoglobin concentration, multivariate analysis revealed 'time above range' (TAR, i.e., with a plasma glucose concentration of >140 mg/dL) to be significantly associated with a greater risk (OR = 1.06, CI 1.01-1.11) of developing diabetes (AUC = 0.94, Brier = 0.035).

Conclusions

Time above range provides additional information to that offered by glycated hemoglobin to identify patients at a higher risk of developing type 2 diabetes in a population-based study.

Understanding the risk of developing weight-related complications associated with different body mass index categories: a systematic review

BACKGROUND

Obesity and overweight are major risk factors for several chronic diseases. There is limited systematic evaluation of risk equations that predict the likelihood of developing an obesity or overweight associated complication. Predicting future risk is essential for health economic modelling. Availability of future treatments rests upon a model's ability to inform clinical and decision-making bodies.  This systematic literature review aimed to identify studies reporting (1) equations that calculate the risk for individuals with obesity, or overweight with a weight-related complication (OWRC), of developing additional complications, namely T2D, cardiovascular (CV) disease (CVD), acute coronary syndrome, stroke, musculoskeletal disorders, knee replacement/arthroplasty, or obstructive sleep apnea; (2) absolute or proportional risk for individuals with severe obesity, obesity or OWRC developing T2D, a CV event or mortality from knee surgery, stroke, or an acute CV event.

METHODS

Databases (MEDLINE and Embase) were searched for English language reports of population-based cohort analyses or large-scale studies in Australia, Canada, Europe, the UK, and the USA between January 1, 2011, and March 29, 2021. Included reports were quality assessed using an adapted version of the Newcastle Ottawa Scale.

RESULTS

Of the 60 included studies, the majority used European cohorts. Twenty-nine reported a risk prediction equation for developing an additional complication. The most common risk prediction equations were logistic regression models that did not differentiate between body mass index (BMI) groups (particularly above 40 kg/m²) and lacked external validation. The remaining included studies (31 studies) reported the absolute or proportional risk of mortality (29 studies), or the risk of developing T2D in a population with obesity and with prediabetes or normal glucose tolerance (NGT) (three studies), or a CV event in populations with severe obesity with NGT or T2D (three studies). Most reported proportional risk, predominantly a hazard ratio.

CONCLUSION

More work is needed to develop and validate these risk equations, specifically in non-European cohorts and that distinguish between BMI class II and III obesity. New data or adjustment of the current risk equations by calibration would allow for more accurate decision making at an individual and population level.

Prediction of Weight Loss in Filipino Americans to Decrease Risk for Type 2 Diabetes: Using Multi-Dimensional Data (Preprint)

BACKGROUND

Type 2 diabetes (T2D) has an immense disease burden, affecting millions of people worldwide and costing billions of dollars in treatment. As T2D is a multifactorial disease with both genetic and nongenetic influences, accurate risk assessments for patients are difficult to perform. Machine learning has served as a useful tool in T2D risk prediction, as it can analyze and detect patterns in large and complex data sets like that of RNA sequencing. However, before machine learning can be implemented, feature selection is a necessary step to reduce the dimensionality in high-dimensional data and optimize modeling results. Different combinations of feature selection methods and machine learning models have been used in studies reporting disease predictions and classifications with high accuracy.

OBJECTIVE

The purpose of this study was to assess the use of feature selection and classification approaches that integrate different data types to predict weight loss for the prevention of T2D.

METHODS

The data of 56 participants (ie, demographic and clinical factors, dietary scores, step counts, and transcriptomics) were obtained from a previously completed randomized clinical trial adaptation of the Diabetes Prevention Program study. Feature selection methods were used to select for subsets of transcripts to be used in the selected classification approaches: support vector machine, logistic regression, decision trees, random forest, and extremely randomized decision trees (extra-trees). Data types were included in different classification approaches in an additive manner to assess model performance for the prediction of weight loss.

RESULTS

Average waist and hip circumference were found to be different between those who exhibited weight loss and those who did not exhibit weight loss (P=.02 and P=.04, respectively). The incorporation of dietary and step count data did not improve modeling performance compared to classifiers that included only demographic and clinical data. Optimal subsets of transcripts identified through feature selection yielded higher prediction accuracy than when all available transcripts were included. After comparison of different feature selection methods and classifiers, DESeq2 as a feature selection method and an extra-trees classifier with and without ensemble learning provided the most optimal results, as defined by differences in training and testing accuracy, cross-validated area under the curve, and other factors. We identified 5 genes in two or more of the feature selection subsets (ie, CDP-diacylglycerol-inositol 3-phosphatidyltransferase [CDIPT], mannose receptor C type 2 [MRC2], PAT1 homolog 2 [PATL2], regulatory factor X-associated ankyrin containing protein [RFXANK], and small ubiquitin like modifier 3 [SUMO3]).

CONCLUSIONS

Our results suggest that the inclusion of transcriptomic data in classification approaches for prediction has the potential to improve weight loss prediction models. Identification of which individuals are likely to respond to interventions for weight loss may help to prevent incident T2D. Out of the 5 genes identified as optimal predictors, 3 (ie, CDIPT, MRC2, and SUMO3) have been previously shown to be associated with T2D or obesity.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02278939; https://clinicaltrials.gov/ct2/show/NCT02278939.

External validation of non-invasive diabetes score in a 15-year prospective study

BACKGROUND

A novel scoring system called Non-invasive Diabetes Score (NDS) was developed. The model showed prominent discrimination and calibration in the original study population. However, before a new model could be adopted in clinical practice and acquire widespread use, it is necessary to confirm that it also performs well in external validations in different settings of people. The aim of this study was to investigate whether the novel user-friendly score predicting diabetes mellitus (DM) could have satisfying performance in predicting DM in Southwest China in a 15-year prospective cohort study.

METHODS

This prospective cohort study was carried out based on a general Chinese population of 711 individuals from 1992 to 2007. We excluded 24 of them at baseline because they were diabetics. The end point was DM, and the risk was calculated using the model formula.

RESULTS

During a follow-up of 15 years, 74 (10.77%) patients reached the end point. Evaluation of this model in our cohort, with Harrell's C-index of 0.662 (95% CI: 0.600-0.723) for the whole cohort and 0.695 (95% CI: 0.635-0.756) in sensitivity analysis, indicated the possibly helpful discrimination. The calibration capability in our cohort was useful that the observed incidence of diabetes mellitus was near the predicted.

CONCLUSIONS

Our external validation suggested NDS had possibly helpful discrimination and satisfying calibration for predicting DM during 15-year follow-up.

Action mechanism of hypoglycemic principle 9-(R)-HODE isolated from cortex lycii based on a metabolomics approach

The 9-(R)-HODE is an active compound isolated from cortex lycii that showed significant hypoglycemic effects in our previous in vitro study. In this study, 9-(R)-HODE’s in vivo hypoglycemic activity and effect on alleviating diabetic complications, together with its molecular mechanism, was investigated using a metabolomics approach. The monitored regulation on dynamic fasting blood glucose, postprandial glucose, body weight, biochemical parameters and histopathological analysis confirmed the hypoglycemic activity and attenuation effect, i.e., renal lesions, of 9-(R)-HODE. Subsequent metabolomic studies indicated that 9-(R)-HODE induced metabolomic alterations primarily by affecting the levels of amino acids, organic acids, alcohols and amines related to amino acid metabolism, glucose metabolism and energy metabolism. By mediating the related metabolism or single molecules related to insulin resistance, e.g., kynurenine, myo-inositol and the branched chain amino acids leucine, isoleucine and valine, 9-(R)-HODE achieved its therapeutic effect. Moreover, the mediation of kynurenine displayed a systematic effect on the liver, kidney, muscle, plasma and faeces. Lipidomic studies revealed that 9-(R)-HODE could reverse the lipid metabolism disorder in diabetic mice mainly by regulating phosphatidylinositols, lysophosphatidylcholines, lysophosphatidylcholines, phosphatidylserine, phosphatidylglycerols, lysophosphatidylglycerols and triglycerides in both tissues and plasma. Treatment with 9-(R)-HODE significantly modified the structure and composition of the gut microbiota. The SCFA-producing bacteria, including Rikenellaceae and Lactobacillaceae at the family level and Ruminiclostridium 6, Ruminococcaceae UCG 014, Mucispirillum, Lactobacillus, Alistipes and Roseburia at the genus level, were increased by 9-(R)-HODE treatment. These results were consistent with the increased SCFA levels in both the colon content and plasma of diabetic mice treated with 9-(R)-HODE. The tissue DESI‒MSI analysis strongly confirmed the validity of the metabolomics approach in illustrating the hypoglycemic and diabetic complications-alleviation effect of 9-(R)-HODE. The significant upregulation of liver glycogen in diabetic mice by 9-(R)-HODE treatment validated the interpretation of the metabolic pathways related to glycogen synthesis in the integrated pathway network. Altogether, 9-(R)-HODE has the potential to be further developed as a promising candidate for the treatment of diabetes.

Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data

We sought to verify the reliability of machine learning (ML) in developing diabetes prediction models by utilizing big data. To this end, we compared the reliability of gradient boosting decision tree (GBDT) and logistic regression (LR) models using data obtained from the Kokuho-database of the Osaka prefecture, Japan. To develop the models, we focused on 16 predictors from health checkup data from April 2013 to December 2014. A total of 277,651 eligible participants were studied. The prediction models were developed using a light gradient boosting machine (LightGBM), which is an effective GBDT implementation algorithm, and LR. Their reliabilities were measured based on expected calibration error (ECE), negative log-likelihood (Logloss), and reliability diagrams. Similarly, their classification accuracies were measured in the area under the curve (AUC). We further analyzed their reliabilities while changing the sample size for training. Among the 277,651 participants, 15,900 (7978 males and 7922 females) were newly diagnosed with diabetes within 3 years. LightGBM (LR) achieved an ECE of 0.0018 ± 0.00033 (0.0048 ± 0.00058), a Logloss of 0.167 ± 0.00062 (0.172 ± 0.00090), and an AUC of 0.844 ± 0.0025 (0.826 ± 0.0035). From sample size analysis, the reliability of LightGBM became higher than LR when the sample size increased more than [Formula: see text]. Thus, we confirmed that GBDT provides a more reliable model than that of LR in the development of diabetes prediction models using big data. ML could potentially produce a highly reliable diabetes prediction model, a helpful tool for improving lifestyle and preventing diabetes.

Opening the black box: interpretable machine learning for predictor finding of metabolic syndrome

OBJECTIVE

The internal workings ofmachine learning algorithms are complex and considered as low-interpretation "black box" models, making it difficult for domain experts to understand and trust these complex models. The study uses metabolic syndrome (MetS) as the entry point to analyze and evaluate the application value of model interpretability methods in dealing with difficult interpretation of predictive models.

METHODS

The study collects data from a chain of health examination institution in Urumqi from 2017 ~ 2019, and performs 39,134 remaining data after preprocessing such as deletion and filling. RFE is used for feature selection to reduce redundancy; MetS risk prediction models (logistic, random forest, XGBoost) are built based on a feature subset, and accuracy, sensitivity, specificity, Youden index, and AUROC value are used to evaluate the model classification performance; post-hoc model-agnostic interpretation methods (variable importance, LIME) are used to interpret the results of the predictive model.

RESULTS

Eighteen physical examination indicators are screened out by RFE, which can effectively solve the problem of physical examination data redundancy. Random forest and XGBoost models have higher accuracy, sensitivity, specificity, Youden index, and AUROC values compared with logistic regression. XGBoost models have higher sensitivity, Youden index, and AUROC values compared with random forest. The study uses variable importance, LIME and PDP for global and local interpretation of the optimal MetS risk prediction model (XGBoost), and different interpretation methods have different insights into the interpretation of model results, which are more flexible in model selection and can visualize the process and reasons for the model to make decisions. The interpretable risk prediction model in this study can help to identify risk factors associated with MetS, and the results showed that in addition to the traditional risk factors such as overweight and obesity, hyperglycemia, hypertension, and dyslipidemia, MetS was also associated with other factors, including age, creatinine, uric acid, and alkaline phosphatase.

CONCLUSION

The model interpretability methods are applied to the black box model, which can not only realize the flexibility of model application, but also make up for the uninterpretable defects of the model. Model interpretability methods can be used as a novel means of identifying variables that are more likely to be good predictors.

COVID-19 risk score as a public health tool to guide targeted testing: A demonstration study in Qatar

We developed a Coronavirus Disease 2019 (COVID-19) risk score to guide targeted RT-PCR testing in Qatar. The Qatar national COVID-19 testing database, encompassing a total of 2,688,232 RT-PCR tests conducted between February 5, 2020-January 27, 2021, was analyzed. Logistic regression analyses were implemented to derive the COVID-19 risk score, as a tool to identify those at highest risk of having the infection. Score cut-off was determined using the ROC curve based on maximum sum of sensitivity and specificity. The score’s performance diagnostics were assessed. Logistic regression analysis identified age, sex, and nationality as significant predictors of infection and were included in the risk score. The ROC curve was generated and the area under the curve was estimated at 0.63 (95% CI: 0.63–0.63). The score had a sensitivity of 59.4% (95% CI: 59.1%-59.7%), specificity of 61.1% (95% CI: 61.1%-61.2%), a positive predictive value of 10.9% (95% CI: 10.8%-10.9%), and a negative predictive value of 94.9% (94.9%-95.0%). The concept and utility of a COVID-19 risk score were demonstrated in Qatar. Such a public health tool can have considerable utility in optimizing testing and suppressing infection transmission, while maximizing efficiency and use of available resources.

Prediction of type 2 diabetes mellitus onset using logistic regression-based scorecards

Background

Type 2 diabetes (T2D) accounts for ~90% of all cases of diabetes, resulting in an estimated 6.7 million deaths in 2021, according to the International Diabetes Federation. Early detection of patients with high risk of developing T2D can reduce the incidence of the disease through a change in lifestyle, diet, or medication. Since populations of lower socio-demographic status are more susceptible to T2D and might have limited resources or access to sophisticated computational resources, there is a need for accurate yet accessible prediction models.

Methods

In this study, we analyzed data from 44,709 nondiabetic UK Biobank participants aged 40-69, predicting the risk of T2D onset within a selected time frame (mean of 7.3 years with an SD of 2.3 years). We started with 798 features that we identified as potential predictors for T2D onset. We first analyzed the data using gradient boosting decision trees, survival analysis, and logistic regression methods. We devised one nonlaboratory model accessible to the general population and one more precise yet simple model that utilizes laboratory tests. We simplified both models to an accessible scorecard form, tested the models on normoglycemic and prediabetes subcohorts, and compared the results to the results of the general cohort. We established the nonlaboratory model using the following covariates: sex, age, weight, height, waist size, hip circumference, waist-to-hip ratio, and body mass index. For the laboratory model, we used age and sex together with four common blood tests: high-density lipoprotein (HDL), gamma-glutamyl transferase, glycated hemoglobin, and triglycerides. As an external validation dataset, we used the electronic medical record database of Clalit Health Services.

Results

The nonlaboratory scorecard model achieved an area under the receiver operating curve (auROC) of 0.81 (95% confidence interval [CI] 0.77-0.84) and an odds ratio (OR) between the upper and fifth prevalence deciles of 17.2 (95% CI 5-66). Using this model, we classified three risk groups, a group with 1% (0.8-1%), 5% (3-6%), and the third group with a 9% (7-12%) risk of developing T2D. We further analyzed the contribution of the laboratory-based model and devised a blood test model based on age, sex, and the four common blood tests noted above. In this scorecard model, we included age, sex, glycated hemoglobin (HbA1c%), gamma glutamyl-transferase, triglycerides, and HDL cholesterol. Using this model, we achieved an auROC of 0.87 (95% CI 0.85-0.90) and a deciles' OR of ×48 (95% CI 12-109). Using this model, we classified the cohort into four risk groups with the following risks: 0.5% (0.4-7%); 3% (2-4%); 10% (8-12%); and a high-risk group of 23% (10-37%) of developing T2D. When applying the blood tests model using the external validation cohort (Clalit), we achieved an auROC of 0.75 (95% CI 0.74-0.75). We analyzed several additional comprehensive models, which included genotyping data and other environmental factors. We found that these models did not provide cost-efficient benefits over the four blood test model. The commonly used German Diabetes Risk Score (GDRS) and Finnish Diabetes Risk Score (FINDRISC) models, trained using our data, achieved an auROC of 0.73 (0.69-0.76) and 0.66 (0.62-0.70), respectively, inferior to the results achieved by the four blood test model and by the anthropometry models.

Conclusions

The four blood test and anthropometric models outperformed the commonly used nonlaboratory models, the FINDRISC and the GDRS. We suggest that our models be used as tools for decision-makers to assess populations at elevated T2D risk and thus improve medical strategies. These models might also provide a personal catalyst for changing lifestyle, diet, or medication modifications to lower the risk of T2D onset.

Funding

The funders had no role in study design, data collection, interpretation, or the decision to submit the work for publication.

Just How Confident Can We Be in Predicting Sports Injuries? A Systematic Review of the Methodological Conduct and Performance of Existing Musculoskeletal Injury Prediction Models in Sport

BACKGROUND

An increasing number of musculoskeletal injury prediction models are being developed and implemented in sports medicine. Prediction model quality needs to be evaluated so clinicians can be informed of their potential usefulness.

OBJECTIVE

To evaluate the methodological conduct and completeness of reporting of musculoskeletal injury prediction models in sport.

METHODS

A systematic review was performed from inception to June 2021. Studies were included if they: (1) predicted sport injury; (2) used regression, machine learning, or deep learning models; (3) were written in English; (4) were peer reviewed.

RESULTS

Thirty studies (204 models) were included; 60% of studies utilized only regression methods, 13% only machine learning, and 27% both regression and machine learning approaches. All studies developed a prediction model and no studies externally validated a prediction model. Two percent of models (7% of studies) were low risk of bias and 98% of models (93% of studies) were high or unclear risk of bias. Three studies (10%) performed an a priori sample size calculation; 14 (47%) performed internal validation. Nineteen studies (63%) reported discrimination and two (7%) reported calibration. Four studies (13%) reported model equations for statistical predictions and no machine learning studies reported code or hyperparameters.

CONCLUSION

Existing sport musculoskeletal injury prediction models were poorly developed and have a high risk of bias. No models could be recommended for use in practice. The majority of models were developed with small sample sizes, had inadequate assessment of model performance, and were poorly reported. To create clinically useful sports musculoskeletal injury prediction models, considerable improvements in methodology and reporting are urgently required.

Development of a machine learning-based prediction model for extremely rapid decline in estimated glomerular filtration rate in patients with chronic kidney disease: a retrospective cohort study using a large data set from a hospital in Japan

OBJECTIVES

Trajectories of estimated glomerular filtration rate (eGFR) decline vary highly among patients with chronic kidney disease (CKD). It is clinically important to identify patients who have high risk for eGFR decline. We aimed to identify clusters of patients with extremely rapid eGFR decline and develop a prediction model using a machine learning approach.

DESIGN

Retrospective single-centre cohort study.

SETTINGS

Tertiary referral university hospital in Toyoake city, Japan.

PARTICIPANTS

A total of 5657 patients with CKD with baseline eGFR of 30 mL/min/1.73 m² and eGFR decline of ≥30% within 2 years.

PRIMARY OUTCOME

Our main outcome was extremely rapid eGFR decline. To study-complicated eGFR behaviours, we first applied a variation of group-based trajectory model, which can find trajectory clusters according to the slope of eGFR decline. Our model identified high-level trajectory groups according to baseline eGFR values and simultaneous trajectory clusters. For each group, we developed prediction models that classified the steepest eGFR decline, defined as extremely rapid eGFR decline compared with others in the same group, where we used the random forest algorithm with clinical parameters.

RESULTS

Our clustering model first identified three high-level groups according to the baseline eGFR (G1, high GFR, 99.7±19.0; G2, intermediate GFR, 62.9±10.3 and G3, low GFR, 43.7±7.8); our model simultaneously found three eGFR trajectory clusters for each group, resulting in nine clusters with different slopes of eGFR decline. The areas under the curve for classifying the extremely rapid eGFR declines in the G1, G2 and G3 groups were 0.69 (95% CI, 0.63 to 0.76), 0.71 (95% CI 0.69 to 0.74) and 0.79 (95% CI 0.75 to 0.83), respectively. The random forest model identified haemoglobin, albumin and C reactive protein as important characteristics.

CONCLUSIONS

The random forest model could be useful in identifying patients with extremely rapid eGFR decline.

TRIAL REGISTRATION

UMIN 000037476; This study was registered with the UMIN Clinical Trials Registry.

Preoperative Prediction of Body Mass Index of Patients with Type 2 Diabetes at 1 Year After Laparoscopic Sleeve Gastrectomy: Cross-Sectional Study

Background: Very few models predict weight loss among type 2 diabetes mellitus (T2D) patients after laparoscopic sleeve gastrectomy (LSG). This retrospective study undertook such a task. Materials and Methods: We identified all patients >18 years old with T2D who underwent primary LSG at our institution and had complete data. The training set comprised 107 patients operated upon during the period April 2011 to June 2014; the validation set comprised 134 patients operated upon during the successive chronological period, July 2014 to December 2015. Sex, age, presurgery BMI, T2D duration, number of T2D medications, insulin use, hypertension, and dyslipidemia were utilized as independent predictors of 1-year BMI. We employed regression analysis, and assessed the goodness of fit and "Residuals versus Fits" plot. Paired sample t-tests compared the observed and predicted BMI at 1 year. Results: The model comprised preoperative BMI (β = 0.757, P = 0.026) + age (β = 0.142, P < 0.0001) with adjusted R² of 0.581 (P < 0.0001), and goodness of fit showed an unbiased model with accurate prediction. The equation was: BMI value 1 year after LSG = 1.777 + 0.614 × presurgery BMI (kg/m²) +0.106 × age (years). For validation, the equation exhibited an adjusted R² 0.550 (P < 0.0001), and the goodness of fit indicated an unbiased model. The BMI predicted by the model fell within -3.78 BMI points to +2.42 points of the observed 1-year BMI. Pairwise difference between the mean 1-year observed and predicted BMI was not significant (-0.41 kg/m², P = 0.225). Conclusions: This predictive model estimates the BMI 1 year after LSG. The model comprises preoperative BMI and age. It allows the forecast of patients' BMI after surgery, hence setting realistic expectations which are critical for patient satisfaction after bariatric surgery. An attainable target motivates the patient to achieve it.

Developing a risk score for undiagnosed prediabetes or type 2 diabetes among Saharawi refugees in Algeria

Aims

To prevent type 2 diabetes mellitus (T2D) and reduce the risk of complications, early identification of people at risk of developing T2D, preferably through simple diabetes risk scores, is essential. The aim of this study was to create a risk score for identifying subjects with undiagnosed prediabetes or T2D among Saharawi refugees in Algeria and compare the performance of this score to the Finnish diabetes risk score (FINDRISC).

Methods

A cross-sectional survey was carried out in five Saharawi refugee camps in Algeria in 2014. A total of 180 women and 175 men were included. HbA1c and cut-offs proposed by the American Diabetes Association (ADA) were used to define cases. Variables to include in the risk score were determined by backwards elimination in logistic regression. Simplified scores were created based on beta coefficients from the multivariable model after internal validation with bootstrapping and shrinkage. The empirical cut-off value for the simplified score and FINDRISC was determined by Area Under the Receiver Operating Curve (AUROC) analysis.

Results

Variables included in the final risk score were age, body mass index (BMI), and waist circumference. The area under the curve (AUC) (C.I) was 0.82 (0.76, 0.88). The sensitivity, specificity, and positive and negative predictive values were 89, 65, 28, and 97%, respectively. AUC and sensitivity were slightly higher and specificity somewhat lower than for FINDRISC.

Conclusions

The risk score developed is a helpful tool to decide who should be screened for prediabetes or T2D by blood sample analysis. The performance of the risk score was adequate based on internal validation with bootstrap analyses, but should be confirmed in external validation studies.

Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review

BACKGROUND

Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology.

METHODS

We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models.

RESULTS

Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available.

CONCLUSIONS

The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.

Development and Validation of a Simple Risk Model for Predicting Metabolic Syndrome (MetS) in Midlife: A Cohort Study

Purpose

To develop and validate a simple risk model for predicting metabolic syndrome in midlife using a prospective cohort data.

Design

Prospective cohort study.

Participants

A total of 7626 members of the 1958 British birth cohort (individuals born in the first week of March 1958) participated in the biomedical survey at age 45 and have completed information on metabolic syndrome.

Methods

Variables utilised were obtained prospectively at birth, 7, 16, 23 and 45 years. Multivariable logistic regression was used to develop a total of ten (10) MetS risk prediction models taking the life course approach. Measures of discrimination and calibration were used to evaluate the performance of the models. A pragmatic criteria developed was used to select one model with the most potential to be useful. The internal validity (overfitting) of the selected model was assessed using bootstrap technique of Stata.

Main Outcome Measure

Metabolic syndrome was defined based on the NCEP-ATP III clinical criteria.

Results

There is high prevalence of MetS among the cohort members (19.6%), with males having higher risk as compared to females (22.8% vs 16.4%, P < 0.001). Individuals with MetS are more likely to have higher levels of HbA1c and low HDL-cholesterol. Similarly, regarding the individual components of MetS, male cohort members are more likely to have higher levels of glycaemia (HbA1c), BP and serum triglycerides. In contrast, female cohort members have lower levels of HDL-cholesterol and higher levels of waist circumference. Furthermore, a total of ten (10) MetS risk prediction models were developed taking the life course approach. Of these, one model with the most potential to be applied in practical setting was selected. The model has good accuracy (AUROC 0.91 (0.90, 0.92)), is well calibrated (Hosmer-Lemeshow 6.47 (0.595)) and has good internal validity.

Conclusion

Early life factors could be included in a risk model to predict MetS in midlife. The developed model has been shown to be accurate and has good internal validity. Therefore, interventions targeting socioeconomic inequality could help in the wider prevention of MetS. However, the validity of the developed model needs to be further established in an external population.

Machine learning models for diabetes management in acute care using electronic medical records: A systematic review

BACKGROUND

Machine learning (ML) is a subset of Artificial Intelligence (AI) that is used to predict and potentially prevent adverse patient outcomes. There is increasing interest in the application of these models in digital hospitals to improve clinical decision-making and chronic disease management, particularly for patients with diabetes. The potential of ML models using electronic medical records (EMR) to improve the clinical care of hospitalised patients with diabetes is currently unknown.

OBJECTIVE

The aim was to systematically identify and critically review the published literature examining the development and validation of ML models using EMR data for improving the care of hospitalised adult patients with diabetes.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines were followed. Four databases were searched (Embase, PubMed, IEEE and Web of Science) for studies published between January 2010 to January 2022. The reference lists of the eligible articles were manually searched. Articles that examined adults and both developed and validated ML models using EMR data were included. Studies conducted in primary care and community care settings were excluded. Studies were independently screened and data was extracted using Covidence® systematic review software. For data extraction and critical appraisal, the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was followed. Risk of bias was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). Quality of reporting was assessed by adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline. The IJMEDI checklist was followed to assess quality of ML models and the reproducibility of their outcomes. The external validation methodology of the studies was appraised.

RESULTS

Of the 1317 studies screened, twelve met inclusion criteria. Eight studies developed ML models to predict disglycaemic episodes for hospitalized patients with diabetes, one study developed a ML model to predict total insulin dosage, two studies predicted risk of readmission, and one study improved the prediction of hospital readmission for inpatients with diabetes. All included studies were heterogeneous with regard to ML types, cohort, input predictors, sample size, performance and validation metrics and clinical outcomes. Two studies adhered to the TRIPOD guideline. The methodological reporting of all the studies was evaluated to be at high risk of bias. The quality of ML models in all studies was assessed as poor. Robust external validation was not performed on any of the studies. No models were implemented or evaluated in routine clinical care.

CONCLUSIONS

This review identified a limited number of ML models which were developed to improve inpatient management of diabetes. No ML models were implemented in real hospital settings. Future research needs to enhance the development, reporting and validation steps to enable ML models for integration into routine clinical care.

Can periodontal measures assist in the identification of adults with undiagnosed hyperglycaemia? A systematic review

AIM

The aim of this review was to answer the following question: Can periodontal measures be used to identify dental patients with undiagnosed hyperglycaemia?

MATERIALS AND METHODS

Systematic searches of electronic databases and the grey literature were carried out to identify studies developing and/or validating prediction models, based on any periodontal measure, to screen adults for undiagnosed hyperglycaemia (pre-diabetes and diabetes). Risk of bias was evaluated using the PRediction mOdel risk-of-Bias ASsessment Tool (PROBAST).

RESULTS

Ten studies were identified, of which eight were model development studies. The remaining two studies reported the external validation of one existing prediction model. The periodontal prediction model with some evidence of external validation showed moderate diagnostic performance in the development sample but lower performance in the external validation samples. According to PROBAST, all studies had high risk of bias mainly due to methodological limitations in data analysis, but also in the recruitment of participants, choice and measurement of periodontal predictors and diabetes.

CONCLUSIONS

There is a need for more robust external validation studies of existing prediction models adhering to current recommendations. Dental professionals who see patients at risk of diabetes and routinely collect periodontal measures have an important role to play in their identification and referral.