External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks

AI-enabled organoids: Construction, analysis, and application

Organoids, miniature and simplified in vitro model systems that mimic the structure and function of organs, have attracted considerable interest due to their promising applications in disease modeling, drug screening, personalized medicine, and tissue engineering. Despite the substantial success in cultivating physiologically relevant organoids, challenges remain concerning the complexities of their assembly and the difficulties associated with data analysis. The advent of AI-Enabled Organoids, which interfaces with artificial intelligence (AI), holds the potential to revolutionize the field by offering novel insights and methodologies that can expedite the development and clinical application of organoids. This review succinctly delineates the fundamental concepts and mechanisms underlying AI-Enabled Organoids, summarizing the prospective applications on rapid screening of construction strategies, cost-effective extraction of multiscale image features, streamlined analysis of multi-omics data, and precise preclinical evaluation and application. We also explore the challenges and limitations of interfacing organoids with AI, and discuss the future direction of the field. Taken together, the AI-Enabled Organoids hold significant promise for advancing our understanding of organ development and disease progression, ultimately laying the groundwork for clinical application.

External Validation of Models for Predicting Disability in Community-Dwelling Older People in the Netherlands: A Comparative Study

Background

Advanced statistical modeling techniques may help predict health outcomes. However, it is not the case that these modeling techniques always outperform traditional techniques such as regression techniques. In this study, external validation was carried out for five modeling strategies for the prediction of the disability of community-dwelling older people in the Netherlands.

Methods

We analyzed data from five studies consisting of community-dwelling older people in the Netherlands. For the prediction of the total disability score as measured with the Groningen Activity Restriction Scale (GARS), we used fourteen predictors as measured with the Tilburg Frailty Indicator (TFI). Both the TFI and the GARS are self-report questionnaires. For the modeling, five statistical modeling techniques were evaluated: general linear model (GLM), support vector machine (SVM), neural net (NN), recursive partitioning (RP), and random forest (RF). Each model was developed on one of the five data sets and then applied to each of the four remaining data sets. We assessed the performance of the models with calibration characteristics, the correlation coefficient, and the root of the mean squared error.

Results

The models GLM, SVM, RP, and RF showed satisfactory performance characteristics when validated on the validation data sets. All models showed poor performance characteristics for the deviating data set both for development and validation due to the deviating baseline characteristics compared to those of the other data sets.

Conclusion

The performance of four models (GLM, SVM, RP, RF) on the development data sets was satisfactory. This was also the case for the validation data sets, except when these models were developed on the deviating data set. The NN models showed a much worse performance on the validation data sets than on the development data sets.

Identifying Metabolic Syndrome Easily and Cost Effectively Using Non-Invasive Methods with Machine Learning Models

Purpose

The objective of this study was to employ machine learning (ML) models utilizing non-invasive factors to achieve early and low-cost identification of MetS in a large physical examination population.

Patients and Methods

The study enrolled 9171 participants who underwent physical examinations at Northern Jiangsu People's Hospital in 2009 and 2019, to determine MetS based on criteria established by the Chinese Diabetes Society. Non-invasive characteristics such as gender, age, body mass index (BMI), systolic blood pressure (SBP), and diastolic blood pressure (DBP) were collected and used as input variables to train and evaluate ML models for MetS identification. Several ML models were used for MetS identification, including logistic regression (LR), k-nearest neighbors algorithm (k-NN), naive bayesian (NB), decision tree (DT), random forest (RF), artificial neural network (ANN), and support vector machine (SVM).

Results

Our ML models all showed good performance in the 10-fold cross-validation except for the SVM model. In the external validation, the NB model exhibited the best performance with an AUC of 0.976, accuracy of 0.923, sensitivity of 98.32%, and specificity of 91.32%.

Conclusion

This study proposed a new non-invasive method for early and low-cost identification of MetS by using ML models. This approach has the potential to serve as a highly sensitive, convenient, and cost-effective tool for large-scale MetS screening.

Assessing model accuracy using random data split: a simulation study

Randomization is considered a safeguard against bias and a gold standard in clinical studies. To assess the generalizability of the accuracy of a model, a common approach is to randomly split a master data set into two parts: one for training and the other for testing. In this paper, we demonstrated the limitations of random split in assessing the generalizability of the accuracy of models through simulation studies. We generated three simulation data for binary or continuous endpoints, each with large sample size (n = 10,000). In each simulation scenario, we randomly split the data into two, one for training and one for testing, and then compare the performance of the model between training and testing data. All simulations were repeated 1,000 times. When random split was used, the model performance based on training and testing data behaved similarly in terms of the true positive fraction and false positive fraction for binary data and mean-squared errors for continuous data. However, when there is a time drift effect in the data, random split will result in large differences between training and testing data. As the training and testing data are similar through a random split, assessing the generalizability of the model on similar data will generate similar results. Generalizability of the accuracy of models is thus best achieved if testing is done in a distinct and independent study.

Development and internal validation of a machine-learning-developed model for predicting 1-year mortality after fragility hip fracture

Background

Fragility hip fracture increases morbidity and mortality in older adult patients, especially within the first year. Identification of patients at high risk of death facilitates modification of associated perioperative factors that can reduce mortality. Various machine learning algorithms have been developed and are widely used in healthcare research, particularly for mortality prediction. This study aimed to develop and internally validate 7 machine learning models to predict 1-year mortality after fragility hip fracture.

Methods

This retrospective study included patients with fragility hip fractures from a single center (Siriraj Hospital, Bangkok, Thailand) from July 2016 to October 2018. A total of 492 patients were enrolled. They were randomly categorized into a training group (344 cases, 70%) or a testing group (148 cases, 30%). Various machine learning techniques were used: the Gradient Boosting Classifier (GB), Random Forests Classifier (RF), Artificial Neural Network Classifier (ANN), Logistic Regression Classifier (LR), Naive Bayes Classifier (NB), Support Vector Machine Classifier (SVM), and K-Nearest Neighbors Classifier (KNN). All models were internally validated by evaluating their performance and the area under a receiver operating characteristic curve (AUC).

Results

For the testing dataset, the accuracies were GB model = 0.93, RF model = 0.95, ANN model = 0.94, LR model = 0.91, NB model = 0.89, SVM model = 0.90, and KNN model = 0.90. All models achieved high AUCs that ranged between 0.81 and 0.99. The RF model also provided a negative predictive value of 0.96, a positive predictive value of 0.93, a specificity of 0.99, and a sensitivity of 0.68.

Conclusions

Our machine learning approach facilitated the successful development of an accurate model to predict 1-year mortality after fragility hip fracture. Several machine learning algorithms (eg, Gradient Boosting and Random Forest) had the potential to provide high predictive performance based on the clinical parameters of each patient. The web application is available at www.hipprediction.com. External validation in a larger group of patients or in different hospital settings is warranted to evaluate the clinical utility of this tool.

Trial registration

Thai Clinical Trials Registry (22 February 2021; reg. no. TCTR20210222003).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03152-x.

Correlation Tests of Ultrasonic Wave and Mechanical Parameters of Spot-Welded Joints

Resistance spot welding as the basic method of joining car body elements has been used in the automotive industry for many years. For these connections, it is required to obtain the appropriate diameter of the weld nugget, which results in a high strength and durability of the connection during vehicle operation. The article presents the methodology of testing spot-welded joints using both destructive methods: shearing test of the spot weld and the ultrasonic method. The main goals of the performed tests are (1) to determine the correlation between the mechanical strength of a joint, measured in kN, and the selected parameters of the ultrasonic longitudinal wave with a frequency of 20 MHz propagating in the area of the spot weld and (2) to build and verify the predictive models of the weld nugget quality. The correlation of these parameters allows assessing the strength of the connection with the use of a non-destructive test method. On the basis of the performed analyses, it was determined that there is a strongly positive correlation between the number of reverse echoes and the force necessary to destroy the spot weld (0.41) and the diameter of the weld nugget (0.50). A strong negative correlation was also obtained between the number of echoes and the strength (−0.69) and diameter of the weld nugget (−0.72).

Evaluation framework to guide implementation of AI systems into healthcare settings

Objectives

To date, many artificial intelligence (AI) systems have been developed in healthcare, but adoption has been limited. This may be due to inappropriate or incomplete evaluation and a lack of internationally recognised AI standards on evaluation. To have confidence in the generalisability of AI systems in healthcare and to enable their integration into workflows, there is a need for a practical yet comprehensive instrument to assess the translational aspects of the available AI systems. Currently available evaluation frameworks for AI in healthcare focus on the reporting and regulatory aspects but have little guidance regarding assessment of the translational aspects of the AI systems like the functional, utility and ethical components.

Methods

To address this gap and create a framework that assesses real-world systems, an international team has developed a translationally focused evaluation framework termed ‘Translational Evaluation of Healthcare AI (TEHAI)’. A critical review of literature assessed existing evaluation and reporting frameworks and gaps. Next, using health technology evaluation and translational principles, reporting components were identified for consideration. These were independently reviewed for consensus inclusion in a final framework by an international panel of eight expert.

Results

TEHAI includes three main components: capability, utility and adoption. The emphasis on translational and ethical features of the model development and deployment distinguishes TEHAI from other evaluation instruments. In specific, the evaluation components can be applied at any stage of the development and deployment of the AI system.

Discussion

One major limitation of existing reporting or evaluation frameworks is their narrow focus. TEHAI, because of its strong foundation in translation research models and an emphasis on safety, translational value and generalisability, not only has a theoretical basis but also practical application to assessing real-world systems.

Conclusion

The translational research theoretic approach used to develop TEHAI should see it having application not just for evaluation of clinical AI in research settings, but more broadly to guide evaluation of working clinical systems.

Toward Predicting Human Performance Outcomes From Wearable Technologies: A Computational Modeling Approach

Wearable technologies for measuring digital and chemical physiology are pervading the consumer market and hold potential to reliably classify states of relevance to human performance including stress, sleep deprivation, and physical exertion. The ability to efficiently and accurately classify physiological states based on wearable devices is improving. However, the inherent variability of human behavior within and across individuals makes it challenging to predict how identified states influence human performance outcomes of relevance to military operations and other high-stakes domains. We describe a computational modeling approach to address this challenge, seeking to translate user states obtained from a variety of sources including wearable devices into relevant and actionable insights across the cognitive and physical domains. Three status predictors were considered: stress level, sleep status, and extent of physical exertion; these independent variables were used to predict three human performance outcomes: reaction time, executive function, and perceptuo-motor control. The approach provides a complete, conditional probabilistic model of the performance variables given the status predictors. Construction of the model leverages diverse raw data sources to estimate marginal probability density functions for each of six independent and dependent variables of interest using parametric modeling and maximum likelihood estimation. The joint distributions among variables were optimized using an adaptive LASSO approach based on the strength and directionality of conditional relationships (effect sizes) derived from meta-analyses of extant research. The model optimization process converged on solutions that maintain the integrity of the original marginal distributions and the directionality and robustness of conditional relationships. The modeling framework described provides a flexible and extensible solution for human performance prediction, affording efficient expansion with additional independent and dependent variables of interest, ingestion of new raw data, and extension to two- and three-way interactions among independent variables. Continuing work includes model expansion to multiple independent and dependent variables, real-time model stimulation by wearable devices, individualized and small-group prediction, and laboratory and field validation.

Predicting the mortality risk of acute respiratory distress syndrome: radial basis function artificial neural network model versus logistic regression model

To predict the mortality of acute respiratory distress syndrome (ARDS) by using a radial basis function (RBF) artificial neural network (ANN) model. This study included 217 patients who were admitted between June 2013 and November 2019. The RBF ANN model and logistic regression (LR) model were based on twelve factors related to ARDS. Statistical indexes were used to determine the value of the prediction in the two models. The sensitivity, specificity and accuracy of the RBF ANN model to predict mortality were 83.6%, 88.5% and 82.5%, respectively. Significant differences were found between the RBF ANN and LR models (P < 0.05). When the RBF ANN model was used to identify ARDS, the area under the ROC curve was 0.854 ± 0.029. LDH, organ failure, SP-D and PaO2/FiO2 were the most important independent variables. The RBF ANN model was more likely to predict the mortality of ARDS than the LR model. In addition, it can extract informative risk factors for ARDS.

The Utility of Artificial Neural Networks for the Non-Invasive Prediction of Metabolic Syndrome Based on Personal Characteristics

This study investigated the diagnostic accuracy of using an artificial neural network (ANN) for the prediction of metabolic syndrome (MetS) based on socioeconomic status and lifestyle factors. The data of 27,415 subjects who went through examinations and answered questionnaires during three stages from 2006 to 2014 at a health institute in Taiwan were collected and analyzed. The repeated measurements over time were set as predictive factors and used to train and test an ANN for MetS prediction. Among the subjects, 18.3%, 24.6%, and 30.1% were diagnosed with MetS during the respective three stages. ANN analysis applied with an over-sampling technique performed with an area under the curve (AUC) of up to 0.93 based on different models. The over-sampling technique helped improve prediction performance in terms of sensitivity and F₂ measures. The results indicated that waist circumference, socioeconomic status (SES), and lifestyle factors can be utilized in a non-invasive screening tool to assist health workers in making primary care decisions when MetS is suspected. By predicting the occurrence of MetS, individuals or healthcare professionals can then develop preventive strategies in time, thus enhancing the effectiveness of health promotion.

Evaluation of the external validity of a joint structure-function model for monitoring glaucoma progression

The dynamic structure–function (DSF) model was previously shown to have better prediction accuracy than ordinary least square linear regression (OLSLR) for short series of visits. The current study assessed the external validity of the DSF model by testing its performance in an independent dataset (Ocular Hypertension Treatment Study–Confocal Scanning Laser Ophthalmoscopy [OHTS–CSLO] ancillary study; N = 178 eyes), and also on different test parameters in a sample selected from the Diagnostic Innovations in Glaucoma Study or the African Descent and Glaucoma Evaluation Study (DIGS/ADAGES). Each model was used to predict structure–function paired data at visits 4–7. The resulting prediction errors for both models were compared using the Wilcoxon signed-rank test. In the independent dataset, the DSF model predicted rim area and mean sensitivity paired measurements more accurately than OLSLR by 1.8–5.5% (p ≤ 0.004) from visits 4–6. Using the DIGS/ADAGES dataset, the DSF model predicted retinal nerve fiber layer thickness and mean deviation paired measurements more accurately than OLSLR by 1.2–2.5% (p ≤ 0. 007). These results demonstrate the external validity of the DSF model and provide a strong basis to develop it into a useful clinical tool.

Technical assessment of the neonatal early-onset sepsis risk calculator

The use of the neonatal early-onset sepsis risk calculator, developed by Kaiser Permanente Northern California (CA, USA), is increasing for the management of late preterm and full term newborn babies at risk for early-onset sepsis. The calculator is based on a robust logistic regression model that provides quantitative individualised estimates of early-onset sepsis risk. Low sensitivity for prediction of sepsis at birth shows that standard perinatal risk factors alone are insufficient for ascertainment of neonatal early-onset sepsis. Performance is improved by the addition of physical examination findings at birth, but the sensitivity of combined findings remains limited. The present implementation of the calculator integrates risk factors and examination findings. A methodological error in adapting the regression for application in the population (rather than the development sample) and several subsequent modifications compromise the accuracy of quantitative predictions of the absolute risk of sepsis, but these factors are not expected to seriously undermine the use of the calculator for risk stratification. The calculator has served as an instrument of change away from previously recommended categorical risk ascertainment strategies, and its implementation reduces the need for diagnostic testing and empirical antibiotic treatment without apparent ill effects. However, the calculator should not be relied on to provide accurate estimates for individuals with regard to absolute risk of early-onset sepsis in newborn babies.

Predicting tibia shaft nonunions at initial fixation: An external validation of the Nonunion Risk Determination (NURD) score in the SPRINT trial data

BACKGROUND

Predictive models are common in orthopedic research; however, most models are not validated in an external population. The Nonunion Risk Determination (NURD) score was developed using a single-center cohort of 382 patients to reliably predict tibia shaft nonunions at the time of initial intramedullary nail fixation. The purpose of this study was to externally validate the NURD score using data from the SPRINT Trial.

METHODS

The SPRINT trial was a multicenter study comparing reamed versus unreamed intramedullary nails in tibial shaft fracture patients. We assessed the prognostic performance of the NURD score in the SPRINT trial data with comparisons of the c-statistics, calibration plots, and a comparison of predicted probabilities at cut-points defined in the study to derive the NURD score. In addition, we compared the odds ratios of the NURD score components between the derivation (NURD) and external validation (SPRINT) data.

RESULTS

The NURD score demonstrated significantly worse discrimination in the SPRINT data than was observed in the original data (c-statistic: 0.61 vs. 0.85, p<0.01). The NURD score was well-calibrated in the derivation and SPRINT data. The SPRINT data had less heterogeneity, as determined by the standard deviation of the linear predictors (NURD: 1.4 vs.

SPRINT

0.4). Once we adjusted for case-mix differences, the NURD score had similarly strong discrimination in the SPRINT data (c-statistic: 0.81 vs. 0.85, p = 0.17).

DISCUSSION

Based on our external validation, the NURD score lacks generalizability as it underperforms with respect to discrimination in the SPRINT trial data. However, after adjusting for case-mix differences, the performance of the NURD score is comparable between the two datasets, suggesting robust reproducibility.

The transfer of clinical prediction models for early trauma care had uncertain effects on mistriage

OBJECTIVES

This study aimed to assess how transfers of clinical prediction models for early trauma care between different care contexts within a single health system affected mistriage rates.

STUDY DESIGN AND SETTING

Patients aged 15 years or older, registered between 2011 and 2016 in the Swedish national trauma registry, SweTrau, were included. Three data set groups were created: high- and low-volume centers, metropolitan and nonmetropolitan centers, and multicenters and single centers. Clinical prediction models were developed using logistic regression in each data set group and transferred between data sets within groups. Model performance was evaluated using mistriage rate, undertriage rate, and overtriage rate. Multiple imputation using chained equations was used to handle missing data. Model performance was reported as medians with 95% confidence intervals (CIs).

RESULTS

A total of 26,965 patients were included. Changes in mistriage rates after transfer ranged from -0.25 (95% CI -0.21 to 0.04) to 0.29 (95% CI 0.13-0.39). Both overtriage and undertriage rates were affected.

CONCLUSIONS

Transferring clinical prediction models for early trauma care is associated with substantial uncertainty in regards to the effect on model performance. Depending on the care context, model transfer led to either increased or decreased mistriage. Overtriage was more affected by model transfer than undertriage.

Usefulness of an Artificial Neural Network in the Prediction of β-Lactam Allergy

BACKGROUND

An accurate diagnosis of β-lactam (BL) allergy improves the use of antibiotics, increases patients' safety, and reduces costs to health systems. Nevertheless, it requires skin and drug provocation tests, which are time-consuming and put the patient at risk. Furthermore, allergy testing is not available in circumstances such as the urgent need for antibiotic therapy.

OBJECTIVE

To evaluate the usefulness of an artificial neural network (ANN) in the prediction of hypersensitivity to BLs, and compare it with logistic regression (LR) analysis.

METHODS

In a single-center study, 656 patients evaluated for BL allergy between 1994 and 2000 were retrospectively analyzed, and the data were used to construct an ANN. The ANN predictive capabilities were compared with LR and then prospectively evaluated in 615 patients who underwent BL evaluation between 2011 and 2017.

RESULTS

A total of 1271 patients were evaluated. All patients had a definite diagnosis as allergic or nonallergic to BL. The prospective sample showed a lower percentage of patients with allergy than the retrospective sample (20.7% vs 25.8%; P = .018). In the retrospective and prospective series, the ANN reached a sensitivity of 89.5% and 81.1%, a specificity of 86.1% and 97.9%, a positive predictive value of 82.1% and 91.1%, and a negative predictive value of 92.1% and 95.2%, respectively. The ANN's performance was far superior to that of the LR, whose best performance reached a sensitivity of 31.9% and a specificity of 98.8%.

CONCLUSIONS

This ANN demonstrated a superior performance than the LR in predicting BL hypersensitivity without misdiagnosing severe allergic reactions. The ANN could be a helpful tool to classify the reaction risk, particularly in the identification of low-risk patients, in which an open challenge could be done to delabel patients.

A literature review of treatment-specific clinical prediction models in patients with breast cancer

Despite advances in the breast cancer treatment, significant variability in patient outcomes remain. This results in significant stress to patients and clinicians. Treatment-specific clinical prediction models allow patients to be matched against historical outcomes of patients with similar characteristics; thereby reducing uncertainty by providing personalised estimates of benefits, harms, and prognosis. To achieve this objective, models need to be clinical-grade with evidence of accuracy, reproducibility, generalizability, and be user-friendly. A structured search was undertaken to identify treatment-specific clinical prediction models for therapeutic or adverse outcomes in breast cancer using clinicopathological data. Significant gaps in the presence of validated models for available treatments was identified, along with gaps in prediction of therapeutic and adverse outcomes. Most models did not have user-friendly tools available. With the aim being to facilitate the selection of the best medicine for a specific patient and shared-decision making, future research will need to address these gaps.

The Modified Heidelberg and the AI Appendicitis Score Are Superior to Current Scores in Predicting Appendicitis in Children: A Two-Center Cohort Study

Background: Acute appendicitis represents the most frequent reason for abdominal surgery in children. Since diagnosis can be challenging various scoring systems have been published. The aim of this study was to evaluate and validate (and improve) different appendicitis scores in a very large cohort of children with abdominal pain. Methods: Retrospective analysis of all children that have been hospitalized due to suspected appendicitis at the Pediatric Surgery Department of the Altonaer Children's Hospital and University Medical Center Hamburg-Eppendorf from 01/2018 until 11/2019. Four different appendicitis scores (Heidelberg Appendicitis Score, Alvarado Score, Pediatric Appendicitis Score and Tzanakis Score) were applied to all data sets. Furthermore, the best score was improved and artificial intelligence (AI) was applied and compare the current scores. Results: In 23 months, 463 patients were included in the study. Of those 348 (75.2%) were operated for suspected appendicitis and in 336 (96.6%) patients the diagnosis was confirmed histopathologically. The best predictors of appendicitis (simple and perforated) were rebound tenderness, cough/hopping tenderness, ultrasound, and laboratory results. After modifying the HAS, it provided excellent results for simple (PPV 95.0%, NPV 70.0%) and very good for perforated appendicitis (PPV 34.4%, NPV 93.8%), outperforming all other appendicitis score. Discussion: The modified HAS and the AI score show excellent predictive capabilities and may be used to identify most cases of appendicitis and more important to rule out perforated appendicitis. The new scores outperform all other scores and are simple to apply. The modified HAS comprises five features that can all be assessed in the emergency department as opposed to current scores that are relatively complex to utilize in a clinical setting as they include of up to eight features with various weighting factors. In conclusion, the modified HAS and the AI score may be used to identify children with appendicitis, yet prospective studies to validate our findings in a large mutli-center cohorts are needed.

Factors associated with de novo metastatic disease in invasive breast cancer: comparison of artificial neural network and logistic regression models

Background

De novo metastasis of breast cancer is a complex clinical issue to be identified. This study was the first to construct artificial neural networks (ANN) and logistic regression (LR) models with comparison to find out important factors associated with occurrence of de novo metastasis in invasive breast cancer.

Methods

A total of 40,899 patients diagnosed with de novo metastatic breast cancer in 2010 from Surveillance, Epidemiology and End Results (SEER) Cancer database were enrolled. ANN models and LR models were constructed based on thirteen relevant factors by 10-fold cross-validation approach respectively. Evaluation indexes as well as processing time were compared.

Results

Overall area under ROC curve (AUC) value of ANN models was significantly higher than that of LR models (0.917±0.01 vs. 0.844±0.011, P<0.001). In ANN models, number of positive ipsilateral axillary lymph nodes, tumor size, lymph node ratio (LNR) and regional lymph nodes status were important associated factors. While under the same experiment environment, ANN models obviously took much more processing time than LR models did (14,400 vs. 15 minutes for 10-fold cross-validation).

Conclusions

ANN models outperformed traditional LR models in identifying de novo metastasis of breast cancer. On the other hand, the much longer processing time of ANN models should also be considered.

A comparison of logistic regression models with alternative machine learning methods to predict the risk of in-hospital mortality in emergency medical admissions via external validation

We compare the performance of logistic regression with several alternative machine learning methods to estimate the risk of death for patients following an emergency admission to hospital based on the patients' first blood test results and physiological measurements using an external validation approach. We trained and tested each model using data from one hospital (n = 24,696) and compared the performance of these models in data from another hospital (n = 13,477). We used two performance measures - the calibration slope and area under the receiver operating characteristic curve. The logistic model performed reasonably well - calibration slope: 0.90, area under the receiver operating characteristic curve: 0.847 compared to the other machine learning methods. Given the complexity of choosing tuning parameters of these methods, the performance of logistic regression with transformations for in-hospital mortality prediction was competitive with the best performing alternative machine learning methods with no evidence of overfitting.

The Heidelberg Appendicitis Score Predicts Perforated Appendicitis in Children

BACKGROUND

In the future, surgical management of pediatric appendicitis might become limited to nonperforating appendicitis. Thus, it becomes increasingly important to differentiate advanced from simple appendicitis and to predict perforated appendicitis among a group of children with right-sided abdominal pain, which was the aim of this study.

METHODS

An institutionally approved, single-center retrospective analysis of all patients with appendectomy from January 2009 to December 2010 was conducted. All diagnostic aspects were evaluated to identify predictors and differentiators of perforated appendicitis.

RESULTS

In 2 years, 157 children suffered from appendicitis. Perforation occurred in 47 (29.9%) of the patients. C-reactive protein (CRP) levels higher than 20 mg/dL ( P = .037) and free abdominal fluid on ultrasonography ( P = .031) are the most important features to differentiate perforated from simple appendicitis. Moreover, all children with perforation had a positive Heidelberg Appendicitis Score (HAS). A negative HAS excludes perforation in all cases (negative predictive value = 100%).

DISCUSSION

Perforated appendicitis can be ruled out by the HAS. In a cohort with right-sided abdominal pain, perforation should be considered in children with high CRP levels and free fluids or abscess formation on ultrasound.

Improving Hospital Performance Rankings Using Discrete Patient Diagnoses for Risk Adjustment of Outcomes

OBJECTIVE

To assess the changes in patient outcome prediction and hospital performance ranking when incorporating diagnoses as risk adjusters rather than comorbidity indices.

DATA SOURCES

Healthcare Cost and Utilization Project State Inpatient Databases for New York State, 2005-2009.

STUDY DESIGN

Conducted tree-based classification for mortality and readmission by incorporating discrete patient diagnoses as predictors, comparing with traditional comorbidity indices such as those used for Centers for Medicare and Medicaid Services (CMS) outcome models.

PRINCIPAL FINDINGS

Diagnosis codes as predictors increased predictive accuracy 5.6 percent (95% CI: 4.5-6.9 percent) relative to CMS condition categories for heart failure 30-day mortality. Most other outcomes exhibited statistically significant accuracy gains and facility ranking shifts. Sensitivity analysis showed improvements even when predictors were limited to only the diagnoses included in CMS models.

CONCLUSIONS

Discretizing patient severity information beyond the levels of traditional comorbidity indices improves patient outcome predictions and substantially shifts facility rankings.

Improved Prediction of the Pathologic Stage of Patient With Prostate Cancer Using the CART–PSO Optimization Analysis in the Korean Population

Objective:

In current practice, medical experts use the pathological stage predictions provided in the Partin tables to support their decisions. Hence, the Partin tables are based on logistic regression built from the US data. In the present study, we developed a data-mining model to predict the pathologic stage of prostate cancer. In this newly developed model, using the classification and regression tree-particle swarm optimization analysis of the Korean population data, we aim to improve the prediction accuracy of the pathologic state of prostate cancer.

Method:

A total of 467 patients from the smart prostate cancer database were evaluated. The results were intended to predict the pathologic stage of prostate cancer: organ-confined disease and non–organ-confined disease. The accuracy of 4 classification and regression tree-particle swarm optimization models was compared; furthermore, the models were validated with the Partin tables using the receiver operating characteristic curve.

Results:

Among the 467 evaluated patients, 235 patients had organ-confined disease and 232 patients had non–organ-confined disease. The area under the receiver operating characteristic curve of the proposed classification and regression tree-particle swarm optimization model (0.858 ± 0.034) was larger than the 1 in the Partin tables (0.666 ± 0.046).

Conclusion:

The proposed classification and regression tree-particle swarm optimization model was superior to the Partin tables in terms of predicting the risk of prostate cancer. Compared to the validation of the Partin tables for the Korean population, the classification and regression tree-particle swarm optimization model resulted in a larger receiver operating characteristic curve and a more accurate prediction of the pathologic stage of prostate cancer in the Korean population.

Comparisons of forecasting for hepatitis in Guangxi Province, China by using three neural networks models

This study compares and evaluates the prediction of hepatitis in Guangxi Province, China by using back propagation neural networks based genetic algorithm (BPNN-GA), generalized regression neural networks (GRNN), and wavelet neural networks (WNN). In order to compare the results of forecasting, the data obtained from 2004 to 2013 and 2014 were used as modeling and forecasting samples, respectively. The results show that when the small data set of hepatitis has seasonal fluctuation, the prediction result by BPNN-GA will be better than the two other methods. The WNN method is suitable for predicting the large data set of hepatitis that has seasonal fluctuation and the same for the GRNN method when the data increases steadily.

The Heidelberg Appendicitis Score Simplifies Identification of Pediatric Appendicitis

OBJECTIVE

To identify the factors that facilitate the diagnosis of pediatric appendicitis.

METHODS

Institutionally approved retrospective, single center analysis of all patients with acute abdominal pain was done. Medical history, symptoms, laboratory and radiologic findings of all children presenting with abdominal pain were evaluated. To identify the best predictors, uni- and multi-variate analysis were used.

RESULTS

In 2 years, 431 patients fulfilled the inclusion criteria. Data was complete in all subjects. Of these, 156 (36.2 %) suffered from appendicitis. The best discriminators for appendicitis were clinical and ultrasound features. The four best factors were identified by CART analysis (continuous abdominal pain, tenderness on the right lower quadrant, rebound tenderness and conspicuous ultrasound) and combined to the Heidelberg Appendicitis score. A positive score (>3 features) is highly predictive for acute appendicitis (PPV 89.3 %, NPV 94.9 %) and includes all cases of perforated appendicitis.

CONCLUSIONS

It is possible to predict acute appendicitis in children. The decision making process can be simplified by the proposed Heidelberg Appendicitis score, which is comprised of four factors. It has great potential to facilitate and accelerate the diagnosis of pediatric appendicitis.

Generalizing results of randomized trials to clinical practice: reliability and cautions

BackgroundWell designed randomized controlled trials provide reliable evidence of treatment effects, but there is no consensus on how best to apply these results to clinical practice. The main concerns are that populations enrolled in trials are more selected than those treated in a clinical setting, and whether the treatment effects observed in trials will also be observed in clinical practice.

MethodsAn informal literature review was undertaken to find studies analysing the issue of generalizing trial results (external validity) to clinical practice.

ResultsMost of the studies focused on differences in patients characteristics (age, gender, severity of disease, concomitant treatments and so on) between the clinical trial population and a ‘real world’ clinical population. None provided good evidence of a reduction in the treatment effect in the trial compared to what might happen in clinical practice for simple pharmacological treatments. However complex treatments like surgery or percutaneous interventional procedures, had a greater potential for variation. Extrapolating treatments to different health care settings from the trial can result in important variations in treatment effects.

ConclusionsComplex therapies need careful consideration before they can be applied routinely from trials into practice, and applying results from one health care environment to a different one should be carried out with caution. Generalizing results from well conducted trials to clinical practice can mostly be carried out with confidence, especially for simple therapies with good evidence of benefit.

Modern modeling techniques had limited external validity in predicting mortality from traumatic brain injury

BACKGROUND AND OBJECTIVE

Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity.

METHODS

We analyzed individual patient data from 15 different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score, and pupillary reactivity), an extended set with computed tomography scan characteristics, and a further extension with two laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modeling techniques: logistic regression (LR), classification and regression trees, random forests (RFs), support vector machines (SVM) and neural nets. For external validation, a model developed on one of the 15 data sets was applied to each of the 14 remaining sets. This process was repeated 15 times for a total of 630 validations. The area under the receiver operating characteristic curve (AUC) was used to assess the discriminative ability of the models.

RESULTS

For the most complex predictor set, the LR models performed best (median validated AUC value, 0.757), followed by RF and support vector machine models (median validated AUC value, 0.735 and 0.732, respectively). With each predictor set, the classification and regression trees models showed poor performance (median validated AUC value, <0.7). The variability in performance across the studies was smallest for the RF- and LR-based models (inter quartile range for validated AUC values from 0.07 to 0.10).

CONCLUSION

In the area of predicting mortality from TBI, nonlinear and nonadditive effects are not pronounced enough to make modern prediction methods beneficial.

Proliferative vitreoretinopathy: A new concept of disease pathogenesis and practical consequences

During the last four decades, proliferative vitreoretinopathy (PVR) has defied the efforts of many researchers to prevent its occurrence or development. Thus, PVR is still the major complication following retinal detachment (RD) surgery and a bottle-neck for advances in cell therapy that require intraocular surgery. In this review we tried to combine basic and clinical knowledge, as an example of translational research, providing new and practical information for clinicians. PVR was defined as the proliferation of cells after RD. This idea was used for classifying PVR and also for designing experimental models used for testing many drugs, none of which were successful in humans. We summarize current information regarding the pathogenic events that follow any RD because this information may be the key for understanding and treating the earliest stages of PVR. A major focus is made on the intraretinal changes derived mainly from retinal glial cell reactivity. These responses can lead to intraretinal PVR, an entity that has not been clearly recognized. Inflammation is one of the major components of PVR, and we describe new genetic biomarkers that have the potential to predict its development. New treatment approaches are analyzed, especially those directed towards neuroprotection, which can also be useful for preventing visual loss after any RD. We also summarize the results of different surgical techniques and clinical information that is oriented toward the identification of high risk patients. Finally, we provide some recommendations for future classification of PVR and for designing comparable protocols for testing new drugs or techniques.

Application of a hybrid method combining grey model and back propagation artificial neural networks to forecast hepatitis B in china

Accurate incidence forecasting of infectious disease provides potentially valuable insights in its own right. It is critical for early prevention and may contribute to health services management and syndrome surveillance. This study investigates the use of a hybrid algorithm combining grey model (GM) and back propagation artificial neural networks (BP-ANN) to forecast hepatitis B in China based on the yearly numbers of hepatitis B and to evaluate the method's feasibility. The results showed that the proposal method has advantages over GM (1, 1) and GM (2, 1) in all the evaluation indexes.

Validation of the APACHE IV model and its comparison with the APACHE II, SAPS 3, and Korean SAPS 3 models for the prediction of hospital mortality in a Korean surgical intensive care unit

Background

The Acute Physiology and Chronic Health Evaluation (APACHE) IV model has not yet been validated in Korea. The aim of this study was to compare the ability of the APACHE IV with those of APACHE II, Simplified Acute Physiology Score (SAPS) 3, and Korean SAPS 3 in predicting hospital mortality in a surgical intensive care unit (SICU) population.

Methods

We retrospectively reviewed electronic medical records for patients admitted to the SICU from March 2011 to February 2012 in a university hospital. Measurements of discrimination and calibration were performed using the area under the receiver operating characteristic curve (AUC) and the Hosmer-Lemeshow test, respectively. We calculated the standardized mortality ratio (SMR, actual mortality predicted mortality) for the four models.

Results

The study included 1,314 patients. The hospital mortality rate was 3.3%. The discriminative powers of all models were similar and very reliable. The AUCs were 0.80 for APACHE IV, 0.85 for APACHE II, 0.86 for SAPS 3, and 0.86 for Korean SAPS 3. Hosmer and Lemeshow C and H statistics showed poor calibration for all of the models (P < 0.05). The SMRs of APACHE IV, APACHE II, SAPS 3, and Korean SAPS 3 were 0.21, 0.11 0.23, 0.34, and 0.25, respectively.

Conclusions

The APACHE IV revealed good discrimination but poor calibration. The overall discrimination and calibration of APACHE IV were similar to those of APACHE II, SAPS 3, and Korean SAPS 3 in this study. A high level of customization is required to improve calibration in this study setting.

Early prediction of complex midgut volvulus in neonates and infants

INTRODUCTION

Prognosis of midgut volvulus in neonates and infants younger than 1 year remains poor, as diagnostic findings may not be apparent until gut infarction had occurred. To characterize factors that help to predict complex midgut volvulus early was aim of this study.

METHODS

Institutionally approved retrospective analysis of all children younger than 1 year treated for midgut volvulus at the author's center from January 2002 to December 2011. Medical history, symptoms, laboratory and radiologic findings as well as sequelae of midgut volvulus were evaluated.

RESULTS

In 10 years, 37 children fulfilled the inclusion criteria. Of these, 43% developed complications, and mortality rate was 16%. In 30% of the patients, the only clinical sign was a sudden worsening of the general condition and abdominal distension (complex 19% vs. simple 38%). In one child with simple midgut volvulus, all clinical, laboratory and radiologic signs were negative. CART analysis identified a base excess below -1.70 and preterm birth (<36 weeks) as the best discriminators of complex and simple midgut volvulus. A score >1pt (comprised of these two factors) was found in all children with complex and in 14% of simple midgut volvulus (p < 0.001). A positive score (>1pt) offers a sensitivity of 100% (81.7-100%), specificity of 85.7% (71.8-85.7%), a PPV of 84.2% (68.8-84.2%) and NPV 100% (83.8-100%).

DISCUSSION

The study shows that midgut volvulus has a substantial morbidity and mortality. Unfortunately, not all affected children get picked up by history, laboratory and imaging. However, the proposed score helps to identify subject with increased risk of complications. It has the potential to facilitate and accelerate diagnosis of complex midgut volvulus; ultimately, it might help to reduce morbidity and mortality.

External validation of existing formulas to predict the risk of developing proliferative vitreoretinopathy: the Retina 1 Project; report 5

PURPOSE

To externally validate the accuracy of previously published formulas for predicting proliferative vitreoretinopathy development after retinal detachment surgery.

METHODS

Clinical variables from consecutive retinal detachment patients (n = 1,047) were collected from the Retina 1 Project conducted in 17 Spanish and Portuguese centers. These data were used for external validation of four previously published formulas, F1 to F4. Receiver-operating characteristic curves were used to validate the quality of formulas, and measures of discrimination, precision, and calibration were calculated for each. Concordance among the formulas was determined by Cohen kappa index.

RESULTS

The areas under the receiver-operating characteristic curves were as follows: F1, 0.5809; F2, 0.5398; F3, 0.5964; and F4, 0.4617. F1 had the highest accuracy, 74.21%. Almost 19% of proliferative vitreoretinopathy cases were correctly classified by F1 compared with 13%, 15%, and 10% for F2, F3, and F4, respectively. There was moderate concordance between F2 and F3 but little between the other formulas.

CONCLUSION

After external validation, none of the formulas were accurate enough for routine clinical use. To increase its usefulness, other factors besides the clinical ones considered here should be incorporated into future formulas for predicting risk of developing proliferative vitreoretinopathy.

Artificial neural network models for predicting 1-year mortality in elderly patients with intertrochanteric fractures in China

The mortality rate of older patients with intertrochanteric fractures has been increasing with the aging of populations in China. The purpose of this study was: 1) to develop an artificial neural network (ANN) using clinical information to predict the 1-year mortality of elderly patients with intertrochanteric fractures, and 2) to compare the ANN's predictive ability with that of logistic regression models. The ANN model was tested against actual outcomes of an intertrochanteric femoral fracture database in China. The ANN model was generated with eight clinical inputs and a single output. ANN's performance was compared with a logistic regression model created with the same inputs in terms of accuracy, sensitivity, specificity, and discriminability. The study population was composed of 2150 patients (679 males and 1471 females): 1432 in the training group and 718 new patients in the testing group. The ANN model that had eight neurons in the hidden layer had the highest accuracies among the four ANN models: 92.46 and 85.79% in both training and testing datasets, respectively. The areas under the receiver operating characteristic curves of the automatically selected ANN model for both datasets were 0.901 (95%CI=0.814-0.988) and 0.869 (95%CI=0.748-0.990), higher than the 0.745 (95%CI=0.612-0.879) and 0.728 (95%CI=0.595-0.862) of the logistic regression model. The ANN model can be used for predicting 1-year mortality in elderly patients with intertrochanteric fractures. It outperformed a logistic regression on multiple performance measures when given the same variables.

Comparison of prediction model for cardiovascular autonomic dysfunction using artificial neural network and logistic regression analysis

Background

This study aimed to develop the artificial neural network (ANN) and multivariable logistic regression (LR) analyses for prediction modeling of cardiovascular autonomic (CA) dysfunction in the general population, and compare the prediction models using the two approaches.

Methods and Materials

We analyzed a previous dataset based on a Chinese population sample consisting of 2,092 individuals aged 30–80 years. The prediction models were derived from an exploratory set using ANN and LR analysis, and were tested in the validation set. Performances of these prediction models were then compared.

Results

Univariate analysis indicated that 14 risk factors showed statistically significant association with the prevalence of CA dysfunction (P<0.05). The mean area under the receiver-operating curve was 0.758 (95% CI 0.724–0.793) for LR and 0.762 (95% CI 0.732–0.793) for ANN analysis, but noninferiority result was found (P<0.001). The similar results were found in comparisons of sensitivity, specificity, and predictive values in the prediction models between the LR and ANN analyses.

Conclusion

The prediction models for CA dysfunction were developed using ANN and LR. ANN and LR are two effective tools for developing prediction models based on our dataset.

Artificial neural network models for prediction of cardiovascular autonomic dysfunction in general Chinese population

BACKGROUND

The present study aimed to develop an artificial neural network (ANN) based prediction model for cardiovascular autonomic (CA) dysfunction in the general population.

METHODS

We analyzed a previous dataset based on a population sample consisted of 2,092 individuals aged 30-80 years. The prediction models were derived from an exploratory set using ANN analysis. Performances of these prediction models were evaluated in the validation set.

RESULTS

Univariate analysis indicated that 14 risk factors showed statistically significant association with CA dysfunction (P < 0.05). The mean area under the receiver-operating curve was 0.762 (95% CI 0.732-0.793) for prediction model developed using ANN analysis. The mean sensitivity, specificity, positive and negative predictive values were similar in the prediction models was 0.751, 0.665, 0.330 and 0.924, respectively. All HL statistics were less than 15.0.

CONCLUSION

ANN is an effective tool for developing prediction models with high value for predicting CA dysfunction among the general population.

In-hospital mortality after traumatic brain injury surgery: a nationwide population-based comparison of mortality predictors used in artificial neural network and logistic regression models

Object

Most reports compare artificial neural network (ANN) models and logistic regression models in only a single data set, and the essential issue of internal validity (reproducibility) of the models has not been adequately addressed. This study proposes to validate the use of the ANN model for predicting in-hospital mortality after traumatic brain injury (TBI) surgery and to compare the predictive accuracy of ANN with that of the logistic regression model.

Methods

The authors of this study retrospectively analyzed 16,956 patients with TBI nationwide who were surgically treated in Taiwan between 1998 and 2009. For every 1000 pairs of ANN and logistic regression models, the area under the receiver operating characteristic curve (AUC), Hosmer-Lemeshow statistics, and accuracy rate were calculated and compared using paired t-tests. A global sensitivity analysis was also performed to assess the relative importance of input parameters in the ANN model and to rank the variables in order of importance.

Results

The ANN model outperformed the logistic regression model in terms of accuracy in 95.15% of cases, in terms of Hosmer-Lemeshow statistics in 43.68% of cases, and in terms of the AUC in 89.14% of cases. The global sensitivity analysis of in-hospital mortality also showed that the most influential (sensitive) parameters in the ANN model were surgeon volume followed by hospital volume, Charlson comorbidity index score, length of stay, sex, and age.

Conclusions

This work supports the continued use of ANNs for predictive modeling of neurosurgery outcomes. However, further studies are needed to confirm the clinical efficacy of the proposed model.

Therapy of bovine endometritis with prostaglandin F2α: a meta-analysis

The objective of the conducted meta-analysis was to assess the efficacy of the treatment of bovine endometritis with PGF(2α) by statistical means. Postpartum uterine infections have a high prevalence and a very negative effect on reproductive performance in dairy cattle. Because of a wide discordance between research results, a meta-analysis of the efficacy of the treatment of bovine endometritis with PGF(2α) was conducted. A comprehensive literature search was performed using online databases to reveal a total of 2,307 references. In addition, 5 articles were retrieved by reviewing citations. After applying specific exclusion criteria and evaluating specific evidence parameters, 5 publications, comprising 6 trials, were eligible for being analyzed by means of meta-analysis. Data for each trial were extracted and analyzed using meta-analysis software Review Manager (version 5.1; The Nordic Cochrane Centre, Copenhagen, Denmark). Estimated effect sizes of PGF(2α) were calculated on calving to first service and calving to conception interval. Prostaglandin F(2α) treatment of cows with chronic endometritis had a negative effect on both reproductive performance parameters. Heterogeneity was substantial for calving to first service and calving to conception interval [I(2) (measure of variation beyond chance)=100 and 87%, respectively]; therefore, random-effects models were used. Sensitivity analysis as well as subgroup analysis showed that the performance of randomization was influential in modifying effect size of PGF(2α) treatment. The funnel plot illustrated a publication bias toward smaller studies that reported a prolonged calving to conception interval after a PGF(2α) treatment. We conclude that the investigation of this subject by means of meta-analysis did not reveal an improvement of reproductive performance of cows with endometritis after treatment with PGF(2α). Furthermore, there is a shortage of comparable high quality studies investigating reproductive performance after PGF(2α) treatment of cows with chronic endometritis.

External validation of two prediction models identifying employees at risk of high sickness absence: cohort study with 1-year follow-up

BACKGROUND

Two models including age, self-rated health (SRH) and prior sickness absence (SA) were found to predict high SA in health care workers. The present study externally validated these prediction models in a population of office workers and investigated the effect of adding gender as a predictor.

METHODS

SRH was assessed at baseline in a convenience sample of office workers. Age, gender and prior SA were retrieved from an occupational health service register. Two pre-defined prediction models were externally validated: a model identifying employees with high (i.e. ≥30) SA days and a model identifying employees with high (i.e. ≥3) SA episodes during 1-year follow-up. Calibration was investigated by plotting the predicted and observed probabilities and calculating the calibration slope. Discrimination was examined by receiver operating characteristic (ROC) analysis and the area under the ROC-curve (AUC).

RESULTS

A total of 593 office workers had complete data and were eligible for analysis. Although the SA days model showed acceptable calibration (slope = 0.89), it poorly discriminated office workers with high SA days from those without high SA days (AUC = 0.65; 95% CI 0.58-0.71). The SA episodes model showed acceptable discrimination (AUC = 0.76, 95% CI 0.70-0.82) and calibration (slope = 0.96). The prognostic performance of the prediction models did not improve in the population of office workers after adding gender.

CONCLUSION

The SA episodes model accurately predicted the risk of high SA episodes in office workers, but needs further multisite validation and requires a simpler presentation format before it can be used to select high-risk employees for interventions to prevent or reduce SA.

A comparative study on entrepreneurial attitudes modeled with logistic regression and Bayes nets

Entrepreneurship research is receiving increasing attention in our context, as entrepreneurs are key social agents involved in economic development. We compare the success of the dichotomic logistic regression model and the Bayes simple classifier to predict entrepreneurship, after manipulating the percentage of missing data and the level of categorization in predictors. A sample of undergraduate university students (N = 1230) completed five scales (motivation, attitude towards business creation, obstacles, deficiencies, and training needs) and we found that each of them predicted different aspects of the tendency to business creation. Additionally, our results show that the receiver operating characteristic (ROC) curve is affected by the rate of missing data in both techniques, but logistic regression seems to be more vulnerable when faced with missing data, whereas Bayes nets underperform slightly when categorization has been manipulated. Our study sheds light on the potential entrepreneur profile and we propose to use Bayesian networks as an additional alternative to overcome the weaknesses of logistic regression when missing data are present in applied research.

The use of regional platforms for managing electronic health records for the production of regional public health indicators in France

Background

In France, recent developments in healthcare system organization have aimed at strengthening decision-making and action in public health at the regional level. Firstly, the 2004 Public Health Act, by setting 100 national and regional public health targets, introduced an evaluative approach to public health programs at the national and regional levels. Meanwhile, the implementation of regional platforms for managing electronic health records (EHRs) has also been under assessment to coordinate the deployment of this important instrument of care within each geographic area. In this context, the development and implementation of a regional approach to epidemiological data extracted from EHRs are an opportunity that must be seized as soon as possible. Our article addresses certain design and organizational aspects so that the technical requirements for such use are integrated into regional platforms in France. The article will base itself on organization of the Rhône-Alpes regional health platform.

Discussion

Different tools being deployed in France allow us to consider the potential of these regional platforms for epidemiology and public health (implementation of a national health identification number and a national information system interoperability framework). The deployment of the Rhône-Alpes regional health platform began in the 2000s in France. By August 2011, 2.6 million patients were identified in this platform. A new development step is emerging because regional decision-makers need to measure healthcare efficiency. To pool heterogeneous information contained in various independent databases, the format, norm and content of the metadata have been defined. Two types of databases will be created according to the nature of the data processed, one for extracting structured data, and the second for extracting non-structured and de-identified free-text documents.

Summary

Regional platforms for managing EHRs could constitute an important data source for epidemiological surveillance in the context of epidemic alerts, but also in monitoring a number of indicators of infectious and chronic diseases for which no data are yet available in France.

Enhancing the fever workup utilizing a multi-technique modeling approach to diagnose infections more accurately

BACKGROUND

Differentiation between infectious and non-infectious etiologies of the systemic inflammatory response syndrome (SIRS) in trauma patients remains elusive. We hypothesized that mathematical modeling in combination with computerized clinical decision support would assist with this differentiation. The purpose of this study was to determine the capability of various mathematical modeling techniques to predict infectious complications in critically ill trauma patients and compare the performance of these models with a standard fever workup practice (identifying infections on the basis of fever or leukocytosis).

METHODS

An 18-mo retrospective database was created using information collected daily from critically ill trauma patients admitted to an academic surgical and trauma intensive care unit. Two hundred forty-three non-infected patient-days were chosen randomly to combine with the 243 infected-days, which created a modeling sample of 486 patient-days. Utilizing ten variables known to be associated with infectious complications, decision trees, neural networks, and logistic regression analysis models were created to predict the presence of urinary tract infections (UTIs), bacteremia, and respiratory tract infections (RTIs). The data sample was split into a 70% training set and a 30% testing set. Models were compared by calculating sensitivity, specificity, positive predictive value, negative predictive value, overall accuracy, and discrimination.

RESULTS

Decision trees had the best modeling performance, with a sensitivity of 83%, an accuracy of 82%, and a discrimination of 0.91 for identifying infections. Both neural networks and decision trees outperformed logistic regression analysis. A second analysis was performed utilizing the same 243 infected days and only those non-infected patient-days associated with negative microbiologic cultures (n = 236). Decision trees again had the best modeling performance for infection identification, with a sensitivity of 79%, an accuracy of 83%, and a discrimination of 0.87.

CONCLUSION

The use of mathematical modeling techniques beyond logistic regression can improve the robustness and accuracy of predicting infections in critically ill trauma patients. Decision tree analysis appears to have the best potential to use in assisting physicians in differentiating infectious from non-infectious SIRS.

Formalized prediction of clinically significant prostate cancer: is it possible?

Greater understanding of the biology and epidemiology of prostate cancer in the last several decades have led to significant advances in its management. Prostate cancer is now detected in greater numbers at lower stages of disease and is amenable to multiple forms of efficacious treatment. However, there is a lack of conclusive data demonstrating a definitive mortality benefit from this earlier diagnosis and treatment of prostate cancer. It is likely due to the treatment of a large proportion of indolent cancers that would have had little adverse impact on health or lifespan if left alone. Due to this overtreatment phenomenon, active surveillance with delayed intervention is gaining traction as a viable management approach in contemporary practice. The ability to distinguish clinically insignificant cancers from those with a high risk of progression and/or lethality is critical to the appropriate selection of patients for surveillance protocols versus immediate intervention. This chapter will review the ability of various prediction models, including risk groupings and nomograms, to predict indolent disease and determine their role in the contemporary management of clinically localized prostate cancer.

Disease-free survival after hepatic resection in hepatocellular carcinoma patients: a prediction approach using artificial neural network

Background

A database for hepatocellular carcinoma (HCC) patients who had received hepatic resection was used to develop prediction models for 1-, 3- and 5-year disease-free survival based on a set of clinical parameters for this patient group.

Methods

The three prediction models included an artificial neural network (ANN) model, a logistic regression (LR) model, and a decision tree (DT) model. Data for 427, 354 and 297 HCC patients with histories of 1-, 3- and 5-year disease-free survival after hepatic resection, respectively, were extracted from the HCC patient database. From each of the three groups, 80% of the cases (342, 283 and 238 cases of 1-, 3- and 5-year disease-free survival, respectively) were selected to provide training data for the prediction models. The remaining 20% of cases in each group (85, 71 and 59 cases in the three respective groups) were assigned to validation groups for performance comparisons of the three models. Area under receiver operating characteristics curve (AUROC) was used as the performance index for evaluating the three models.

Conclusions

The ANN model outperformed the LR and DT models in terms of prediction accuracy. This study demonstrated the feasibility of using ANNs in medical decision support systems for predicting disease-free survival based on clinical databases in HCC patients who have received hepatic resection.

A Comparison of Intensive Care Unit Mortality Prediction Models through the Use of Data Mining Techniques

OBJECTIVES

The intensive care environment generates a wealth of critical care data suited to developing a well-calibrated prediction tool. This study was done to develop an intensive care unit (ICU) mortality prediction model built on University of Kentucky Hospital (UKH)'s data and to assess whether the performance of various data mining techniques, such as the artificial neural network (ANN), support vector machine (SVM) and decision trees (DT), outperform the conventional logistic regression (LR) statistical model.

METHODS

The models were built on ICU data collected regarding 38,474 admissions to the UKH between January 1998 and September 2007. The first 24 hours of the ICU admission data were used, including patient demographics, admission information, physiology data, chronic health items, and outcome information.

RESULTS

Only 15 study variables were identified as significant for inclusion in the model development. The DT algorithm slightly outperformed (AUC, 0.892) the other data mining techniques, followed by the ANN (AUC, 0.874), and SVM (AUC, 0.876), compared to that of the APACHE III performance (AUC, 0.871).

CONCLUSIONS

With fewer variables needed, the machine learning algorithms that we developed were proven to be as good as the conventional APACHE III prediction.

Use of nomograms for predictions of outcome in patients with advanced bladder cancer

INTRODUCTION

Accurate estimates of risk are essential for physicians if they are to recommend a specific management to patients with bladder cancer. In this review, we discuss the criteria for the evaluation of nomograms and review current available nomograms for advanced bladder cancer.

METHODS

A retrospective review of the Pubmed database between 2002 and 2008 was performed using the keywords 'nomogram' and 'bladder'. We limited the articles to advanced bladder cancer. We recorded input variables, prediction form, number of patients used to develop the prediction tools, the outcome being predicted, prediction tool-specific features, predictive accuracy, and whether validation was performed.

RESULTS

We discuss the characteristics needed to evaluate nomograms such as predictive accuracy, calibration, generalizability, level of complexity, effect of competing risks, conditional probabilities, and head-to-head comparison with other prediction methods. The predictive accuracies of the pre-cystectomy tools (n = 2) range from ∼65-75% and that of the post-cystectomy tools (n = 5) range from ∼75-80%. While some of these nomograms are well-calibrated and outperform AJCC staging, none has been externally validated. To date, four studies demonstrated a statistically significant improvement in predictive accuracy of nomograms by including biomarkers.

CONCLUSIONS

Nomograms provide accurate individualized estimates of outcomes. They currently represent the most accurate and discriminatory decision-making aids tools for predicting outcomes in patients with bladder cancer. Use of current nomograms could improve current selection of patients for standard therapy and investigational trial design by ensuring homogeneous groups. The addition of biological markers to the currently available nomograms using clinical and pathologic data holds the promise of improving prediction and refining management of patients with bladder cancer.

Statistical process control for validating a classification tree model for predicting mortality--a novel approach towards temporal validation

Prediction models are postulated as useful tools to support tasks such as clinical decision making and benchmarking. In particular, classification tree models have enjoyed much interest in the Biomedical Informatics literature. However, their prospective predictive performance over the course of time has not been investigated. In this paper we suggest and apply statistical process control methods to monitor over more than 5 years the prospective predictive performance of TM80+, one of the few classification-tree models published in the clinical literature. TM80+ is a model for predicting mortality among very elderly patients in the intensive care based on a multi-center dataset. We also inspect the predictive performance at the tree's leaves. This study provides important insights into patterns of (in)stability of the tree's performance and its "shelf life". The study underlies the importance of continuous validation of prognostic models over time using statistical tools and the timely recalibration of tree models.

Prediction of chronic allograft nephropathy using classification trees

INTRODUCTION

For its intrinsic potential to mine causal relations, machine learning techniques are useful to identify new risk indicators. In this work, we have shown two classification trees to predict chronic allograft nephropathy (CAN), through an evaluation of routine blood and urine tests.

METHODS

We retrospectively analyzed 80 renal transplant patients with 60-month follow-up (mean = 55.20 +/- 12.74) including 52 males and 28 females of overall average age of 41.65 +/- 12.52 years. The primary endpoint was biopsy-proven CAN within 5 years from transplantation (n = 16). Exclusion criteria were multiorgan transplantations, patients aged less than 18 years, graft failure, or patient death in the first 6 months posttransplantation. Classification trees based on the C 4.8 algorithm were used to predict CAN development starting from patient features at transplantation and biochemical test at 6-month follow-up. Model performance was showed as sensitivity (S), false-positive rate (FPR), and area under the receiver operating characteristic curve (AUC).

RESULTS

The two class of patients (no CAN versus CAN) showed significant differences in serum creatinine, estimated Glomerular Filtration Rate with Modification of Diet in Renal Disease study formula (MDRD), serum hemoglobin, hematocrit, blood urea nitrogen, and 24-hour urine protein excretion. Among the 23 evaluated variables, the first model selected six predictors of CAN, showing S = 62.5%, TFP = 7.2%, and AUC = 0.847 (confidence interval [CI] 0.749-0.945). The second model selected four variables, showing S = 81.3%, TFP = 25%, and AUC = 0.824 (CI 0.713-0.934).

CONCLUSIONS

Identification models have predicted the onset of multifactorial, complex pathology, like CAN. The use of classification trees represent a valid alternative to traditional statistical models, especially for the evaluation of interactions of risk factors.