On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology

Bladder Cancer and Artificial Intelligence: Emerging Applications

Bladder cancer is a common and heterogeneous disease that poses a significant burden to the patient and health care system. Major unmet needs include effective early detection strategy, imprecision of risk stratification, and treatment-associated morbidities. The existing clinical paradigm is imprecise, which results in missed tumors, suboptimal therapy, and disease progression. Artificial intelligence holds immense potential to address many unmet needs in bladder cancer, including early detection, risk stratification, treatment planning, quality assessment, and outcome prediction. Despite recent advances, extensive work remains to affirm the efficacy of artificial intelligence as a decision-making tool for bladder cancer management.

Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images

BACKGROUND

Applying deep learning to digital histopathology is hindered by the scarcity of manually annotated datasets. While data augmentation can ameliorate this obstacle, its methods are far from standardized. Our aim was to systematically explore the effects of skipping data augmentation; applying data augmentation to different subsets of the whole dataset (training set, validation set, test set, two of them, or all of them); and applying data augmentation at different time points (before, during, or after dividing the dataset into three subsets). Different combinations of the above possibilities resulted in 11 ways to apply augmentation. The literature contains no such comprehensive systematic comparison of these augmentation ways.

RESULTS

Non-overlapping photographs of all tissues on 90 hematoxylin-and-eosin-stained urinary bladder slides were obtained. Then, they were manually classified as either inflammation (5948 images), urothelial cell carcinoma (5811 images), or invalid (3132 images; excluded). If done, augmentation was eight-fold by flipping and rotation. Four convolutional neural networks (Inception-v3, ResNet-101, GoogLeNet, and SqueezeNet), pre-trained on the ImageNet dataset, were fine-tuned to binary classify images of our dataset. This task was the benchmark for our experiments. Model testing performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. Model validation accuracy was also estimated. The best testing performance was achieved when augmentation was done to the remaining data after test-set separation, but before division into training and validation sets. This leaked information between the training and the validation sets, as evidenced by the optimistic validation accuracy. However, this leakage did not cause the validation set to malfunction. Augmentation before test-set separation led to optimistic results. Test-set augmentation yielded more accurate evaluation metrics with less uncertainty. Inception-v3 had the best overall testing performance.

CONCLUSIONS

In digital histopathology, augmentation should include both the test set (after its allocation), and the remaining combined training/validation set (before being split into separate training and validation sets). Future research should try to generalize our results.

A field-deployable water quality monitoring with machine learning-based smartphone colorimetry

Water quality monitoring is an increasing global concern as the pollution of water sources causes adverse effects on economic growth and human health. Traditional approaches to the detection of pollutants are time-consuming and labor-intensive due to the requirement of sophisticated equipment or laboratory settings. Therefore, portable devices featuring rapid response and easy operation are indispensable in water quality monitoring. Herein, smartphone-based colorimetric pollutant quantification is demonstrated in a machine learning (ML) framework. As a proof of concept, the presence of seven ions in water was analyzed using colorimetric strips. The color variation on the strip indicators was captured under eight lighting conditions with five smartphones, providing robustness against the illumination variation and camera optics for ML classifiers. Color and texture features were extracted from the images to train the classifiers. Among the twenty-three classifiers, K-Nearest Neighbors exhibits the best classification performance, leading to the integration with our custom-designed Android application called Hydro Sens. The proposed approach was also tested with real samples taken from local water sources. The results prove that incorporating color strips with ML with a smartphone application can be used for water quality monitoring, which offers promising alternatives for sophisticated equipment that is especially applicable in resource-limited settings.

Avoiding C-hacking when evaluating survival distribution predictions with discrimination measures

MOTIVATION

In this article, we consider how to evaluate survival distribution predictions with measures of discrimination. This is non-trivial as discrimination measures are the most commonly used in survival analysis and yet there is no clear method to derive a risk prediction from a distribution prediction. We survey methods proposed in literature and software and consider their respective advantages and disadvantages.

RESULTS

Whilst distributions are frequently evaluated by discrimination measures, we find that the method for doing so is rarely described in the literature and often leads to unfair comparisons or 'C-hacking'. We demonstrate by example how simple it can be to manipulate results and use this to argue for better reporting guidelines and transparency in the literature. We recommend that machine learning survival analysis software implements clear transformations between distribution and risk predictions in order to allow more transparent and accessible model evaluation.

AVAILABILITY AND IMPLEMENTATION

The code used in the final experiment is available at https://github.com/RaphaelS1/distribution_discrimination.

Research on the Application of Artificial Neural Network-Based Virtual Image Technology in College Tennis Teaching

At the same time that my country has shifted from high-speed development to high-quality development, my country has also put forward new requirements for education development. Due to the limited study time during college, each student's study habits and learning process are also different, and the degree of connection between tennis lessons is high, so there will be polarization when learning tennis. With the development of science and technology, more and more technological innovations are integrated into the classroom, and traditional teaching methods can no longer keep up with the pace of the times. Tennis teaching is a subject of equal proportion between theory and practice. The traditional teaching method simplifies the theory, which makes students to have some bad phenomena when they practice. Aiming at this series of problems, this paper uses algorithms such as softmax function and threshold function to construct an application model of virtual image technology based on the artificial neural network in tennis teaching. The research results of the article show that: (1) the average accuracy rate of the method in this paper is 97.22%, and the highest accuracy rate is 99.17%. The average accuracy rate also tends to increase with the increase of sample size; the recall rate is the highest, and the highest recall rate is 99.36%. The average recall rate is 96.77%; the highest correct rate is close to 100% and is significantly higher than the other three methods; the average correct rate reaches 98.8%; the response time is the shortest; the average response time is 33 ms; and the response time increases with the increase of the sample size. (2) After using this model, tennis skills have been improved, with an average of 12 in situ flips, an average of 7 in situ rackets, an average of 5 in situ forehand draws, and an average of 3 in situ backhand draws. (3) The average forehand and backhand scores of the class after the experiment were 90 and 86; the average forehand and backhand stability were 8 and 7; and the average forehand and backhand accuracy were 31 and 29, respectively. The average depth of forehand and backhand is 36 and 32. (4) Most of the students are satisfied with this model, and they all choose to strongly agree and relatively agree, and the percentage of very agree that helps stimulate learning has reached 60.52%, and no students choose to disagree very much.

A noninvasive artificial neural network model to predict IgA nephropathy risk in Chinese population

Renal biopsy is the gold standard for Immunoglobulin A nephropathy (IgAN) but poses several problems. Thus, we aimed to establish a noninvasive model for predicting the risk probability of IgAN by analyzing routine and serological parameters. A total of 519 biopsy-diagnosed IgAN and 211 non-IgAN patients were recruited retrospectively. Artificial neural networks and logistic modeling were used. The receiver operating characteristic (ROC) curve and performance characteristics were determined to compare the diagnostic value between the two models. The training and validation sets did not differ significantly in terms of any variables. There were 19 significantly different parameters between the IgAN and non-IgAN groups. After multivariable logistic regression analysis, age, serum albumin, serum IgA, serum immunoglobulin G, estimated glomerular filtration rate, serum IgA/C3 ratio, and hematuria were found to be independently associated with the presence of IgAN. A backpropagation network model based on the above parameters was constructed and applied to the validation cohorts, revealing a sensitivity of 82.68% and a specificity of 84.78%. The area under the ROC curve for this model was higher than that for logistic regression model (0.881 vs. 0.839). The artificial neural network model based on routine markers can be a valuable noninvasive tool for predicting IgAN in screening practice.

Teasing out Artificial Intelligence in Medicine: An Ethical Critique of Artificial Intelligence and Machine Learning in Medicine

The rapid adoption and implementation of artificial intelligence in medicine creates an ontologically distinct situation from prior care models. There are both potential advantages and disadvantages with such technology in advancing the interests of patients, with resultant ontological and epistemic concerns for physicians and patients relating to the instatiation of AI as a dependent, semi- or fully-autonomous agent in the encounter. The concept of libertarian paternalism potentially exercised by AI (and those who control it) has created challenges to conventional assessments of patient and physician autonomy. The unclear legal relationship between AI and its users cannot be settled presently, an progress in AI and its implementation in patient care will necessitate an iterative discourse to preserve humanitarian concerns in future models of care. This paper proposes that physicians should neither uncritically accept nor unreasonably resist developments in AI but must actively engage and contribute to the discourse, since AI will affect their roles and the nature of their work. One's moral imaginative capacity must be engaged in the questions of beneficence, autonomy, and justice of AI and whether its integration in healthcare has the potential to augment or interfere with the ends of medical practice.

Predicting Cardiovascular Risk in Athletes: Resampling Improves Classification Performance

Cardiovascular diseases are the main cause of death worldwide. The aim of the present study is to verify the performances of a data mining methodology in the evaluation of cardiovascular risk in athletes, and whether the results may be used to support clinical decision making. Anthropometric (height and weight), demographic (age and sex) and biomedical (blood pressure and pulse rate) data of 26,002 athletes were collected in 2012 during routine sport medical examinations, which included electrocardiography at rest. Subjects were involved in competitive sport practice, for which medical clearance was needed. Outcomes were negative for the largest majority, as expected in an active population. Resampling was applied to balance positive/negative class ratio. A decision tree and logistic regression were used to classify individuals as either at risk or not. The receiver operating characteristic curve was used to assess classification performances. Data mining and resampling improved cardiovascular risk assessment in terms of increased area under the curve. The proposed methodology can be effectively applied to biomedical data in order to optimize clinical decision making, and-at the same time-minimize the amount of unnecessary examinations.

At the crossroads of oral health inequities and precision public health

OBJECTIVES

This paper reviews the precision public health literature pertaining to oral health, identifies possible threats that could inadvertently increase health inequities, and proposes potential opportunities that precision public health could utilize to reduce oral health inequities.

METHODS

The health sciences literature was reviewed and supplemented with new data to identify important issues relating to precision medicine, precision oral health, precision public health, and health equity.

RESULTS

Examples from general health and oral health were provided to illustrate salient concepts.

CONCLUSIONS

Future precision public health should utilize multifactorial, multi-level conceptual frameworks and conceptual causal models with upstream social determinants and downstream health effects, as well as a proportionate universalism perspective; and proper analytic methods, including sufficient sample sizes, appropriate statistical competitors, health disparity indices, causal modeling, and internal and external validation.

eDoctor: machine learning and the future of medicine

Machine learning (ML) is a burgeoning field of medicine with huge resources being applied to fuse computer science and statistics to medical problems. Proponents of ML extol its ability to deal with large, complex and disparate data, often found within medicine and feel that ML is the future for biomedical research, personalized medicine, computer-aided diagnosis to significantly advance global health care. However, the concepts of ML are unfamiliar to many medical professionals and there is untapped potential in the use of ML as a research tool. In this article, we provide an overview of the theory behind ML, explore the common ML algorithms used in medicine including their pitfalls and discuss the potential future of ML in medicine.

A genetic programming approach to development of clinical prediction models: A case study in symptomatic cardiovascular disease

Background

Genetic programming (GP) is an evolutionary computing methodology capable of identifying complex, non-linear patterns in large data sets. Despite the potential advantages of GP over more typical, frequentist statistical approach methods, its applications to survival analyses are rare, at best. The aim of this study was to determine the utility of GP for the automatic development of clinical prediction models.

Methods

We compared GP against the commonly used Cox regression technique in terms of the development and performance of a cardiovascular risk score using data from the SMART study, a prospective cohort study of patients with symptomatic cardiovascular disease. The composite endpoint was cardiovascular death, non-fatal stroke, and myocardial infarction. A total of 3,873 patients aged 19–82 years were enrolled in the study 1996–2006. The cohort was split 70:30 into derivation and validation sets. The derivation set was used for development of both GP and Cox regression models. These models were then used to predict the discrete hazards at t = 1, 3, and 5 years. The predictive ability of both models was evaluated in terms of their risk discrimination and calibration using the validation set.

Results

The discrimination of both models was comparable. At time points t = 1, 3, and 5 years the C-index was 0.59, 0.69, 0.64 and 0.66, 0.70, 0.70 for the GP and Cox regression models respectively. At the same time points, the calibration of both models, which was assessed using calibration plots and the generalization of the Hosmer-Lemeshow test statistic, was also comparable, but with the Cox model being better calibrated to the validation data.

Conclusion

Using empirical data, we demonstrated that a prediction model developed automatically by GP has predictive ability comparable to that of manually tuned Cox regression. The GP model was more complex, but it was developed in a fully automated way and comprised fewer covariates. Furthermore, it did not require the expertise normally needed for its derivation, thereby alleviating the knowledge elicitation bottleneck. Overall, GP demonstrated considerable potential as a method for the automated development of clinical prediction models for diagnostic and prognostic purposes.

Comparison of coronary artery disease guidelines with extracted knowledge from data mining

Introduction: Coronary artery disease (CAD) is one of the major causes of disability and death in the world. Accordingly utilizing from a national and update guideline in heart-related disease are essential. Finding interesting rules from CAD data and comparison with guidelines was the objectives of this study. Methods: In this study 1993 valid and completed records related to patients (from 2009 to 2014) who had suffered from CAD were recruited and analyzed. Total of 25 variable including a target variable (CAD) and 24 inputs or predictor variables were used for knowledge discovery. To perform comparison between extracted knowledge and well trusted guidelines, Canadian Cardiovascular Society (CCS) guideline and US National Institute of Health (NIH) guideline were selected. Results of valid datamining rules were compared with guidelines and then were ranked based on their importance. Results: The most significant factor influencing CAD was chest pain. Elderly males (age >54) have a high probability to be diagnosed with CAD. Diagnostic methods that are listed in guidelines were confirmed and ranked based on analyzing of local CAD patients data. Knowledge discovery revealed that blood test has more diagnostic value among other medical tests that were recommended in guidelines. Conclusion: Guidelines confirm the achieved results from data mining (DM) techniques and help to rank important risk factors based on national and local information. Evaluation of extracted rules determined new patterns for CAD patients.

Artificial neural networks predict the incidence of portosplenomesenteric venous thrombosis in patients with acute pancreatitis

Essentials Predicting the occurrence of portosplenomesenteric vein thrombosis (PSMVT) is difficult. We studied 72 patients with acute pancreatitis. Artificial neural networks modeling was more accurate than logistic regression in predicting PSMVT. Additional predictive factors may be incorporated into artificial neural networks.

SUMMARY

Objective To construct and validate artificial neural networks (ANNs) for predicting the occurrence of portosplenomesenteric venous thrombosis (PSMVT) and compare the predictive ability of the ANNs with that of logistic regression. Methods The ANNs and logistic regression modeling were constructed using simple clinical and laboratory data of 72 acute pancreatitis (AP) patients. The ANNs and logistic modeling were first trained on 48 randomly chosen patients and validated on the remaining 24 patients. The accuracy and the performance characteristics were compared between these two approaches by SPSS17.0 software. Results The training set and validation set did not differ on any of the 11 variables. After training, the back propagation network training error converged to 1 × 10-20 , and it retained excellent pattern recognition ability. When the ANNs model was applied to the validation set, it revealed a sensitivity of 80%, specificity of 85.7%, a positive predictive value of 77.6% and negative predictive value of 90.7%. The accuracy was 83.3%. Differences could be found between ANNs modeling and logistic regression modeling in these parameters (10.0% [95% CI, -14.3 to 34.3%], 14.3% [95% CI, -8.6 to 37.2%], 15.7% [95% CI, -9.9 to 41.3%], 11.8% [95% CI, -8.2 to 31.8%], 22.6% [95% CI, -1.9 to 47.1%], respectively). When ANNs modeling was used to identify PSMVT, the area under receiver operating characteristic curve was 0.849 (95% CI, 0.807-0.901), which demonstrated better overall properties than logistic regression modeling (AUC = 0.716) (95% CI, 0.679-0.761). Conclusions ANNs modeling was a more accurate tool than logistic regression in predicting the occurrence of PSMVT following AP. More clinical factors or biomarkers may be incorporated into ANNs modeling to improve its predictive ability.

Improving Gastric Cancer Outcome Prediction Using Single Time-Point Artificial Neural Network Models

In cancer studies, the prediction of cancer outcome based on a set of prognostic variables has been a long-standing topic of interest. Current statistical methods for survival analysis offer the possibility of modelling cancer survivability but require unrealistic assumptions about the survival time distribution or proportionality of hazard. Therefore, attention must be paid in developing nonlinear models with less restrictive assumptions. Artificial neural network (ANN) models are primarily useful in prediction when nonlinear approaches are required to sift through the plethora of available information. The applications of ANN models for prognostic and diagnostic classification in medicine have attracted a lot of interest. The applications of ANN models in modelling the survival of patients with gastric cancer have been discussed in some studies without completely considering the censored data. This study proposes an ANN model for predicting gastric cancer survivability, considering the censored data. Five separate single time-point ANN models were developed to predict the outcome of patients after 1, 2, 3, 4, and 5 years. The performance of ANN model in predicting the probabilities of death is consistently high for all time points according to the accuracy and the area under the receiver operating characteristic curve.

Dental Data Mining: Potential Pitfalls and Practical Issues

Knowledge Discovery and Data Mining (KDD) have become popular buzzwords. But what exactly is data mining? What are its strengths and limitations? Classic regression, artificial neural network (ANN), and classification and regression tree (CART) models are common KDD tools. Some recent reports ( e.g., Kattan et al., 1998 ) show that ANN and CART models can perform better than classic regression models: CART models excel at covariate interactions, while ANN models excel at nonlinear covariates. Model prediction performance is examined with the use of validation procedures and evaluating concordance, sensitivity, specificity, and likelihood ratio. To aid interpretation, various plots of predicted probabilities are utilized, such as lift charts, receiver operating characteristic curves, and cumulative captured-response plots. A dental caries study is used as an illustrative example. This paper compares the performance of logistic regression with KDD methods of CART and ANN in analyzing data from the Rochester caries study. With careful analysis, such as validation with sufficient sample size and the use of proper competitors, problems of naïve KDD analyses ( Schwarzer et al., 2000 ) can be carefully avoided.

Health-Enabling and Ambient Assistive Technologies: Past, Present, Future

BACKGROUND

During the last decades, health-enabling and ambient assistive technologies became of considerable relevance for new informatics-based forms of diagnosis, prevention, and therapy.

OBJECTIVES

To describe the state of the art of health-enabling and ambient assistive technologies in 1992 and today, and its evolution over the last 25 years as well as to project where the field is expected to be in the next 25 years. In the context of this review, we define health-enabling and ambient assistive technologies as ambiently used sensor-based information and communication technologies, aiming at contributing to a person's health and health care as well as to her or his quality of life.

METHODS

Systematic review of all original articles with research focus in all volumes of the IMIA Yearbook of Medical Informatics. Surveying authors independently on key projects and visions as well as on their lessons learned in the context of health-enabling and ambient assistive technologies and summarizing their answers. Surveying authors independently on their expectations for the future and summarizing their answers.

RESULTS

IMIA Yearbook papers containing statements on health-enabling and ambient assistive technologies appear first in 2002. These papers form a minor part of published research articles in medical informatics. However, during recent years the number of articles published has increased significantly. Key projects were identified. There was a clear progress on the use of technologies. However proof of diagnostic relevance and therapeutic efficacy remains still limited. Reforming health care processes and focussing more on patient needs are required.

CONCLUSIONS

Health-enabling and ambient assistive technologies remain an important field for future health care and for interdisciplinary research. More and more publications assume that a person's home and their interaction therein, are becoming important components in health care provision, assessment, and management.

Big Data Analytics for Prostate Radiotherapy

Radiation therapy is a first-line treatment option for localized prostate cancer and radiation-induced normal tissue damage are often the main limiting factor for modern radiotherapy regimens. Conversely, under-dosing of target volumes in an attempt to spare adjacent healthy tissues limits the likelihood of achieving local, long-term control. Thus, the ability to generate personalized data-driven risk profiles for radiotherapy outcomes would provide valuable prognostic information to help guide both clinicians and patients alike. Big data applied to radiation oncology promises to deliver better understanding of outcomes by harvesting and integrating heterogeneous data types, including patient-specific clinical parameters, treatment-related dose-volume metrics, and biological risk factors. When taken together, such variables make up the basis for a multi-dimensional space (the "RadoncSpace") in which the presented modeling techniques search in order to identify significant predictors. Herein, we review outcome modeling and big data-mining techniques for both tumor control and radiotherapy-induced normal tissue effects. We apply many of the presented modeling approaches onto a cohort of hypofractionated prostate cancer patients taking into account different data types and a large heterogeneous mix of physical and biological parameters. Cross-validation techniques are also reviewed for the refinement of the proposed framework architecture and checking individual model performance. We conclude by considering advanced modeling techniques that borrow concepts from big data analytics, such as machine learning and artificial intelligence, before discussing the potential future impact of systems radiobiology approaches.

Prediction of medial tibiofemoral compartment joint space loss progression using volumetric cartilage measurements: Data from the FNIH OA biomarkers consortium

OBJECTIVES

Investigating the association between baseline cartilage volume measurements (and initial 24th month volume loss) with medial compartment Joint-Space-Loss (JSL) progression (>0.7 mm) during 24-48th months of study.

METHODS

Case and control cohorts (Biomarkers Consortium subset from the Osteoarthritis Initiative (OAI)) were defined as participants with (n=297) and without (n=303) medial JSL progression (during 24-48th months). Cartilage volume measurements (baseline and 24th month loss) were obtained at five knee plates (medial-tibial, lateral-tibial, medial-femoral, lateral-femoral and patellar), and standardized values were analysed. Multivariate logistic regression was used with adjustment for known confounders. Artificial-Neural-Network analysis was conducted by Multi-Layer-Perceptrons (MLPs) including baseline determinants, and baseline (1) and interval changes (2) in cartilage volumes.

RESULTS

Larger baseline lateral-femoral cartilage volume was predictive of medial JSL (OR: 1.29 (1.01-1.64)). Greater initial 24th month lateral-femoral cartilage volume-loss (OR: 0.48 (0.27-0.84)) had protective effect on medial JSL during 24-48th months of study. Baseline and interval changes in lateral-femoral cartilage volume, were the most important estimators for medial JSL progression (importance values: 0.191(0.177-0.204), 0.218(0.207-0.228)) in the ANN analyses.

CONCLUSIONS

Cartilage volumes (both at baseline and their change during the initial 24 months) in the lateral femoral plate were predictive of medial JSL progression.

KEY POINTS

• Baseline lateral femoral cartilage volume is directly associated with medial JSL progression. • 24-month lateral femoral cartilage loss is inversely associated with medial JSL progression. • Lateral femoral cartilage volume is most important in association with medial JSL progression.

The clinical use of statistical permutation test methodology: a tool for identifying predictive variables of outcome

OBJECTIVES

To identify the predictive variables affecting the outcome after radical surgery for bladder cancer by a newer statistical methodology, i.e. nonparametric combination (NPC).

METHODS

A multicenter study enrolled 1,312 patients who had undergone radical cystectomy for bladder cancer in 11 Italian oncological centers from January 1982 to December 2002. A statistical analysis of their medical history and diagnostic, pathological and postoperative variables was performed using a NPC test. The patients were included in a comprehensive database with medical history and clinical and pathological data. Five-year survival was used as the dependent variable, and p values were corrected for multiplicity using a closed testing procedure. The newer nonparametric approach was used to evaluate the prognostic importance of the variables. All of the analyses were performed using routines developed in MATLAB© and the significance level was set at α = 0.05.

RESULTS

A significant prognostic predictive value (p < 0.01) for tumor clinical staging, hydronephrosis, tumor pathological staging, grading, presence of concomitant carcinoma in situ, regional lymph node involvement, corpora cavernosa invasion, microvascular invasion, lymphatic invasion and prostatic stroma involvement was found.

CONCLUSIONS

The NPC test could handle any type of variable (categorical and quantitative) and take into account the multivariate relation among variables. This newer methodology offers a significant contribution in biomedical studies with several endpoints and is recommended in presence of non-normal data and missing values, as well as solving high-dimensional data and problems relating to small sample sizes.

Prediction of survival with alternative modeling techniques using pseudo values

Background

The use of alternative modeling techniques for predicting patient survival is complicated by the fact that some alternative techniques cannot readily deal with censoring, which is essential for analyzing survival data. In the current study, we aimed to demonstrate that pseudo values enable statistically appropriate analyses of survival outcomes when used in seven alternative modeling techniques.

Methods

In this case study, we analyzed survival of 1282 Dutch patients with newly diagnosed Head and Neck Squamous Cell Carcinoma (HNSCC) with conventional Kaplan-Meier and Cox regression analysis. We subsequently calculated pseudo values to reflect the individual survival patterns. We used these pseudo values to compare recursive partitioning (RPART), neural nets (NNET), logistic regression (LR) general linear models (GLM) and three variants of support vector machines (SVM) with respect to dichotomous 60-month survival, and continuous pseudo values at 60 months or estimated survival time. We used the area under the ROC curve (AUC) and the root of the mean squared error (RMSE) to compare the performance of these models using bootstrap validation.

Results

Of a total of 1282 patients, 986 patients died during a median follow-up of 66 months (60-month survival: 52% [95% CI: 50%−55%]). The LR model had the highest optimism corrected AUC (0.791) to predict 60-month survival, followed by the SVM model with a linear kernel (AUC 0.787). The GLM model had the smallest optimism corrected RMSE when continuous pseudo values were considered for 60-month survival or the estimated survival time followed by SVM models with a linear kernel. The estimated importance of predictors varied substantially by the specific aspect of survival studied and modeling technique used.

Conclusions

The use of pseudo values makes it readily possible to apply alternative modeling techniques to survival problems, to compare their performance and to search further for promising alternative modeling techniques to analyze survival time.

Identification of biomarkers that distinguish chemical contaminants based on gene expression profiles

Background

High throughput transcriptomics profiles such as those generated using microarrays have been useful in identifying biomarkers for different classification and toxicity prediction purposes. Here, we investigated the use of microarrays to predict chemical toxicants and their possible mechanisms of action.

Results

In this study, in vitro cultures of primary rat hepatocytes were exposed to 105 chemicals and vehicle controls, representing 14 compound classes. We comprehensively compared various normalization of gene expression profiles, feature selection and classification algorithms for the classification of these 105 chemicals into14 compound classes. We found that normalization had little effect on the averaged classification accuracy. Two support vector machine (SVM) methods, LibSVM and sequential minimal optimization, had better classification performance than other methods. SVM recursive feature selection (SVM-RFE) had the highest overfitting rate when an independent dataset was used for a prediction. Therefore, we developed a new feature selection algorithm called gradient method that had a relatively high training classification as well as prediction accuracy with the lowest overfitting rate of the methods tested. Analysis of biomarkers that distinguished the 14 classes of compounds identified a group of genes principally involved in cell cycle function that were significantly downregulated by metal and inflammatory compounds, but were induced by anti-microbial, cancer related drugs, pesticides, and PXR mediators.

Conclusions

Our results indicate that using microarrays and a supervised machine learning approach to predict chemical toxicants, their potential toxicity and mechanisms of action is practical and efficient. Choosing the right feature and classification algorithms for this multiple category classification and prediction is critical.

γ -H2AX: A Novel Prognostic Marker in a Prognosis Prediction Model of Patients with Early Operable Non-Small Cell Lung Cancer

Cancer is a leading cause of death worldwide and the prognostic evaluation of cancer patients is of great importance in medical care. The use of artificial neural networks in prediction problems is well established in human medical literature. The aim of the current study was to assess the prognostic value of a series of clinical and molecular variables with the addition of γ -H2AX-a new DNA damage response marker-for the prediction of prognosis in patients with early operable non-small cell lung cancer by comparing the γ -H2AX-based artificial network prediction model with the corresponding LR one. Two prognostic models of 96 patients with 27 input variables were constructed by using the parameter-increasing method in order to compare the predictive accuracy of neural network and logistic regression models. The quality of the models was evaluated by an independent validation data set of 11 patients. Neural networks outperformed logistic regression in predicting the patient's outcome according to the experimental results. To assess the importance of the two factors p53 and γ -H2AX, models without these two variables were also constructed. JR and accuracy of these models were lower than those of the models using all input variables, suggesting that these biological markers are very important for optimal performance of the models. This study indicates that neural networks may represent a potentially more useful decision support tool than conventional statistical methods for predicting the outcome of patients with non-small cell lung cancer and that some molecular markers, such as γ -H2AX, enhance their predictive ability.

SN algorithm: analysis of temporal clinical data for mining periodic patterns and impending augury

BACKGROUND

EHR (Electronic Health Record) system has led to development of specialized form of clinical databases which enable storage of information in temporal prospective. It has been a big challenge for mining this form of clinical data considering varied temporal points. This study proposes a conjoined solution to analyze the clinical parameters akin to a disease. We have used "association rule mining algorithm" to discover association rules among clinical parameters that can be augmented with the disease. Furthermore, we have proposed a new algorithm, SN algorithm, to map clinical parameters along with a disease state at various temporal points.

RESULT

SN algorithm is based on Jacobian approach, which augurs the state of a disease 'Sn' at a given temporal point 'Tn' by mapping the derivatives with the temporal point 'T0', whose state of disease 'S0' is known. The predictive ability of the proposed algorithm is evaluated in a temporal clinical data set of brain tumor patients. We have obtained a very high prediction accuracy of ~97% for a brain tumor state 'Sn' for any temporal point 'Tn'.

CONCLUSION

The results indicate that the methodology followed may be of good value to the diagnostic procedure, especially for analyzing temporal form of clinical data.

Real-data comparison of data mining methods in prediction of diabetes in iran

OBJECTIVES

Diabetes is one of the most common non-communicable diseases in developing countries. Early screening and diagnosis play an important role in effective prevention strategies. This study compared two traditional classification methods (logistic regression and Fisher linear discriminant analysis) and four machine-learning classifiers (neural networks, support vector machines, fuzzy c-mean, and random forests) to classify persons with and without diabetes.

METHODS

The data set used in this study included 6,500 subjects from the Iranian national non-communicable diseases risk factors surveillance obtained through a cross-sectional survey. The obtained sample was based on cluster sampling of the Iran population which was conducted in 2005-2009 to assess the prevalence of major non-communicable disease risk factors. Ten risk factors that are commonly associated with diabetes were selected to compare the performance of six classifiers in terms of sensitivity, specificity, total accuracy, and area under the receiver operating characteristic (ROC) curve criteria.

RESULTS

Support vector machines showed the highest total accuracy (0.986) as well as area under the ROC (0.979). Also, this method showed high specificity (1.000) and sensitivity (0.820). All other methods produced total accuracy of more than 85%, but for all methods, the sensitivity values were very low (less than 0.350).

CONCLUSIONS

The results of this study indicate that, in terms of sensitivity, specificity, and overall classification accuracy, the support vector machine model ranks first among all the classifiers tested in the prediction of diabetes. Therefore, this approach is a promising classifier for predicting diabetes, and it should be further investigated for the prediction of other diseases.

Hip fracture risk assessment: artificial neural network outperforms conditional logistic regression in an age- and sex-matched case control study

BACKGROUND

Osteoporotic hip fractures with a significant morbidity and excess mortality among the elderly have imposed huge health and economic burdens on societies worldwide. In this age- and sex-matched case control study, we examined the risk factors of hip fractures and assessed the fracture risk by conditional logistic regression (CLR) and ensemble artificial neural network (ANN). The performances of these two classifiers were compared.

METHODS

The study population consisted of 217 pairs (149 women and 68 men) of fractures and controls with an age older than 60 years. All the participants were interviewed with the same standardized questionnaire including questions on 66 risk factors in 12 categories. Univariate CLR analysis was initially conducted to examine the unadjusted odds ratio of all potential risk factors. The significant risk factors were then tested by multivariate analyses. For fracture risk assessment, the participants were randomly divided into modeling and testing datasets for 10-fold cross validation analyses. The predicting models built by CLR and ANN in modeling datasets were applied to testing datasets for generalization study. The performances, including discrimination and calibration, were compared with non-parametric Wilcoxon tests.

RESULTS

In univariate CLR analyses, 16 variables achieved significant level, and six of them remained significant in multivariate analyses, including low T score, low BMI, low MMSE score, milk intake, walking difficulty, and significant fall at home. For discrimination, ANN outperformed CLR in both 16- and 6-variable analyses in modeling and testing datasets (p?<?0.005). For calibration, ANN outperformed CLR only in 16-variable analyses in modeling and testing datasets (p?=?0.013 and 0.047, respectively).

CONCLUSIONS

The risk factors of hip fracture are more personal than environmental. With adequate model construction, ANN may outperform CLR in both discrimination and calibration. ANN seems to have not been developed to its full potential and efforts should be made to improve its performance.

Artificial neural networks in mammography interpretation and diagnostic decision making

Screening mammography is the most effective means for early detection of breast cancer. Although general rules for discriminating malignant and benign lesions exist, radiologists are unable to perfectly detect and classify all lesions as malignant and benign, for many reasons which include, but are not limited to, overlap of features that distinguish malignancy, difficulty in estimating disease risk, and variability in recommended management. When predictive variables are numerous and interact, ad hoc decision making strategies based on experience and memory may lead to systematic errors and variability in practice. The integration of computer models to help radiologists increase the accuracy of mammography examinations in diagnostic decision making has gained increasing attention in the last two decades. In this study, we provide an overview of one of the most commonly used models, artificial neural networks (ANNs), in mammography interpretation and diagnostic decision making and discuss important features in mammography interpretation. We conclude by discussing several common limitations of existing research on ANN-based detection and diagnostic models and provide possible future research directions.

ANN as a prognostic tool after treatment of non-seminoma testicular cancer

Testicular cancer is rare but is the most common cancer in males between 15 and 34 years of age. Two principal types of testicular cancer are distinguished: seminomas and non-seminomas. If detected early, the overall cure rate for testicular cancer exceeds 90%. In this study, artificial neural network (ANN) analysis as a prognostic tool was demonstrated regard to five year recurrence after the non-seminoma treatment. Data from 202 patients treated for non-seminoma were available for evaluation and comparison. A total of 32 variables were analysed using the ANN. The ANN approach, as an advanced multivariate data processing method, was demon-strated to provide objective prognostic data. Some of these prognostic factors are consistent or even imperceptible with previously evaluated by other statistical methods.

Avoiding pitfalls in applying prediction models, as illustrated by the example of prostate cancer diagnosis

BACKGROUND

The use of different mathematical models to support medical decisions is accompanied by increasing uncertainties when they are applied in practice. Using prostate cancer (PCa) risk models as an example, we recommend requirements for model development and draw attention to possible pitfalls so as to avoid the uncritical use of these models.

CONTENT

We conducted MEDLINE searches for applications of multivariate models supporting the prediction of PCa risk. We critically reviewed the methodological aspects of model development and the biological and analytical variability of the parameters used for model development. In addition, we reviewed the role of prostate biopsy as the gold standard for confirming diagnoses. In addition, we analyzed different methods of model evaluation with respect to their application to different populations. When using models in clinical practice, one must validate the results with a population from the application field. Typical model characteristics (such as discrimination performance and calibration) and methods for assessing the risk of a decision should be used when evaluating a model's output. The choice of a model should be based on these results and on the practicality of its use.

SUMMARY

To avoid possible errors in applying prediction models (the risk of PCa, for example) requires examining the possible pitfalls of the underlying mathematical models in the context of the individual case. The main tools for this purpose are discrimination, calibration, and decision curve analysis.

Data mining methods in the prediction of Dementia: A real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests

Background

Dementia and cognitive impairment associated with aging are a major medical and social concern. Neuropsychological testing is a key element in the diagnostic procedures of Mild Cognitive Impairment (MCI), but has presently a limited value in the prediction of progression to dementia. We advance the hypothesis that newer statistical classification methods derived from data mining and machine learning methods like Neural Networks, Support Vector Machines and Random Forests can improve accuracy, sensitivity and specificity of predictions obtained from neuropsychological testing. Seven non parametric classifiers derived from data mining methods (Multilayer Perceptrons Neural Networks, Radial Basis Function Neural Networks, Support Vector Machines, CART, CHAID and QUEST Classification Trees and Random Forests) were compared to three traditional classifiers (Linear Discriminant Analysis, Quadratic Discriminant Analysis and Logistic Regression) in terms of overall classification accuracy, specificity, sensitivity, Area under the ROC curve and Press'Q. Model predictors were 10 neuropsychological tests currently used in the diagnosis of dementia. Statistical distributions of classification parameters obtained from a 5-fold cross-validation were compared using the Friedman's nonparametric test.

Results

Press' Q test showed that all classifiers performed better than chance alone (p < 0.05). Support Vector Machines showed the larger overall classification accuracy (Median (Me) = 0.76) an area under the ROC (Me = 0.90). However this method showed high specificity (Me = 1.0) but low sensitivity (Me = 0.3). Random Forest ranked second in overall accuracy (Me = 0.73) with high area under the ROC (Me = 0.73) specificity (Me = 0.73) and sensitivity (Me = 0.64). Linear Discriminant Analysis also showed acceptable overall accuracy (Me = 0.66), with acceptable area under the ROC (Me = 0.72) specificity (Me = 0.66) and sensitivity (Me = 0.64). The remaining classifiers showed overall classification accuracy above a median value of 0.63, but for most sensitivity was around or even lower than a median value of 0.5.

Conclusions

When taking into account sensitivity, specificity and overall classification accuracy Random Forests and Linear Discriminant analysis rank first among all the classifiers tested in prediction of dementia using several neuropsychological tests. These methods may be used to improve accuracy, sensitivity and specificity of Dementia predictions from neuropsychological testing.

Artificial neural network analysis of circulating tumor cells in metastatic breast cancer patients

A cut-off of 5 circulating tumor cells (CTCs) per 7.5 ml of blood in metastatic breast cancer (MBC) patients is highly predictive of outcome. We analyzed the relationship between CTCs as a continuous variable and overall survival in immunohistochemically defined primary tumor molecular subtypes using an artificial neural network (ANN) prognostic tool to determine the shape of the relationship between risk of death and CTC count and to predict individual survival. We analyzed a training dataset of 311 of 517 (60%) consecutive MBC patients who had been treated at MD Anderson Cancer Center from September 2004 to 2009 and who had undergone pre-therapy CTC counts (CellSearch(®)). Age; estrogen, progesterone receptor, and HER2 status; visceral metastasis; metastatic disease sites; therapy type and line; and CTCs as a continuous value were evaluated using ANN. A model with parameter estimates obtained from the training data was tested in a validation set of the remaining 206 (40%) patients. The model estimates were accurate, with good discrimination and calibration. Risk of death, as estimated by ANN, linearly increased with increasing CTC count in all molecular tumor subtypes but was higher in ER+ and triple-negative MBC than in HER2+. The probabilities of survival for the four subtypes with 0 CTC were as follows: ER+/HER2- 0.947, ER+/HER2+ 0.959, ER-/HER2+ 0.902, and ER-/HER2- 0.875. For patients with 200 CTCs, they were ER+/HER2- 0.439, ER+/HER2+ 0.621, ER-/HER2+ 0.307, ER-/HER2- 0.130. In this large study, ANN revealed a linear increase of risk of death in MBC patients with increasing CTC counts in all tumor subtypes. CTCs' prognostic effect was less evident in HER2+ MBC patients treated with targeted therapy. This study may support the concept that the number of CTCs, along with the biologic characteristics, needs to be carefully taken into account in future analysis.

Data mining in healthcare and biomedicine: a survey of the literature

As a new concept that emerged in the middle of 1990's, data mining can help researchers gain both novel and deep insights and can facilitate unprecedented understanding of large biomedical datasets. Data mining can uncover new biomedical and healthcare knowledge for clinical and administrative decision making as well as generate scientific hypotheses from large experimental data, clinical databases, and/or biomedical literature. This review first introduces data mining in general (e.g., the background, definition, and process of data mining), discusses the major differences between statistics and data mining and then speaks to the uniqueness of data mining in the biomedical and healthcare fields. A brief summarization of various data mining algorithms used for classification, clustering, and association as well as their respective advantages and drawbacks is also presented. Suggested guidelines on how to use data mining algorithms in each area of classification, clustering, and association are offered along with three examples of how data mining has been used in the healthcare industry. Given the successful application of data mining by health related organizations that has helped to predict health insurance fraud and under-diagnosed patients, and identify and classify at-risk people in terms of health with the goal of reducing healthcare cost, we introduce how data mining technologies (in each area of classification, clustering, and association) have been used for a multitude of purposes, including research in the biomedical and healthcare fields. A discussion of the technologies available to enable the prediction of healthcare costs (including length of hospital stay), disease diagnosis and prognosis, and the discovery of hidden biomedical and healthcare patterns from related databases is offered along with a discussion of the use of data mining to discover such relationships as those between health conditions and a disease, relationships among diseases, and relationships among drugs. The article concludes with a discussion of the problems that hamper the clinical use of data mining by health professionals.

The development of composite circulating biomarker models for use in anticancer drug clinical development

The development of informative composite circulating biomarkers predicting cancer presence or therapy response is clinically attractive but optimal approaches to modeling are as yet unclear. This study investigated multidimensional relationships within an example panel of serum insulin-like growth factor (IGF) peptides using logistic regression (LR), fractional polynomial (FP), regression, artificial neural networks (ANNs) and support vector machines (SVMs) to derive predictive models for colorectal cancer (CRC). Two phase 2 biomarker validation analyses were performed: controls were ambulant adults (n = 722); cases were: (i) CRC patients (n = 100) and (ii) patients with acromegaly (n = 52), the latter as "positive" discriminators. Serum IGF-I, IGF-II, IGF binding protein (IGFBP)-2 and -3 were measured. Discriminatory characteristics were compared within and between models. For the LR, FP and ANN models, and to a lesser extent SVMs, the addition of covariates at several steps improved discrimination characteristics. The optimum biomarker combination discriminating CRC vs. controls was achieved using ANN models [sensitivity, 94%; specificity, 90%; accuracy, 0.975 (95% CIs: 0.948 1.000)]. ANN modeling significantly outperformed LR, FP and SVM in terms of discrimination (p < 0.0001) and calibration. The acromegaly analysis demonstrated expected high performance characteristics in the ANN model [accuracy, 0.993 (95% CIs: 0.977, 1.000)]. Curved decision surfaces generated from the ANNs revealed the potential clinical utility. This example demonstrated improved discriminatory characteristics within the composite biomarker ANN model and a final model that outperformed the three other models. This modeling approach forms the basis to evaluate composite biomarkers as pharmacological and predictive biomarkers in future clinical trials.

Contribution of artificial intelligence to the knowledge of prognostic factors in Hodgkin's lymphoma

Hodgkin's lymphoma is one of the most curable malignancies and most patients achieve a lasting complete remission. In this study, artificial neural network (ANN) analysis was shown to provide significant factors with regard to 5-year recurrence after lymphoma treatment. Data from 114 patients treated for Hodgkin's disease were available for evaluation and comparison. A total of 31 variables were subjected to ANN analysis. The ANN approach as an advanced multivariate data processing method was shown to provide objective prognostic data. Some of these prognostic factors are consistent or even identical to the factors evaluated earlier by other statistical methods.

Prediction of outcome in cancer patients with febrile neutropenia: a prospective validation of the Multinational Association for Supportive Care in Cancer risk index in a Chinese population and comparison with the Talcott model and artificial neural network

PURPOSE

We aimed to validate the Multinational Association for Supportive Care in Cancer (MASCC) risk index, and compare it with the Talcott model and artificial neural network (ANN) in predicting the outcome of febrile neutropenia in a Chinese population.

METHODS

We prospectively enrolled adult cancer patients who developed febrile neutropenia after chemotherapy and risk classified them according to MASCC score and Talcott model. ANN models were constructed and temporally validated in prospectively collected cohorts.

RESULTS

From October 2005 to February 2008, 227 consecutive patients were enrolled. Serious medical complications occurred in 22% of patients and 4% died. The positive predictive value of low risk prediction was 86% (95% CI = 81-90%) for MASCC score ≥ 21, 84% (79-89%) for Talcott model, and 85% (78-93%) for the best ANN model. The sensitivity, specificity, negative predictive value, and misclassification rate were 81%, 60%, 52%, and 24%, respectively, for MASCC score ≥ 21; and 50%, 72%, 33%, and 44%, respectively, for Talcott model; and 84%, 60%, 58%, and 22%, respectively, for ANN model. The area under the receiver-operating characteristic curve was 0.808 (95% CI = 0.717-0.899) for MASCC, 0.573 (0.455-0.691) for Talcott, and 0.737 (0.633-0.841) for ANN model. In the low risk group identified by MASCC score ≥ 21 (70% of all patients), 12.5% developed complications and 1.9% died, compared with 43.3%, and 9.0%, respectively, in the high risk group (p < 0.0001).

CONCLUSIONS

The MASCC risk index is prospectively validated in a Chinese population. It demonstrates a better overall performance than the Talcott model and is equivalent to ANN model.

Prediction of clinical conditions after coronary bypass surgery using dynamic data analysis

This work studies the impact of using dynamic information as features in a machine learning algorithm for the prediction task of classifying critically ill patients in two classes according to the time they need to reach a stable state after coronary bypass surgery: less or more than 9 h. On the basis of five physiological variables (heart rate, systolic arterial blood pressure, systolic pulmonary pressure, blood temperature and oxygen saturation), different dynamic features were extracted, namely the means and standard deviations at different moments in time, coefficients of multivariate autoregressive models and cepstral coefficients. These sets of features served subsequently as inputs for a Gaussian process and the prediction results were compared with the case where only admission data was used for the classification. The dynamic features, especially the cepstral coefficients (aROC: 0.749, Brier score: 0.206), resulted in higher performances when compared to static admission data (aROC: 0.547, Brier score: 0.247). The differences in performance are shown to be significant. In all cases, the Gaussian process classifier outperformed to logistic regression.