Avoiding pitfalls in applying prediction models, as illustrated by the example of prostate cancer diagnosis

The Experimentalist's Guide to Machine Learning for Small Molecule Design

Initially part of the field of artificial intelligence, machine learning (ML) has become a booming research area since branching out into its own field in the 1990s. After three decades of refinement, ML algorithms have accelerated scientific developments across a variety of research topics. The field of small molecule design is no exception, and an increasing number of researchers are applying ML techniques in their pursuit of discovering, generating, and optimizing small molecule compounds. The goal of this review is to provide simple, yet descriptive, explanations of some of the most commonly utilized ML algorithms in the field of small molecule design along with those that are highly applicable to an experimentally focused audience. The algorithms discussed here span across three ML paradigms: supervised learning, unsupervised learning, and ensemble methods. Examples from the published literature will be provided for each algorithm. Some common pitfalls of applying ML to biological and chemical data sets will also be explained, alongside a brief summary of a few more advanced paradigms, including reinforcement learning and semi-supervised learning.

The Clinical Relevance of Methods for Handling Inconclusive Medical Test Results: Quantification of Uncertainty in Medical Decision-Making and Screening

BACKGROUND

although the existence of inconclusive medical test results or bio-markers is widely recognized, there are indications that this inherent diagnostic uncertainty is sometimes ignored. This paper discusses three methods for defining and determining inconclusive medical test results, which use different definitions and differ in clinical relevance.

METHODS

the TG-ROC (two graphs receiver operating characteristics) method is the easiest to use, while the grey zone method and the uncertain interval method require more extensive calculations.

RESULTS

this paper discusses the technical details of the methods, as well as advantages and disadvantages for their clinical use. TG-ROC and the grey zone method can help in the acquisition of high rates of diagnostic certainty, but can exclude large groups. The uncertain interval method can prevent decisions that are the most uncertain, invalid and unreliable, while excluding smaller groups.

CONCLUSIONS

the identification of uncertain test scores is relevant, because these scores indicate the need to obtain better information or to await further developments. The methods presented help to determine inconclusive test scores and can help to reduce erroneous decisions. However, further research and development is desirable.

Prediction of Low Birth Weight Delivery by Maternal Status and Its Validation: Decision Curve Analysis

Background:

In this study, we evaluated assessed elements connected with low birth weight (LBW) and used decision curve analysis (DCA) to define a scale to anticipate the probability of having a LBW newborn child.

Methods:

This hospital-based case–control study was led in Arak Hospital in Iran. The study included 470 mothers with LBW neonate and 470 mothers with natural neonates. Information were gathered by meeting moms utilizing preplanned organized questionnaire and from hospital records. The estimated probabilities of detecting LBW were calculated using the logistic regression and DCA to quantify the clinical consequences and its validation.

Results:

Factors significantly associated with LBW were premature membrane rupture (odds ratio [OR] = 3.18 [1.882–5.384]), former LBW infants (OR = 2.99 [1.510–5.932]), premature pain (OR = 2.70 [1.659–4.415]), hypertension in pregnancy (OR = 2.39 [1.429–4.019]), last trimester of pregnancy bleeding (OR = 2.58 [1.018–6.583]), mother age >30 (OR = 2.17 [1.350–3.498]). However, with DCA, the prediction model made on these 15 variables has a net benefit (NB) of 0.3110 is best predictive with the highest NB. NB has simple clinical interpretation and utilizing the model is what might as well be called a procedure that distinguished what might as well be called 31.1 LBW per 100 cases with no superfluous recognize.

Conclusions:

It is conceivable to foresee LBW utilizing a prediction model show in light of noteworthy hazard components connected with LBW. The majority of the hazard elements for LBW are preventable, and moms can be alluded amid early pregnancy to a middle which is furnished with facilities for administration of high hazard pregnancy and LBW infant.

Decision support systems for personalized and participative radiation oncology

A paradigm shift from current population based medicine to personalized and participative medicine is underway. This transition is being supported by the development of clinical decision support systems based on prediction models of treatment outcome. In radiation oncology, these models 'learn' using advanced and innovative information technologies (ideally in a distributed fashion - please watch the animation: http://youtu.be/ZDJFOxpwqEA) from all available/appropriate medical data (clinical, treatment, imaging, biological/genetic, etc.) to achieve the highest possible accuracy with respect to prediction of tumor response and normal tissue toxicity. In this position paper, we deliver an overview of the factors that are associated with outcome in radiation oncology and discuss the methodology behind the development of accurate prediction models, which is a multi-faceted process. Subsequent to initial development/validation and clinical introduction, decision support systems should be constantly re-evaluated (through quality assurance procedures) in different patient datasets in order to refine and re-optimize the models, ensuring the continuous utility of the models. In the reasonably near future, decision support systems will be fully integrated within the clinic, with data and knowledge being shared in a standardized, dynamic, and potentially global manner enabling truly personalized and participative medicine.

Interval of Uncertainty: An Alternative Approach for the Determination of Decision Thresholds, with an Illustrative Application for the Prediction of Prostate Cancer

Often, for medical decisions based on test scores, a single decision threshold is determined and the test results are dichotomized into positive and negative diagnoses. It is therefore important to identify the decision threshold with the least number of misclassifications. The proposed method uses trichotomization: it defines an Uncertain Interval around the point of intersection between the two distributions of individuals with and without the targeted disease. In this Uncertain Interval the diagnoses are intermixed and the numbers of correct and incorrect diagnoses are (almost) equal. This Uncertain Interval is considered to be a range of test scores that is inconclusive and does not warrant a decision. It is expected that defining such an interval with some precision, prevents a relatively large number of false decisions, and therefore results in an increased accuracy or correct classifications rate (CCR) for the test scores outside this Uncertain Interval. Clinical data and simulation results confirm this. The results show that the CCR is systematically higher outside the Uncertain Interval when compared to the CCR of the decision threshold based on the maximized Youden index. For strong tests with a very small overlap between the two distributions, it can be difficult to determine an Uncertain Interval. In simulations, the comparison with an existing method for test-score trichotomization, the Two-graph Receiver Operating Characteristic (TG-ROC), showed smaller differences between the two distributions for the Uncertain Interval than for TG-ROC's Intermediate Range and consequently a more improved CCR outside the Uncertain Interval. The main conclusion is that the Uncertain Interval method offers two advantages: 1. Identification of patients for whom the test results are inconclusive; 2. A higher estimated rate of correct decisions for the remaining patients.

Risk factors and biomarkers of age-related macular degeneration

A biomarker can be a substance or structure measured in body parts, fluids or products that can affect or predict disease incidence. As age-related macular degeneration (AMD) is the leading cause of blindness in the developed world, much research and effort has been invested in the identification of different biomarkers to predict disease incidence, identify at risk individuals, elucidate causative pathophysiological etiologies, guide screening, monitoring and treatment parameters, and predict disease outcomes. To date, a host of genetic, environmental, proteomic, and cellular targets have been identified as both risk factors and potential biomarkers for AMD. Despite this, their use has been confined to research settings and has not yet crossed into the clinical arena. A greater understanding of these factors and their use as potential biomarkers for AMD can guide future research and clinical practice. This article will discuss known risk factors and novel, potential biomarkers of AMD in addition to their application in both academic and clinical settings.

A Nomogram for Predicting the Likelihood of Obstructive Sleep Apnea to Reduce the Unnecessary Polysomnography Examinations

Background:

The currently available polysomnography (PSG) equipments and operating personnel are facing increasing pressure, such situation may result in the problem that a large number of obstructive sleep apnea (OSA) patients cannot receive timely diagnosis and treatment, we sought to develop a nomogram quantifying the risk of OSA for a better decision of using PSG, based on the clinical syndromes and the demographic and anthropometric characteristics.

Methods:

The nomogram was constructed through an ordinal logistic regression procedure. Predictive accuracy and performance characteristics were assessed with the area under the curve (AUC) of the receiver operating characteristics and calibration plots, respectively. Decision curve analyses were applied to assess the net benefit of the nomogram.

Results:

Among the 401 patients, 73 (18.2%) were diagnosed and grouped as the none OSA (apnea-hypopnea index [AHI] <5), 67 (16.7%) the mild OSA (5 ≤ AHI < 15), 82 (20.4%) the moderate OSA (15 ≤ AHI < 30), and 179 (44.6%) the severe OSA (AHI ≥ 30). The multivariable analysis suggested the significant factors were duration of disease, smoking status, difficulty of falling asleep, lack of energy, and waist circumference. A nomogram was created for the prediction of OSA using these clinical parameters and was internally validated using bootstrapping method. The discrimination accuracies of the nomogram for any OSA, moderate-severe OSA, and severe OSA were 83.8%, 79.9%, and 80.5%, respectively, which indicated good calibration. Decision curve analysis showed that using nomogram could reduce the unnecessary polysomnography (PSG) by 10% without increasing the false negatives.

Conclusions:

The established clinical nomogram provides high accuracy in predicting the individual risk of OSA. This tool may help physicians better make decisions on PSG arrangement for the patients referred to sleep centers.

Artificial neural networks and prostate cancer--tools for diagnosis and management

Artificial neural networks (ANNs) are mathematical models that are based on biological neural networks and are composed of interconnected groups of artificial neurons. ANNs are used to map and predict outcomes in complex relationships between given 'inputs' and sought-after 'outputs' and can also be used find patterns in datasets. In medicine, ANN applications have been used in cancer diagnosis, staging and recurrence prediction since the mid-1990s, when an enormous effort was initiated, especially in prostate cancer detection. Modern ANNs can incorporate new biomarkers and imaging data to improve their predictive power and can offer a number of advantages as clinical decision making tools, such as easy handling of distribution-free input parameters. Most importantly, ANNs consider nonlinear relationships among input data that cannot always be recognized by conventional analyses. In the future, complex medical diagnostic and treatment decisions will be increasingly based on ANNs and other multivariate models.

Predicting outcomes in radiation oncology--multifactorial decision support systems

With the emergence of individualized medicine and the increasing amount and complexity of available medical data, a growing need exists for the development of clinical decision-support systems based on prediction models of treatment outcome. In radiation oncology, these models combine both predictive and prognostic data factors from clinical, imaging, molecular and other sources to achieve the highest accuracy to predict tumour response and follow-up event rates. In this Review, we provide an overview of the factors that are correlated with outcome-including survival, recurrence patterns and toxicity-in radiation oncology and discuss the methodology behind the development of prediction models, which is a multistage process. Even after initial development and clinical introduction, a truly useful predictive model will be continuously re-evaluated on different patient datasets from different regions to ensure its population-specific strength. In the future, validated decision-support systems will be fully integrated in the clinic, with data and knowledge being shared in a standardized, instant and global manner.