Comparing models for quantitative risk assessment: an application to the European Registry of foreign body injuries in children

A causal Bayesian network approach for consumer product safety and risk assessment

INTRODUCTION

Product risk assessment is the overall process of determining whether a product is judged safe for consumers to use. Among several methods for product risk assessment, RAPEX is the primary one used by regulators in the UK and EU. Despite its widespread use we identify several limitations of RAPEX, including a limited approach to handling uncertainty, especially in the absence of data, and the inability to incorporate causal explanations for using and interpreting the data.

METHOD

We develop a Bayesian Network (BN) model to provide an improved systematic method for product risk assessment that resolves the identified limitations with RAPEX. BNs are a rigorous, normative method for modelling uncertainty and causality which are already used for risk assessment in domains such as medicine and finance, as well as critical systems generally.

RESULTS

We use the BN approach to demonstrate risk assessments for products where relevant test and product instance data are and are not available. Whereas RAPEX can only produce results given relevant data, the BN approach produce results for products with and with no relevant data - replicating RAPEX in the former and providing deeper insights in both cases.

CONCLUSION

The BN approach is powerful and flexible for systematic product risk assessment. While it can complement more traditional methods like RAPEX, it is able to provide quantified, auditable assessments in situations where such methods cannot because of lack of data. Practical Applications: Safety regulators, manufacturers, and risk professionals can use the BN approach for all types of consumer product risk assessment, including for novel products or products with little or no historical data. They can also use it to validate the results of existing methods when data becomes available. It informs risk management decisions and helps understand the effect of those decisions on the consumer risk perception.

Analysis Factors That Influence Escalator-Related Injuries in Metro Stations Based on Bayesian Networks: A Case Study in China

Escalator-related injuries have become an important issue in daily metro operation. To reduce the probability and severity of escalator-related injuries, this study conducted a probability and severity analysis of escalator-related injuries by using a Bayesian network to identify the risk factors that affect the escalator safety in metro stations. The Bayesian network structure was constructed based on expert knowledge and Dempster–Shafer evidence theory, and further modified based on conditional-independence test. Then, 950 escalator-related injuries were used to estimate the posterior probabilities of the Bayesian network with expectation–maximization (EM) algorithm. The results of probability analysis indicate that the most influential factor in four passenger behaviors is failing to stand firm (p = 0.48), followed by carrying out other tasks (p = 0.32), not holding the handrail (p = 0.23), and another passenger’s movement (p = 0.20). Women (p = 0.64) and elderly people (aged 66 years and above, p = 0.48) are more likely to be involved in escalator-related injuries. Riding an escalator with company (p = 0.63) has a relatively high likelihood of resulting in escalator-related injuries. The results from the severity analysis show that head and neck injuries seem to be more serious and are more likely to require an ambulance for treatment. Passengers who suffer from entrapment injury tend to claim for compensation. Severe injuries, as expected, significantly increase the probability of a claim for compensation. These findings could provide valuable references for metro operation corporations to understand the characteristics of escalator-related injuries and develop effective injury prevention measures.

Reactive oxygen species metabolism-based prediction model and drug for patients with recurrent glioblastoma

Background: Tumor recurrence is the main cause of poor prognosis of GBM. Finding the characteristics of recurrent GBM that provide early warning of tumor recurrence can provide guidance for the clinical treatment of recurrent GBM.

Results: Reactive oxygen species (ROS) biosynthetic processes was significantly elevated in recurrent GBM. The recurrent risk score based on the ROS biosynthetic process was closely related to tumor purity and tumor immune functions. The quantitative risk assessment system could be used to predict the recurrence time of GBM. Gallic acid, a compound with high anti-oxidation activity and low cytotoxicity, was screened as a potential chemotherapy sensitizer for recurrent GBM.

Conclusion: The quantitative risk assessment system based on ROS biosynthetic process could be used for early warning of GBM recurrence. Combination of low-dose gallic acid and temozolomide could improve therapeutic outcomes in recurrent GBM.

Methods: A total of 663 primary and recurrent GBM samples with clinical and microarray data were included in this study. GSVA, LASSO-COX, and Kaplan-Meier survive curve were performed to construct and verify a quantitative risk assessment system for GBM recurrence prediction. An antioxidant capacity test and cell viability test were used to discover potential drugs for recurrent GBM.

The Role of Genetic Factors in Characterizing Extra-Intestinal Manifestations in Crohn's Disease Patients: Are Bayesian Machine Learning Methods Improving Outcome Predictions?

(1) Background: The high heterogeneity of inflammatory bowel disease (IBD) makes the study of this condition challenging. In subjects affected by Crohn’s disease (CD), extra-intestinal manifestations (EIMs) have a remarkable potential impact on health status. Increasing numbers of patient characteristics and the small size of analyzed samples make EIMs prediction very difficult. Under such constraints, Bayesian machine learning techniques (BMLTs) have been proposed as a robust alternative to classical models for outcome prediction. This study aims to determine whether BMLT could improve EIM prediction and statistical support for the decision-making process of clinicians. (2) Methods: Three of the most popular BMLTs were employed in this study: Naϊve Bayes (NB), Bayesian Network (BN) and Bayesian Additive Regression Trees (BART). They were applied to a retrospective observational Italian study of IBD genetics. (3) Results: The performance of the model is strongly affected by the features of the dataset, and BMLTs poorly classify EIM appearance. (4) Conclusions: This study shows that BMLTs perform worse than expected in classifying the presence of EIMs compared to classical statistical tools in a context where mixed genetic and clinical data are available but relevant data are also missing, as often occurs in clinical practice.

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models

OBJECTIVES

The objective of this study was to compare performance of logistic regression (LR) with machine learning (ML) for clinical prediction modeling in the literature.

STUDY DESIGN AND SETTING

We conducted a Medline literature search (1/2016 to 8/2017) and extracted comparisons between LR and ML models for binary outcomes.

RESULTS

We included 71 of 927 studies. The median sample size was 1,250 (range 72-3,994,872), with 19 predictors considered (range 5-563) and eight events per predictor (range 0.3-6,697). The most common ML methods were classification trees, random forests, artificial neural networks, and support vector machines. In 48 (68%) studies, we observed potential bias in the validation procedures. Sixty-four (90%) studies used the area under the receiver operating characteristic curve (AUC) to assess discrimination. Calibration was not addressed in 56 (79%) studies. We identified 282 comparisons between an LR and ML model (AUC range, 0.52-0.99). For 145 comparisons at low risk of bias, the difference in logit(AUC) between LR and ML was 0.00 (95% confidence interval, -0.18 to 0.18). For 137 comparisons at high risk of bias, logit(AUC) was 0.34 (0.20-0.47) higher for ML.

CONCLUSION

We found no evidence of superior performance of ML over LR. Improvements in methodology and reporting are needed for studies that compare modeling algorithms.

An Innovative Model to Predict Pediatric Emergency Department Return Visits

OBJECTIVES

Return visit (RV) to the emergency department (ED) is considered a benchmarking clinical indicator for health care quality. The purpose of this study was to develop a predictive model for early readmission risk in pediatric EDs comparing the performances of 2 learning machine algorithms.

METHODS

A retrospective study based on all children younger than 15 years spontaneously returning within 120 hours after discharge was conducted in an Italian university children's hospital between October 2012 and April 2013. Two predictive models, artificial neural network (ANN) and classification tree (CT), were used. Accuracy, specificity, and sensitivity were assessed.

RESULTS

A total of 28,341 patient records were evaluated. Among them, 626 patients returned to the ED within 120 hours after their initial visit. Comparing ANN and CT, our analysis has shown that CT is the best model to predict RVs. The CT model showed an overall accuracy of 81%, slightly lower than the one achieved by the ANN (91.3%), but CT outperformed ANN with regard to sensitivity (79.8% vs 6.9%, respectively). The specificity was similar for the 2 models (CT, 97% vs ANN, 98.3%). In addition, the time of arrival and discharge along with the priority code assigned in triage, age, and diagnosis play a pivotal role to identify patients at high risk of RVs.

CONCLUSIONS

These models provide a promising predictive tool for supporting the ED staff in preventing unnecessary RVs.

Integration of PKPD relationships into benefit-risk analysis

AIM

Despite the continuous endeavour to achieve high standards in medical care through effectiveness measures, a quantitative framework for the assessment of the benefit-risk balance of new medicines is lacking prior to regulatory approval. The aim of this short review is to summarise the approaches currently available for benefit-risk assessment. In addition, we propose the use of pharmacokinetic-pharmacodynamic (PKPD) modelling as the pharmacological basis for evidence synthesis and evaluation of novel therapeutic agents.

METHODS

A comprehensive literature search has been performed using MESH terms in PubMed, in which articles describing benefit-risk assessment and modelling and simulation were identified. In parallel, a critical review of multi-criteria decision analysis (MCDA) is presented as a tool for characterising a drug's safety and efficacy profile.

RESULTS

A definition of benefits and risks has been proposed by the European Medicines Agency (EMA), in which qualitative and quantitative elements are included. However, in spite of the value of MCDA as a quantitative method, decisions about benefit-risk balance continue to rely on subjective expert opinion. By contrast, a model-informed approach offers the opportunity for a more comprehensive evaluation of benefit-risk balance before extensive evidence is generated in clinical practice.

CONCLUSIONS

Benefit-risk balance should be an integral part of the risk management plan and as such considered before marketing authorisation. Modelling and simulation can be incorporated into MCDA to support the evidence synthesis as well evidence generation taking into account the underlying correlations between favourable and unfavourable effects. In addition, it represents a valuable tool for the optimization of protocol design in effectiveness trials.