Human and machine learning in non-Markovian decision making

Personalized next-best action recommendation with multi-party interaction learning for automated decision-making

Automated next-best action recommendation for each customer in a sequential, dynamic and interactive context has been widely needed in natural, social and business decision-making. Personalized next-best action recommendation must involve past, current and future customer demographics and circumstances (states) and behaviors, long-range sequential interactions between customers and decision-makers, multi-sequence interactions between states, behaviors and actions, and their reactions to their counterpart’s actions. No existing modeling theories and tools, including Markovian decision processes, user and behavior modeling, deep sequential modeling, and personalized sequential recommendation, can quantify such complex decision-making on a personal level. We take a data-driven approach to learn the next-best actions for personalized decision-making by a reinforced coupled recurrent neural network (CRN). CRN represents multiple coupled dynamic sequences of a customer’s historical and current states, responses to decision-makers’ actions, decision rewards to actions, and learns long-term multi-sequence interactions between parties (customer and decision-maker). Next-best actions are then recommended on each customer at a time point to change their state for an optimal decision-making objective. Our study demonstrates the potential of personalized deep learning of multi-sequence interactions and automated dynamic intervention for personalized decision-making in complex systems.

Deep reinforcement learning in medical imaging: A literature review

Deep reinforcement learning (DRL) augments the reinforcement learning framework, which learns a sequence of actions that maximizes the expected reward, with the representative power of deep neural networks. Recent works have demonstrated the great potential of DRL in medicine and healthcare. This paper presents a literature review of DRL in medical imaging. We start with a comprehensive tutorial of DRL, including the latest model-free and model-based algorithms. We then cover existing DRL applications for medical imaging, which are roughly divided into three main categories: (i) parametric medical image analysis tasks including landmark detection, object/lesion detection, registration, and view plane localization; (ii) solving optimization tasks including hyperparameter tuning, selecting augmentation strategies, and neural architecture search; and (iii) miscellaneous applications including surgical gesture segmentation, personalized mobile health intervention, and computational model personalization. The paper concludes with discussions of future perspectives.

Dental Caries Diagnosis and Detection Using Neural Networks: A Systematic Review

Dental caries is the most prevalent dental disease worldwide, and neural networks and artificial intelligence are increasingly being used in the field of dentistry. This systematic review aims to identify the state of the art of neural networks in caries detection and diagnosis. A search was conducted in PubMed, Institute of Electrical and Electronics Engineers (IEEE) Xplore, and ScienceDirect. Data extraction was performed independently by two reviewers. The quality of the selected studies was assessed using the Cochrane Handbook tool. Thirteen studies were included. Most of the included studies employed periapical, near-infrared light transillumination, and bitewing radiography. The image databases ranged from 87 to 3000 images, with a mean of 669 images. Seven of the included studies labeled the dental caries in each image by experienced dentists. Not all of the studies detailed how caries was defined, and not all detailed the type of carious lesion detected. Each study included in this review used a different neural network and different outcome metrics. All this variability complicates the conclusions that can be made about the reliability or not of a neural network to detect and diagnose caries. A comparison between neural network and dentist results is also necessary.

A brief introduction to concepts and applications of artificial intelligence in dental imaging

This report aims to summarize the fundamental concepts of Artificial Intelligence (AI), and to provide a non-exhaustive overview of AI applications in dental imaging, comprising diagnostics, forensics, image processing and image reconstruction. AI has arguably become the hottest topic in radiology in recent years owing to the increased computational power available to researchers, the continuing collection of digital data, as well as the development of highly efficient algorithms for machine learning and deep learning. It is now feasible to develop highly robust AI applications that make use of the vast amount of data available to us, and that keep learning and improving over time.

Using Stochastic Spiking Neural Networks on SpiNNaker to Solve Constraint Satisfaction Problems

Constraint satisfaction problems (CSP) are at the core of numerous scientific and technological applications. However, CSPs belong to the NP-complete complexity class, for which the existence (or not) of efficient algorithms remains a major unsolved question in computational complexity theory. In the face of this fundamental difficulty heuristics and approximation methods are used to approach instances of NP (e.g., decision and hard optimization problems). The human brain efficiently handles CSPs both in perception and behavior using spiking neural networks (SNNs), and recent studies have demonstrated that the noise embedded within an SNN can be used as a computational resource to solve CSPs. Here, we provide a software framework for the implementation of such noisy neural solvers on the SpiNNaker massively parallel neuromorphic hardware, further demonstrating their potential to implement a stochastic search that solves instances of P and NP problems expressed as CSPs. This facilitates the exploration of new optimization strategies and the understanding of the computational abilities of SNNs. We demonstrate the basic principles of the framework by solving difficult instances of the Sudoku puzzle and of the map color problem, and explore its application to spin glasses. The solver works as a stochastic dynamical system, which is attracted by the configuration that solves the CSP. The noise allows an optimal exploration of the space of configurations, looking for the satisfiability of all the constraints; if applied discontinuously, it can also force the system to leap to a new random configuration effectively causing a restart.