Nonlinear neural networks: Principles, mechanisms, and architectures

Accentuation, Boolean maps and perception of (dis)similarity in a neural model of visual segmentation

We developed an interactive cortical circuit for visual segmentation that integrates bottom-up and top-down processing to segregate or group visual elements. A bottom-up pathway incorporates stimulus-driven saliency computation, top-down feature-based weighting by relevance and winner-take-all selection. A top-down pathway encompasses multiscale feedback projections, an object-based attention network and a visual segmentation network. Computer simulations have shown that a salient element in the stimulus guides spatial attention and further influences the decomposition of the nearby object into its parts, as postulated by the principle of accentuation. By contrast, when no single salient element is present, top-down feature-based attention highlights all locations occupied by the attended feature and the model forms a Boolean map, i.e., a spatial representation that makes the feature-based grouping explicit. The same distinction between bottom-up and top-down influences in perceptual organization can also be applied to texture perception. The model suggests that the principle of accentuation and feature-based similarity grouping are two manifestations of the same cortical circuit designed to detect similarities and dissimilarities of visual elements in a stimulus.

How the Complexity of Psychological Processes Reframes the Issue of Reproducibility in Psychological Science

In the past decade, various recommendations have been published to enhance the methodological rigor and publication standards in psychological science. However, adhering to these recommendations may have limited impact on the reproducibility of causal effects as long as psychological phenomena continue to be viewed as decomposable into separate and additive statistical structures of causal relationships. In this article, we show that (a) psychological phenomena are patterns emerging from nondecomposable and nonisolable complex processes that obey idiosyncratic nonlinear dynamics, (b) these processual features jeopardize the chances of standard reproducibility of statistical results, and (c) these features call on researchers to reconsider what can and should be reproduced, that is, the psychological processes per se, and the signatures of their complexity and dynamics. Accordingly, we argue for a greater consideration of process causality of psychological phenomena reflected by key properties of complex dynamical systems (CDSs). This implies developing and testing formal models of psychological dynamics, which can be implemented by computer simulation. The scope of the CDS paradigm and its convergences with other paradigms are discussed regarding the reproducibility issue. Ironically, the CDS approach could account for both reproducibility and nonreproducibility of the statistical effects usually sought in mainstream psychological science.

Self-Organizing Maps for Secondary Ion Mass Spectrometry

Secondary ion mass spectrometry (SIMS) is a powerful analytical technique for characterizing the molecular and elemental composition of surfaces. Individual mass spectra can provide information about the mean surface composition, while spatial mapping can elucidate the spatial distributions of molecular species in 2D and 3D with no prior labeling of molecular targets. The data sets produced by SIMS techniques are large and inherently complex, often containing subtle relationships between spatial and molecular features. Machine learning algorithms are well suited to exploring this complexity, making them ideal for data analysis, interpretation, and visualization of SIMS data sets. One such algorithm, the self-organizing map (SOM), is particularly well suited to clustering similar samples and reducing the dimensionality of hyperspectral data sets. Here, we present an introduction to the SOM, a concise mathematical description, and recent examples of its use in SIMS and other related mass spectrometry techniques. These examples demonstrate how SOMs may be used to interpret high volumes of individual mass spectra, imaging, or depth profiling data sets. This review will be useful for specialists in SIMS and other mass spectral techniques seeking to explore self-organizing maps for data analysis.

On convergence properties of the brain-state-in-a-convex-domain

Convergence in the presence of multiple equilibrium points is one of the most fundamental dynamical properties of a neural network (NN). Goal of the paper is to investigate convergence for the classic Brain-State-in-a-Box (BSB) NN model and some of its relevant generalizations named Brain-State-in-a-Convex-Body (BSCB). In particular, BSCB is a class of discrete-time NNs obtained by projecting a linear system onto a convex body of Rn. The main result in the paper is that the BSCB is convergent when the matrix of the linear system is symmetric and positive semidefinite or, otherwise, it is symmetric and the step size does not exceed a given bound depending only on the minimum eigenvalue of the matrix. This result generalizes previous results in the literature for BSB and BSCB and it gives a solid foundation for the use of BSCB as a content addressable memory (CAM). The result is proved via Lyapunov method and LaSalle's Invariance Principle for discrete-time systems and by using some fundamental inequalities enjoyed by the projection operator onto convex sets as Bourbaki-Cheney-Goldstein inequality.

Enhancing Emergency Department Management: A Data-Driven Approach to Detect and Predict Surge Persistence

The prediction of patient attendance in emergency departments (ED) is crucial for effective healthcare planning and resource allocation. This paper proposes an early warning system that can detect emerging trends in ED attendance, offering timely alerts for proactive operational planning. Over 13 years of historical ED attendance data (from January 2010 till December 2022) with 1,700,887 data points were used to develop and validate: (1) a Seasonal Autoregressive Integrated Moving Average with eXogenous factors (SARIMAX) forecasting model; (2) an Exponentially Weighted Moving Average (EWMA) surge prediction model, and (3) a trend persistence prediction model. Drift detection was achieved with the EWMA control chart, and the slopes of a kernel-regressed ED attendance curve were used to train various machine learning (ML) models to predict trend persistence. The EWMA control chart effectively detected significant COVID-19 events in Singapore. The surge prediction model generated preemptive signals on changes in the trends of ED attendance over the COVID-19 pandemic period from January 2020 until December 2022. The persistence of novel trends was further estimated using the trend persistence model, with a mean absolute error of 7.54 (95% CI: 6.77-8.79) days. This study advanced emergency healthcare management by introducing a proactive surge detection framework, which is vital for bolstering the preparedness and agility of emergency departments amid unforeseen health crises.

Prediction of heart failure patients with distinct left ventricular ejection fraction levels using circadian ECG features and machine learning

Heart failure (HF) encompasses a diverse clinical spectrum, including instances of transient HF or HF with recovered ejection fraction, alongside persistent cases. This dynamic condition exhibits a growing prevalence and entails substantial healthcare expenditures, with anticipated escalation in the future. It is essential to classify HF patients into three groups based on their ejection fraction: reduced (HFrEF), mid-range (HFmEF), and preserved (HFpEF), such as for diagnosis, risk assessment, treatment choice, and the ongoing monitoring of heart failure. Nevertheless, obtaining a definitive prediction poses challenges, requiring the reliance on echocardiography. On the contrary, an electrocardiogram (ECG) provides a straightforward, quick, continuous assessment of the patient's cardiac rhythm, serving as a cost-effective adjunct to echocardiography. In this research, we evaluate several machine learning (ML)-based classification models, such as K-nearest neighbors (KNN), neural networks (NN), support vector machines (SVM), and decision trees (TREE), to classify left ventricular ejection fraction (LVEF) for three categories of HF patients at hourly intervals, using 24-hour ECG recordings. Information from heterogeneous group of 303 heart failure patients, encompassing HFpEF, HFmEF, or HFrEF classes, was acquired from a multicenter dataset involving both American and Greek populations. Features extracted from ECG data were employed to train the aforementioned ML classification models, with the training occurring in one-hour intervals. To optimize the classification of LVEF levels in coronary artery disease (CAD) patients, a nested cross-validation approach was employed for hyperparameter tuning. HF patients were best classified using TREE and KNN models, with an overall accuracy of 91.2% and 90.9%, and average area under the curve of the receiver operating characteristics (AUROC) of 0.98, and 0.99, respectively. Furthermore, according to the experimental findings, the time periods of midnight-1 am, 8-9 am, and 10-11 pm were the ones that contributed to the highest classification accuracy. The results pave the way for creating an automated screening system tailored for patients with CAD, utilizing optimal measurement timings aligned with their circadian cycles.

An Investigation of Efficiency Issues in a Low-Pressure Steam Turbine Using Neural Modelling

This study utilizes neural networks to detect and locate thermal anomalies in low-pressure steam turbines, some of which experienced a drop in efficiency. Standard approaches relying on expert knowledge or statistical methods struggled to identify the anomalous steam line due to difficulty in capturing nonlinear and weak relations in the presence of linear and strong ones. In this research, some inputs that linearly relate to outputs have been intentionally neglected. The remaining inputs have been used to train shallow feedforward or long short-term memory neural networks using measured data. The resulting models have been analyzed by Shapley additive explanations, which can determine the impact of individual inputs or model features on outputs. This analysis identified unexpected relations between lines that should not be connected. Subsequently, during periodic plant shutdown, a leak was discovered in the indicated line.

Bio-Inspired Neural Network for Real-Time Evasion of Multi-Robot Systems in Dynamic Environments

In complex and dynamic environments, traditional pursuit-evasion studies may face challenges in offering effective solutions to sudden environmental changes. In this paper, a bio-inspired neural network (BINN) is proposed that approximates a pursuit-evasion game from a neurodynamic perspective instead of formulating the problem as a differential game. The BINN is topologically organized to represent the environment with only local connections. The dynamics of neural activity, characterized by the neurodynamic shunting model, enable the generation of real-time evasive trajectories with moving or sudden-change obstacles. Several simulation and experimental results indicate that the proposed approach is effective and efficient in complex and dynamic environments.

Continual learning framework for a multicenter study with an application to electrocardiogram

Deep learning has been increasingly utilized in the medical field and achieved many goals. Since the size of data dominates the performance of deep learning, several medical institutions are conducting joint research to obtain as much data as possible. However, sharing data is usually prohibited owing to the risk of privacy invasion. Federated learning is a reasonable idea to train distributed multicenter data without direct access; however, a central server to merge and distribute models is needed, which is expensive and hardly approved due to various legal regulations. This paper proposes a continual learning framework for a multicenter study, which does not require a central server and can prevent catastrophic forgetting of previously trained knowledge. The proposed framework contains the continual learning method selection process, assuming that a single method is not omnipotent for all involved datasets in a real-world setting and that there could be a proper method to be selected for specific data. We utilized the fake data based on a generative adversarial network to evaluate methods prospectively, not ex post facto. We used four independent electrocardiogram datasets for a multicenter study and trained the arrhythmia detection model. Our proposed framework was evaluated against supervised and federated learning methods, as well as finetuning approaches that do not include any regulation to preserve previous knowledge. Even without a central server and access to the past data, our framework achieved stable performance (AUROC 0.897) across all involved datasets, achieving comparable performance to federated learning (AUROC 0.901).

Intelligent Fish-Inspired Foraging of Swarm Robots with Sub-Group Behaviors Based on Neurodynamic Models

This paper proposes a novel intelligent approach to swarm robotics, drawing inspiration from the collective foraging behavior exhibited by fish schools. A bio-inspired neural network (BINN) and a self-organizing map (SOM) algorithm are used to enable the swarm to emulate fish-like behaviors such as collision-free navigation and dynamic sub-group formation. The swarm robots are designed to adaptively reconfigure their movements in response to environmental changes, mimicking the flexibility and robustness of fish foraging patterns. The simulation results show that the proposed approach demonstrates improved cooperation, efficiency, and adaptability in various scenarios. The proposed approach shows significant strides in the field of swarm robotics by successfully implementing fish-inspired foraging strategies. The integration of neurodynamic models with swarm intelligence not only enhances the autonomous capabilities of individual robots, but also improves the collective efficiency of the swarm robots.

Canonical circuit computations for computer vision

Advanced computer vision mechanisms have been inspired by neuroscientific findings. However, with the focus on improving benchmark achievements, technical solutions have been shaped by application and engineering constraints. This includes the training of neural networks which led to the development of feature detectors optimally suited to the application domain. However, the limitations of such approaches motivate the need to identify computational principles, or motifs, in biological vision that can enable further foundational advances in machine vision. We propose to utilize structural and functional principles of neural systems that have been largely overlooked. They potentially provide new inspirations for computer vision mechanisms and models. Recurrent feedforward, lateral, and feedback interactions characterize general principles underlying processing in mammals. We derive a formal specification of core computational motifs that utilize these principles. These are combined to define model mechanisms for visual shape and motion processing. We demonstrate how such a framework can be adopted to run on neuromorphic brain-inspired hardware platforms and can be extended to automatically adapt to environment statistics. We argue that the identified principles and their formalization inspires sophisticated computational mechanisms with improved explanatory scope. These and other elaborated, biologically inspired models can be employed to design computer vision solutions for different tasks and they can be used to advance neural network architectures of learning.

How children learn to understand language meanings: a neural model of adult-child multimodal interactions in real-time

This article describes a biological neural network model that can be used to explain how children learn to understand language meanings about the perceptual and affective events that they consciously experience. This kind of learning often occurs when a child interacts with an adult teacher to learn language meanings about events that they experience together. Multiple types of self-organizing brain processes are involved in learning language meanings, including processes that control conscious visual perception, joint attention, object learning and conscious recognition, cognitive working memory, cognitive planning, emotion, cognitive-emotional interactions, volition, and goal-oriented actions. The article shows how all of these brain processes interact to enable the learning of language meanings to occur. The article also contrasts these human capabilities with AI models such as ChatGPT. The current model is called the ChatSOME model, where SOME abbreviates Self-Organizing MEaning.

Adverse Event Profiles of Antiseizure Medications and the Impact of Coadministration on Drug Tolerability in Adults with Epilepsy

BACKGROUND

Antiseizure medication (ASM) as monotherapy or in combination is the treatment of choice for most patients with epilepsy. Therefore, knowledge about the typical adverse events (AEs) for ASMs and other coadministered drugs (CDs) is essential for practitioners and patients. Due to frequent polypharmacy, it is often difficult to clinically assess the AE profiles of ASMs and differentiate the influence of CDs.

OBJECTIVE

This retrospective analysis aimed to determine typical AE profiles for ASMs and assess the impact of CDs on AEs in clinical practice.

METHODS

The Liverpool AE Profile (LAEP) and its domains were used to identify the AE profiles of ASMs based on data from a large German multicenter study (Epi2020). Following established classifications, drugs were grouped according to their mode of action (ASMs) or clinical indication (CDs). Bivariate correlation, multivariate ordinal regression (MORA), and artificial neural network (ANNA) analyses were performed. Bivariate correlation with Fisher's z-transformation was used to compare the correlation strength of LAEP with the Hospital Anxiety and Depression Scale (HADS) and Neurological Disorders Depression Inventory for Epilepsy (NDDI-E) to avoid LAEP bias in the context of antidepressant therapy.

RESULTS

Data from 486 patients were analyzed. The AE profiles of ASM categories and single ASMs matched those reported in the literature. Synaptic vesicle glycoprotein 2A (SV2A) and voltage-gated sodium channel (VGSC) modulators had favorable AE profiles, while brivaracetam was superior to levetiracetam regarding psychobehavioral AEs. MORA revealed that, in addition to seizure frequency, α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) modulators and antidepressants were the only independent predictors of high LAEP values. After Fisher's z-transformation, correlations were significantly lower between LAEP and antidepressants than between LAEP and HADS or NDDI-E. Therefore, a bias in the results toward over interpreting the impact of antidepressants on LAEP was presumed. In the ANNA, perampanel, zonisamide, topiramate, and valproic acid were important nodes in the network, while VGSC and SV2A modulators had low relevance for predicting relevant AEs. Similarly, cardiovascular agents, analgesics, and antipsychotics were important CDs in the ANNA model.

CONCLUSION

ASMs have characteristic AE profiles that are highly reproducible and must be considered in therapeutic decision-making. Therapy using perampanel as an AMPA modulator should be considered cautiously due to its relatively high AE profile. Drugs acting via VGSCs and SV2A receptors are significantly better tolerated than other ASM categories or substances (e.g., topiramate, zonisamide, and valproate). Switching to brivaracetam is advisable in patients with psychobehavioral AEs who take levetiracetam. Because CDs frequently pharmacokinetically interact with ASMs, the cumulative AE profile must be considered.

TRIAL REGISTRATION

DRKS00022024, U1111-1252-5331.

Aperiodic switching event-triggered stabilization of continuous memristive neural networks with interval delays

The stabilization problem is studied for memristive neural networks with interval delays under aperiodic switching event-triggered control. Note that, most of delayed memristive neural networks models studied are discontinuous, which are not the real memristive neural networks. First, a real model of memristive neural networks is proposed by continuous differential equations, furthermore, it is simplified to neural networks with interval matrix uncertainties. Secondly, an aperiodic switching event-trigger is given, and the considered system switches between aperiodic sampled-data system and continuous event-triggered system. Thirdly, by constructing a time-dependent piecewise-defined Lyapunov functional, the stability criterion and the feedback gain design are obtained by linear matrix inequalities. Compared with the existing results, the stability criterion is with lower conservatism. Finally, two neurons are taken as examples to ensure the feasibility of the results.

A Data-Driven Machine Learning Algorithm for Predicting the Outcomes of NBA Games

We propose a new, data-driven model for the prediction of the outcomes of NBA and possibly other basketball league games by using machine learning methods. The paper starts with a strict mathematical formulation of the basketball statistical quantities and the performance indicators derived from them. The backbone of our model is the extended team efficiency index, which consists of two asymmetric parts: (i) the team efficiency index, generally based on some individual efficiency index—in our case, the NBA player efficiency index, and (ii) the comparing part, in which the observed team is rewarded for every selected feature in which it outperforms its rival. Based on the average of the past extended indices, the predicted extended indices are calculated symmetrically for both teams competing in the observed future game. The relative value of those indices defines the win function, which predicts the game outcome. The prediction model includes the concept of the optimal time window (OTW) for the training data. The training datasets were extracted from maximally four and the testing datasets from maximally two of the five consecutive observed NBA seasons (2013/2014–2017/2018). The model uses basic, derived, advanced, and league-wise basketball game elements as its features, whose preparation and extraction were briefly discussed. The proposed model was tested for several choices of the training and testing sets’ seasons, without and with OTWs. The average obtained prediction accuracy is around 66%, and the maximal obtained accuracy is around 78%. This is satisfactory and in the range of better results in the works of other authors.

Epidemic Waves and Exact Solutions of a Sequence of Nonlinear Differential Equations Connected to the SIR Model of Epidemics

The SIR model of epidemic spreading can be reduced to a nonlinear differential equation with an exponential nonlinearity. This differential equation can be approximated by a sequence of nonlinear differential equations with polynomial nonlinearities. The equations from the obtained sequence are treated by the Simple Equations Method (SEsM). This allows us to obtain exact solutions to some of these equations. We discuss several of these solutions. Some (but not all) of the obtained exact solutions can be used for the description of the evolution of epidemic waves. We discuss this connection. In addition, we use two of the obtained solutions to study the evolution of two of the COVID-19 epidemic waves in Bulgaria by a comparison of the solutions with the available data for the infected individuals.

An Overview of the Stability Analysis of Recurrent Neural Networks With Multiple Equilibria

The stability analysis of recurrent neural networks (RNNs) with multiple equilibria has received extensive interest since it is a prerequisite for successful applications of RNNs. With the increasing theoretical results on this topic, it is desirable to review the results for a systematical understanding of the state of the art. This article provides an overview of the stability results of RNNs with multiple equilibria including complete stability and multistability. First, preliminaries on the complete stability and multistability analysis of RNNs are introduced. Second, the complete stability results of RNNs are summarized. Third, the multistability results of various RNNs are reviewed in detail. Finally, future directions in these interesting topics are suggested.

Patterns and Stability of Coupled Multi-Stable Nonlinear Oscillators

Nonlinear isolated and coupled oscillators are extensively studied as prototypical nonlinear dynamics models. Much attention has been devoted to oscillator synchronization or the lack thereof. Here, we study the synchronization and stability of coupled driven-damped Helmholtz-Duffing oscillators in bi-stability regimes. We find that despite the fact that the system parameters and the driving force are identical, the stability of the two states to spatially non-uniform perturbations is very different. Moreover, the final stable states, resulting from these spatial perturbations, are not solely dictated by the wavelength of the perturbing mode and take different spatial configurations in terms of the coupled oscillator phases.

A neural model for vector decomposition and relative-motion perception

The perception of motion not only depends on the detection of motion signals but also on choosing and applying reference-frames according to which motion is interpreted. Here we propose a neural model that implements the common-fate principle for reference-frame selection. The model starts with a retinotopic layer of directionally-tuned motion detectors. The Gestalt common-fate principle is applied to the activities of these detectors to implement in two neural populations the direction and the magnitude (speed) of the reference-frame. The output activities of retinotopic motion-detectors are decomposed using the direction of the reference-frame. The direction and magnitude of the reference-frame are then applied to these decomposed motion-vectors to generate activities that reflect relative-motion perception, i.e., the perception of motion with respect to the prevailing reference-frame. We simulated this model for classical relative motion stimuli, viz., the three-dot, rotating-wheel, and point-walker (biological motion) paradigms and found the model performance to be close to theoretical vector decomposition values. In the three-dot paradigm, the model made the prediction of perceived curved-trajectories for the target dot when its horizontal velocity was slower or faster than the flanking dots. We tested this prediction in two psychophysical experiments and found a good qualitative and quantitative agreement between the model and the data. Our results show that a simple neural network using solely motion information can account for the perception of group and relative motion.

Multi-UAV Path Planning Algorithm Based on BINN-HHO

Multi-UAV (multiple unmanned aerial vehicles) flying in three-dimensional (3D) mountain environments suffer from low stability, long-planned path, and low dynamic obstacle avoidance efficiency. Spurred by these constraints, this paper proposes a multi-UAV path planning algorithm that consists of a bioinspired neural network and improved Harris hawks optimization with a periodic energy decline regulation mechanism (BINN-HHO) to solve the multi-UAV path planning problem in a 3D space. Specifically, in the procession of global path planning, an energy cycle decline mechanism is introduced into HHO and embed it into the energy function, which balances the algorithm's multi-round dynamic iteration between global exploration and local search. Additionally, when the onboard sensors detect a dynamic obstacle during the flight, the improved BINN algorithm conducts a local path replanning for dynamic obstacle avoidance. Once the dynamic obstacles in the sensor detection area disappear, the local path planning is completed, and the UAV returns to the trajectory determined by the global planning. The simulation results show that the proposed Harris hawks algorithm has apparent superiorities in path planning and dynamic obstacle avoidance efficiency compared with the basic Harris hawks optimization, particle swarm optimization (PSO), and the sparrow search algorithm (SSA).

Simple Equations Method (SEsM): An Effective Algorithm for Obtaining Exact Solutions of Nonlinear Differential Equations

Exact solutions of nonlinear differential equations are of great importance to the theory and practice of complex systems. The main point of this review article is to discuss a specific methodology for obtaining such exact solutions. The methodology is called the SEsM, or the Simple Equations Method. The article begins with a short overview of the literature connected to the methodology for obtaining exact solutions of nonlinear differential equations. This overview includes research on nonlinear waves, research on the methodology of the Inverse Scattering Transform method, and the method of Hirota, as well as some of the nonlinear equations studied by these methods. The overview continues with articles devoted to the phenomena described by the exact solutions of the nonlinear differential equations and articles about mathematical results connected to the methodology for obtaining such exact solutions. Several articles devoted to the numerical study of nonlinear waves are mentioned. Then, the approach to the SEsM is described starting from the Hopf-Cole transformation, the research of Kudryashov on the Method of the Simplest Equation, the approach to the Modified Method of the Simplest Equation, and the development of this methodology towards the SEsM. The description of the algorithm of the SEsM begins with the transformations that convert the nonlinearity of the solved complicated equation into a treatable kind of nonlinearity. Next, we discuss the use of composite functions in the steps of the algorithms. Special attention is given to the role of the simple equation in the SEsM. The connection of the methodology with other methods for obtaining exact multisoliton solutions of nonlinear differential equations is discussed. These methods are the Inverse Scattering Transform method and the Hirota method. Numerous examples of the application of the SEsM for obtaining exact solutions of nonlinear differential equations are demonstrated. One of the examples is connected to the exact solution of an equation that occurs in the SIR model of epidemic spreading. The solution of this equation can be used for modeling epidemic waves, for example, COVID-19 epidemic waves. Other examples of the application of the SEsM methodology are connected to the use of the differential equation of Bernoulli and Riccati as simple equations for obtaining exact solutions of more complicated nonlinear differential equations. The SEsM leads to a definition of a specific special function through a simple equation containing polynomial nonlinearities. The special function contains specific cases of numerous well-known functions such as the trigonometric and hyperbolic functions and the elliptic functions of Jacobi, Weierstrass, etc. Among the examples are the solutions of the differential equations of Fisher, equation of Burgers-Huxley, generalized equation of Camassa-Holm, generalized equation of Swift-Hohenberg, generalized Rayleigh equation, etc. Finally, we discuss the connection between the SEsM and the other methods for obtaining exact solutions of nonintegrable nonlinear differential equations. We present a conjecture about the relationship of the SEsM with these methods.

Trend-following with better adaptation to large downside risks

Avoiding losses from long-term trend reversals is challenging, and trend-following is one of the few trading approaches to explore it. While trend-following is popular among investors and has gained increased attention in academia, the recent diminished profitability in equity markets casts doubt on its effectiveness. To clarify its cause and suggest remedies, we thoroughly examine the effect of market conditions and averaging window on recent profitability using four major stock indices in an out-of-sample experiment comparing trend-following rules (moving average and momentum) and a machine-classification-based non-trend-following rule. In addition to the significant advantage of trend-following rules in avoiding downside risks, we find a discrepancy in the optimum averaging window size between trend direction phases, which is exacerbated by a higher positive trend direction ratio. A higher positive trend direction ratio leads to poor performance relative to buy-and-hold returns. This discrepancy creates the dilemma of choosing which trend direction phase to emphasize. Incorporating machine-learning into trend-following is effective for alleviating this dilemma. We find that the profit-maximizing averaging window realizes the level that best balances the dilemma and suggest a simple guideline for selecting the optimum averaging window. We attribute the sluggishness of trend-following in recent equity markets to the insufficient chances of trend reversals rather than their loss of profitability. Our results contribute to improving the performance of trend following by mitigating the dilemma.

CondiS web app: imputation of censored lifetimes for machine learning-based survival analysis

SUMMARY

In the era of big data, machine learning techniques are widely applied to every area in biomedical research including survival analysis. It is well recognized that censoring, which is a common missing issue in survival time data, hampers the direct usage of these machine learning techniques. Here, we present CondiS, a web toolkit with graphical user interface to help impute the survival times for censored observations and predict the survival times for future enrolled patients. CondiS imputes a censored survival time based on its distribution conditional on its observed part. When covariates are available, CondiS-X incorporates this information to further increase the imputation accuracy. Users can also upload data of newly enrolled patients and predict their survival times. As the first web-app tool with an imputation function for censored lifetime data, CondiS web can facilitate conducting survival analysis with machine learning approaches.

AVAILABILITY AND IMPLEMENTATION

CondiS is an open-source application implemented with Shiny in R, available free at: https://biostatistics.mdanderson.org/shinyapps/CondiS/.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

An efficient deep equilibrium model for medical image segmentation

In this paper, we propose an effective method that takes the advantages of classical methods and deep learning technology for medical image segmentation through modeling the neural network as a fixed point iteration seeking for system equilibrium by adding a feedback loop. In particular, the nuclear segmentation of medical image is used as an example to demonstrate the proposed method where it can successfully complete the challenge of segmenting nuclei from cells in different histopathological images. Specifically, the nuclei segmentation is formulated as a dynamic process to search for the system equilibrium. Starting from an initial segmentation generated either by a classic algorithm or pre-trained deep learning model, a sequence of segmentation output is created and combined with the original image to dynamically drive the segmentation towards the expected value. This dynamical extension to neural networks requires little extra change on the backbone deep neural network while it significantly increased model accuracy, generalizability, and stability as demonstrated by intensive experimental results from pathological images of different tissue types across different open datasets.

CondiS: A conditional survival distribution-based method for censored data imputation overcoming the hurdle in machine learning-based survival analysis

Data analyses by machine learning (ML) algorithms are gaining popularity in biomedical research. When time-to-event data are of interest, censoring is common and needs to be properly addressed. Most ML methods cannot conveniently and appropriately take the censoring information into consideration, potentially leading to inaccurate or biased results. We aim to develop a general-purpose method for imputing censored survival data, facilitating downstream ML analysis. In this study, we propose a novel method of imputing the survival times for censored observations. The proposal is based on their conditional survival distributions (CondiS) derived from Kaplan-Meier estimators. CondiS can replace censored observations with their best approximations from the statistical model, allowing for direct application of ML methods. When covariates are available, we extend CondiS by incorporating the covariate information through ML modeling (CondiS-X), which further improves the accuracy of the imputed survival time. Compared with existing methods with similar purposes, the proposed methods achieved smaller prediction errors and higher concordance with the underlying true survival times in extensive simulation studies. We also demonstrated the usage and advantages of the proposed methods through two real-world cancer datasets. The major advantage of CondiS is that it allows for the direct application of standard ML techniques for analysis once the censored survival times are imputed. We present a user-friendly R package to implement our method, which is a useful tool for ML-based biomedical research in this era of big data.

Effects of spatial attention on spatial and temporal acuity: A computational account

In our daily lives, the visual system receives a plethora of visual information that competes for the brain's limited processing capacity. Nevertheless, not all visual information is useful for our cognitive, emotional, social, and ultimately survival purposes. Therefore, the brain employs mechanisms to select critical information and thereby optimizes its limited resources. Attention is the selective process that serves such a function. In particular, covert spatial attention - attending to a particular location in the visual field without eye movements - improves spatial resolution and paradoxically deteriorates temporal resolution. The neural correlates underlying these attentional effects still remainelusive. In this work, we tested a neural model's predictions that explain these phenomena based on interactions between channels with different spatiotemporal sensitivities - namely, the magnocellular (transient) and parvocellular (sustained) channels. More specifically, our model postulates that spatial attention enhances activities in the parvocellular pathway, thereby producing improved performance in spatial resolution tasks. However, the enhancement of parvocellular activities leads to decreased magnocellular activities due to parvo-magno inhibitory interactions. As a result, spatial attention hampers temporal resolution. We compared the predictions of the model to psychophysical data, and show that our model can account qualitatively and quantitatively for the effects of spatial attention on spatial and temporal acuity.

Application of machine learning methods in clinical trials for precision medicine

OBJECTIVE

A key component for precision medicine is a good prediction algorithm for patients' response to treatments. We aim to implement machine learning (ML) algorithms into the response-adaptive randomization (RAR) design and improve the treatment outcomes.

MATERIALS AND METHODS

We incorporated 9 ML algorithms to model the relationship of patient responses and biomarkers in clinical trial design. Such a model predicted the response rate of each treatment for each new patient and provide guidance for treatment assignment. Realizing that no single method may fit all trials well, we also built an ensemble of these 9 methods. We evaluated their performance through quantifying the benefits for trial participants, such as the overall response rate and the percentage of patients who receive their optimal treatments.

RESULTS

Simulation studies showed that the adoption of ML methods resulted in more personalized optimal treatment assignments and higher overall response rates among trial participants. Compared with each individual ML method, the ensemble approach achieved the highest response rate and assigned the largest percentage of patients to their optimal treatments. For the real-world study, we successfully showed the potential improvements if the proposed design had been implemented in the study.

CONCLUSION

In summary, the ML-based RAR design is a promising approach for assigning more patients to their personalized effective treatments, which makes the clinical trial more ethical and appealing. These features are especially desirable for late-stage cancer patients who have failed all the Food and Drug Administration (FDA)-approved treatment options and only can get new treatments through clinical trials.

Ovarian cancer detection using optical coherence tomography and convolutional neural networks

Ovarian cancer has the sixth-largest fatality rate in the United States among all cancers. A non-surgical assay capable of detecting ovarian cancer with acceptable sensitivity and specificity has yet to be developed. However, such a discovery would profoundly impact the pace of the treatment and improvement to patients' quality of life. Achieving such a solution requires high-quality imaging, image processing, and machine learning to support an acceptably robust automated diagnosis. In this work, we propose an automated framework that learns to identify ovarian cancer in transgenic mice from optical coherence tomography (OCT) recordings. Classification is accomplished using a neural network that perceives spatially ordered sequences of tomograms. We present three neural network-based approaches, namely a VGG-supported feed-forward network, a 3D convolutional neural network, and a convolutional LSTM (Long Short-Term Memory) network. Our experimental results show that our models achieve a favorable performance with no manual tuning or feature crafting, despite the challenging noise inherent in OCT images. Specifically, our best performing model, the convolutional LSTM-based neural network, achieves a mean AUC (± standard error) of 0.81 ± 0.037. To the best of the authors' knowledge, no application of machine learning to analyze depth-resolved OCT images of whole ovaries has been documented in the literature. A significant broader impact of this research is the potential transferability of the proposed diagnostic system from transgenic mice to human organs, which would enable medical intervention from early detection of an extremely deadly affliction.

Information Integration and Information Storage in Retinotopic and Non-Retinotopic Sensory Memory

The first stage of the Atkinson–Shiffrin model of human memory is a sensory memory (SM). The visual component of the SM was shown to operate within a retinotopic reference frame. However, a retinotopic SM (rSM) is unable to account for vision under natural viewing conditions because, for example, motion information needs to be analyzed across space and time. For this reason, the SM store of the Atkinson–Shiffrin model has been extended to include a non-retinotopic component (nrSM). In this paper, we analyze findings from two experimental paradigms and show drastically different properties of rSM and nrSM. We show that nrSM involves complex processes such as motion-based reference frames and Gestalt grouping, which establish object identities across space and time. We also describe a quantitative model for nrSM and show drastic differences between the spatio-temporal properties of rSM and nrSM. Since the reference-frame of the latter is non-retinotopic and motion-stream based, we suggest that the spatiotemporal properties of the nrSM are in accordance with the spatiotemporal properties of the motion system. Overall, these findings indicate that, unlike the traditional rSM, which is a relatively passive store, nrSM exhibits sophisticated processing properties to manage the complexities of ecological perception.

Simple Equations Method and Non-Linear Differential Equations with Non-Polynomial Non-Linearity

We discuss the application of the Simple Equations Method (SEsM) for obtaining exact solutions of non-linear differential equations to several cases of equations containing non-polynomial non-linearity. The main idea of the study is to use an appropriate transformation at Step (1.) of SEsM. This transformation has to convert the non-polynomial non- linearity to polynomial non-linearity. Then, an appropriate solution is constructed. This solution is a composite function of solutions of more simple equations. The application of the solution reduces the differential equation to a system of non-linear algebraic equations. We list 10 possible appropriate transformations. Two examples for the application of the methodology are presented. In the first example, we obtain kink and anti- kink solutions of the solved equation. The second example illustrates another point of the study. The point is as follows. In some cases, the simple equations used in SEsM do not have solutions expressed by elementary functions or by the frequently used special functions. In such cases, we can use a special function, which is the solution of an appropriate ordinary differential equation, containing polynomial non-linearity. Specific cases of the use of this function are presented in the second example.

High-level cognition during story listening is reflected in high-order dynamic correlations in neural activity patterns

Our thoughts arise from coordinated patterns of interactions between brain structures that change with our ongoing experiences. High-order dynamic correlations in neural activity patterns reflect different subgraphs of the brain's functional connectome that display homologous lower-level dynamic correlations. Here we test the hypothesis that high-level cognition is reflected in high-order dynamic correlations in brain activity patterns. We develop an approach to estimating high-order dynamic correlations in timeseries data, and we apply the approach to neuroimaging data collected as human participants either listen to a ten-minute story or listen to a temporally scrambled version of the story. We train across-participant pattern classifiers to decode (in held-out data) when in the session each neural activity snapshot was collected. We find that classifiers trained to decode from high-order dynamic correlations yield the best performance on data collected as participants listened to the (unscrambled) story. By contrast, classifiers trained to decode data from scrambled versions of the story yielded the best performance when they were trained using first-order dynamic correlations or non-correlational activity patterns. We suggest that as our thoughts become more complex, they are reflected in higher-order patterns of dynamic network interactions throughout the brain.

On the Use of Composite Functions in the Simple Equations Method to Obtain Exact Solutions of Nonlinear Differential Equations

We discuss the Simple Equations Method (SEsM) for obtaining exact solutions of a class of nonlinear differential equations containing polynomial nonlinearities. We present an amended version of the methodology, which is based on the use of composite functions. The number of steps of the SEsM was reduced from seven to four in the amended version of the methodology. For the case of nonlinear differential equations with polynomial nonlinearities, SEsM can reduce the solved equations to a system of nonlinear algebraic equations. Each nontrivial solution of this algebraic system leads to an exact solution of the solved nonlinear differential equations. We prove the theorems and present examples for the use of composite functions in the methodology of the SEsM for the following three kinds of composite functions: (i) a composite function of one function of one independent variable; (ii) a composite function of two functions of two independent variables; (iii) a composite function of three functions of two independent variables.

Adaptive mechanisms of visual motion discrimination, integration, and segregation

Under ecological conditions, the luminance impinging on the retina varies within a dynamic range of 220 dB. Stimulus contrast can also vary drastically within a scene and eye movements leave little time for sampling luminance. Given these fundamental problems, the human brain allocates a significant amount of resources and deploys both structural and functional solutions that work in tandem to compress this range. Here we propose a new dynamic neural model built upon well-established canonical neural mechanisms. The model consists of two feed-forward stages. The first stage encodes the stimulus spatially and normalizes its activity by extracting contrast and discounting the background luminance. These normalized activities allow a second stage to implement a contrast-dependent spatial-integration strategy. We show how the properties of this model can account for adaptive properties of motion discrimination, integration, and segregation.

Neural dynamics based complete grid coverage by single and multiple mobile robots

Navigation of mobile robots in a grid based environment is useful in applications like warehouse automation. The environment comprises of a number of free grid cells for navigation and remaining grid cells are occupied by obstacles and/or other mobile robots. Such obstructions impose situations of collisions and dead-end. In this work, a neural dynamics based algorithm is proposed for complete coverage of a grid based environment while addressing collision avoidance and dead-end situations. The relative heading of the mobile robot with respect to the neighbouring grid cells is considered to calculate the neural activity. Moreover, diagonal movement of the mobile robot through inter grid cells is restricted to ensure safety from the collision with obstacles and other mobile robots. The circumstances where the proposed algorithm will fail to provide completeness are also discussed along with the possible ways to overcome those situations. Simulation results are presented to show the effectiveness of the proposed algorithm for a single and multiple mobile robots. Moreover, comparative studies illustrate improvements over other algorithms on collision free effective path planning of mobile robots within a grid based environment.

A Canonical Laminar Neocortical Circuit Whose Bottom-Up, Horizontal, and Top-Down Pathways Control Attention, Learning, and Prediction

All perceptual and cognitive circuits in the human cerebral cortex are organized into layers. Specializations of a canonical laminar network of bottom-up, horizontal, and top-down pathways carry out multiple kinds of biological intelligence across different neocortical areas. This article describes what this canonical network is and notes that it can support processes as different as 3D vision and figure-ground perception; attentive category learning and decision-making; speech perception; and cognitive working memory (WM), planning, and prediction. These processes take place within and between multiple parallel cortical streams that obey computationally complementary laws. The interstream interactions that are needed to overcome these complementary deficiencies mix cell properties so thoroughly that some authors have noted the difficulty of determining what exactly constitutes a cortical stream and the differences between streams. The models summarized herein explain how these complementary properties arise, and how their interstream interactions overcome their computational deficiencies to support effective goal-oriented behaviors.

Interactions of Medial and Lateral Prefrontal Cortex in Hierarchical Predictive Coding

Cognitive control and decision-making rely on the interplay of medial and lateral prefrontal cortex (mPFC/lPFC), particularly for circumstances in which correct behavior requires integrating and selecting among multiple sources of interrelated information. While the interaction between mPFC and lPFC is generally acknowledged as a crucial circuit in adaptive behavior, the nature of this interaction remains open to debate, with various proposals suggesting complementary roles in (i) signaling the need for and implementing control, (ii) identifying and selecting appropriate behavioral policies from a candidate set, and (iii) constructing behavioral schemata for performance of structured tasks. Although these proposed roles capture salient aspects of conjoint mPFC/lPFC function, none are sufficiently well-specified to provide a detailed account of the continuous interaction of the two regions during ongoing behavior. A recent computational model of mPFC and lPFC, the Hierarchical Error Representation (HER) model, places the regions within the framework of hierarchical predictive coding, and suggests how they interact during behavioral periods preceding and following salient events. In this manuscript, we extend the HER model to incorporate real-time temporal dynamics and demonstrate how the extended model is able to capture single-unit neurophysiological, behavioral, and network effects previously reported in the literature. Our results add to the wide range of results that can be accounted for by the HER model, and provide further evidence for predictive coding as a unifying framework for understanding PFC function and organization.

Platoon control design for unmanned surface vehicles subject to input delay

Vessel train formation as a new trend has been raised in cooperative control for multiple vessels. This paper addresses formation control design for a group of unmanned surface vehicles platoon considering input delay. To account for connectivity-preserving and collision-avoiding, Barrier Lyapunov function is incorporated into the constraints design of line-of-sight range and bearing. To alleviate the computational burden, neural dynamic model is employed to simplify the control design and smooth the input signals. Besides, input control arising from time delay due to mechanisms and communication is considered in the marine vessels. Within the framework of the backstepping technique, distributed coordination is accomplished in finite time and the uniformly ultimately boundness of overall system is guaranteed via rigorous stability analysis. Finally, the simulation is performed to verify the effectiveness of the proposed control method.

A divided and prioritized experience replay approach for streaming regression

In the streaming learning setting, an agent is presented with a data stream on which to learn from in an online fashion. A common problem is catastrophic forgetting of old knowledge due to updates to the model. Mitigating catastrophic forgetting has received a lot of attention, and a variety of methods exist to solve this problem. In this paper, we present a divided and prioritized experience replay approach for streaming regression, in which relevant observations are retained in the replay, and extra focus is added to poorly estimated observations through prioritization. Using a real-world dataset, the method is compared to the standard sliding window approach. A statistical power analysis is performed, showing how our approach improves performance on rare, important events at a trade-off in performance for more common observations. Close inspections of the dataset are provided, with emphasis on areas where the standard approach fails. A rephrasing of the problem to a binary classification problem is performed to separate common and rare, important events. These results provide an added perspective regarding the improvement made on rare events.•

We divide the prediction space in a streaming regression setting

•

Observations in the experience replay are prioritized for further training by the model’s current error

An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement

Recent speech enhancement research has shown that deep learning techniques are very effective in removing background noise. Many deep neural networks are being proposed, showing promising results for improving overall speech perception. The Deep Multilayer Perceptron, Convolutional Neural Networks, and the Denoising Autoencoder are well-established architectures for speech enhancement; however, choosing between different deep learning models has been mainly empirical. Consequently, a comparative analysis is needed between these three architecture types in order to show the factors affecting their performance. In this paper, this analysis is presented by comparing seven deep learning models that belong to these three categories. The comparison includes evaluating the performance in terms of the overall quality of the output speech using five objective evaluation metrics and a subjective evaluation with 23 listeners; the ability to deal with challenging noise conditions; generalization ability; complexity; and, processing time. Further analysis is then provided while using two different approaches. The first approach investigates how the performance is affected by changing network hyperparameters and the structure of the data, including the Lombard effect. While the second approach interprets the results by visualizing the spectrogram of the output layer of all the investigated models, and the spectrograms of the hidden layers of the convolutional neural network architecture. Finally, a general evaluation is performed for supervised deep learning-based speech enhancement while using SWOC analysis, to discuss the technique’s Strengths, Weaknesses, Opportunities, and Challenges. The results of this paper contribute to the understanding of how different deep neural networks perform the speech enhancement task, highlight the strengths and weaknesses of each architecture, and provide recommendations for achieving better performance. This work facilitates the development of better deep neural networks for speech enhancement in the future.

Simple Equations Method (SEsM): Algorithm, Connection with Hirota Method, Inverse Scattering Transform Method, and Several Other Methods

The goal of this article is to discuss the Simple Equations Method (SEsM) for obtaining exact solutions of nonlinear partial differential equations and to show that several well-known methods for obtaining exact solutions of such equations are connected to SEsM. In more detail, we show that the Hirota method is connected to a particular case of SEsM for a specific form of the function from Step 2 of SEsM and for simple equations of the kinds of differential equations for exponential functions. We illustrate this particular case of SEsM by obtaining the three- soliton solution of the Korteweg-de Vries equation, two-soliton solution of the nonlinear Schrödinger equation, and the soliton solution of the Ishimori equation for the spin dynamics of ferromagnetic materials. Then we show that a particular case of SEsM can be used in order to reproduce the methodology of the inverse scattering transform method for the case of the Burgers equation and Korteweg-de Vries equation. This particular case is connected to use of a specific case of Step 2 of SEsM. This step is connected to: (i) representation of the solution of the solved nonlinear partial differential equation as expansion as power series containing powers of a "small" parameter ϵ; (ii) solving the differential equations arising from this representation by means of Fourier series, and (iii) transition from the obtained solution for small values of ϵ to solution for arbitrary finite values of ϵ. Finally, we show that the much-used homogeneous balance method, extended homogeneous balance method, auxiliary equation method, Jacobi elliptic function expansion method, F-expansion method, modified simple equation method, trial function method and first integral method are connected to particular cases of SEsM.

Quantum-like influence diagrams for decision-making

This article proposes a novel and comprehensive framework on how to describe the probabilistic nature of decision-making process. We suggest extending the quantum-like Bayesian network formalism to incorporate the notion of maximum expected utility to model human paradoxical, sub-optimal and irrational decisions. What distinguishes this work is that we take advantage of the quantum interference effects produced in quantum-like Bayesian Networks during the inference process to influence the probabilities used to compute the maximum expected utility of some decision. The proposed quantum-like decision model is able to (1) predict the probability distributions found in different experiments reported in the literature by modelling uncertainty through quantum interference, (2) to identify decisions that the decision-makers perceive to be optimal within their belief space, but that are actually irrational with respect to expected utility theory, (3) gain an understanding of how the decision-maker's beliefs evolve within a decision-making scenario. The proposed model has the potential to provide new insights in decision science, as well as having direct implications for decision support systems that deal with human data, such as in the fields of economics, finance, psychology, etc.

Asymmetrical performance of a laser-based reservoir computer with optoelectronic feedback

We numerically quantify the performance of a photonic reservoir computer based on a semiconductor laser subject to high-pass filtered optoelectronic feedback. We assess its memory capacity, computational ability, and performance in solving a multi-step prediction task. By analyzing the complex bifurcation landscape of the corresponding delay-differential equation model, we observe that optimal performance occurs at the edge of instability, at the onset of periodic regimes, and unveil a parity asymmetry in the performance with a slight advantage for positive over negative feedback.