Adaptive recognition of machining features in sheet metal parts based on a graph class-incremental learning strategy

The integration of computer-aided design (CAD), computer-aided process planning (CAPP), and computer-aided manufacturing (CAM) systems is significantly enhanced by employing deep learning-based automatic feature recognition (AFR) methods. These methods outperform traditional, rule-based approaches, particularly in handling the complexities of intersecting features. However, existing deep learning-based AFR methods face two major challenges. The initial challenge stems from the frequent utilization of voxelized or point-cloud representations of CAD models, resulting in the unfortunate loss of valuable geometric and topological information inherent in original Boundary representation (B-Rep) models. The second challenge involves the limitation of supervised deep learning methods in identifying machining features that are not present in the predefined dataset. This constraint renders them suboptimal for the continually evolving datasets of real industrial scenarios. To address the first challenge, this study introduces a graph-structured language, Multidimensional Attributed Face-Edge Graph (maFEG), crafted to encapsulate the intricate geometric and topological details of CAD models. Furthermore, a graph neural network, Sheet-metalNet, is proposed for the efficient learning and interpretation of maFEGs. To tackle the second challenge, a three-component incremental learning strategy is proposed: an initial phase of pre-training and fine-tuning, a prototype sampling-based replay, and a stage employing knowledge distillation for parameter regularization. The effectiveness of Sheet-metalNet and its complementary incremental learning strategy is evaluated using the open-source MFCAD++ dataset and the newly created SMCAD dataset. Experimental results show that Sheet-metalNet surpasses state-of-the-art AFR methods in machining feature recognition accuracy. Moreover, Sheet-metalNet demonstrates adaptability to dynamic dataset changes, maintaining high performance when encountering newly introduced features, thanks to its innovative incremental learning strategy.

Context-informed incremental learning improves both the performance and resilience of myoelectric control

Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.

Visual Prompting Based Incremental Learning for Semantic Segmentation of Multiplex Immuno-Flourescence Microscopy Imagery

Deep learning approaches are state-of-the-art for semantic segmentation of medical images, but unlike many deep learning applications, medical segmentation is characterized by small amounts of annotated training data. Thus, while mainstream deep learning approaches focus on performance in domains with large training sets, researchers in the medical imaging field must apply new methods in creative ways to meet the more constrained requirements of medical datasets. We propose a framework for incrementally fine-tuning a multi-class segmentation of a high-resolution multiplex (multi-channel) immuno-flourescence image of a rat brain section, using a minimal amount of labelling from a human expert. Our framework begins with a modified Swin-UNet architecture that treats each biomarker in the multiplex image separately and learns an initial "global" segmentation (pre-training). This is followed by incremental learning and refinement of each class using a very limited amount of additional labeled data provided by a human expert for each region and its surroundings. This incremental learning utilizes the multi-class weights as an initialization and uses the additional labels to steer the network and optimize it for each region in the image. In this way, an expert can identify errors in the multi-class segmentation and rapidly correct them by supplying the model with additional annotations hand-picked from the region. In addition to increasing the speed of annotation and reducing the amount of labelling, we show that our proposed method outperforms a traditional multi-class segmentation by a large margin.

Mitigate forgetting in few-shot class-incremental learning using different image views

In the few-shot class incremental learning (FSCIL) setting, new classes with few training examples become available incrementally, and deep learning models suffer from catastrophic forgetting of the previous classes when trained on new classes. Data augmentation techniques are generally used to increase the training data and improve the model performance. In this work, we demonstrate that differently augmented views of the same image obtained by applying data augmentations may not necessarily activate the same set of neurons in the model. Therefore, the information gained by a model regarding a class, when trained using data augmentation, may not necessarily be stored in the same set of neurons in the model. Consequently, during incremental training, even if some of the model weights that store the previously seen class information for a particular view get overwritten, the information of the previous classes for the other views may still remain intact in the other model weights. Therefore, the impact of catastrophic forgetting on the model predictions is different for different data augmentations used during training. Based on this, we present an Augmentation-based Prediction Rectification (APR) approach to reduce the impact of catastrophic forgetting in the FSCIL setting. APR can also augment other FSCIL approaches and significantly improve their performance. We also propose a novel feature synthesis module (FSM) for synthesizing features relevant to the previously seen classes without requiring training data from these classes. FSM outperforms other generative approaches in this setting. We experimentally show that our approach outperforms other methods on benchmark datasets.

Improving transparency and representational generalizability through parallel continual learning

This paper takes a parallel learning approach in continual learning scenarios. We define parallel continual learning as learning a sequence of tasks where the data for the previous tasks, whose distribution may have shifted over time, are also available while learning new tasks. We propose a parallel continual learning method by assigning subnetworks to each task, and simultaneously training only the assigned subnetworks on their corresponding tasks. In doing so, some parts of the network will be shared across multiple tasks. This is unlike the existing literature in continual learning which aims at learning incoming tasks sequentially, with the assumption that the data for the previous tasks have a fixed distribution. Our proposed method offers promises in: (1) Transparency in the network and in the relationship across tasks by enabling examination of the learned representations by independent and shared subnetworks, (2) Representation generalizability through sharing and training subnetworks on multiple tasks simultaneously. Our analysis shows that compared to many competing approaches such as continual learning, neural architecture search, and multi-task learning, parallel continual learning is capable of learning more generalizable representations. Also, (3)Parallel continual learning overcomes the common issue of catastrophic forgetting in continual learning algorithms. This is the first effort to train a neural network on multiple tasks and input domains simultaneously in a continual learning scenario. Our code is available at https://github.com/yours-anonym/PaRT.

Leveraging joint incremental learning objective with data ensemble for class incremental learning

A class-incremental learning problem is characterized by training data becoming available in a phase-by-phase manner. Deep learning models suffer from catastrophic forgetting of the classes in the older phases as they get trained on the classes introduced in the new phase. In this work, we show that the change in orientation of an image has a considerable effect on the model prediction accuracy, which in turn demonstrates the different rates of catastrophic forgetting for the different orientations of the same image, which is a novel finding. Based on this, we propose a data-ensemble approach that combines the predictions for the different orientations of the image to help the model retain information regarding the previously seen classes and thereby reduce the rate of forgetting in the model predictions. However, we cannot directly use the data-ensemble approach if the model is trained using traditional techniques. Therefore, we also propose a novel training approach using a joint-incremental learning objective (JILO) that involves jointly training the network with two incremental learning objectives, i.e., the class-incremental learning objective and our proposed data-incremental learning objective. We empirically demonstrate that JILO is vital to the data-ensemble approach. We apply our proposed approach to state-of-the-art class-incremental learning methods and empirically show that our approach significantly improves the performance of these methods. Our proposed approach significantly improves the performance of the state-of-the-art method (AANets) on the CIFAR-100 dataset by absolute margins of 3.30%, 4.28%, 3.55%, 4.03%, for the number of phases P=50, 25, 10, and 5, respectively, which establishes the efficacy of the proposed work.

An efficient real-time stock prediction exploiting incremental learning and deep learning

Intraday trading is popular among traders due to its ability to leverage price fluctuations in a short timeframe. For traders, real-time price predictions for the next few minutes can be beneficial for making strategies. Real-time prediction is challenging due to the stock market's non-stationary, complex, noisy, chaotic, dynamic, volatile, and non-parametric nature. Machine learning models are considered effective for stock forecasting, yet, their hyperparameters need tuning with the latest market data to incorporate the market's complexities. Usually, models are trained and tested in batches, which smooths the correction process and speeds up the learning. When making intraday stock predictions, the models should forecast for each instance in contrast to the whole batch and learn simultaneously to ensure high accuracy. In this paper, we propose a strategy based on two different learning approaches: incremental learning and Offline-Online learning, to forecast the stock price using the real-time stream of the live market. In incremental learning, the model is updated continuously upon receiving the stock's next instance from the live-stream, while in Offline-Online learning, the model is retrained after each trading session to make sure it incorporates the latest data complexities. These methods were applied to univariate time-series (established from historical stock price) and multivariate time-series (considering historical stock price as well as technical indicators). Extensive experiments were performed on the eight most liquid stocks listed on the American NASDAQ and Indian NSE stock exchanges, respectively. The Offline-Online models outperformed incremental models in terms of low forecasting error.

IDT: An incremental deep tree framework for biological image classification

Nowadays, breast and cervical cancers are respectively the first and fourth most common causes of cancer death in females. It is believed that, automated systems based on artificial intelligence would allow the early diagnostic which increases significantly the chances of proper treatment and survival. Although Convolutional Neural Networks (CNNs) have achieved human-level performance in object classification tasks, the regular growing of the amount of medical data and the continuous increase of the number of classes make them difficult to learn new tasks without being re-trained from scratch. Nevertheless, fine tuning and transfer learning in deep models are techniques that lead to the well-known catastrophic forgetting problem. In this paper, an Incremental Deep Tree (IDT) framework for biological image classification is proposed to address the catastrophic forgetting of CNNs allowing them to learn new classes while maintaining acceptable accuracies on the previously learnt ones. To evaluate the performance of our approach, the IDT framework is compared against with three popular incremental methods, namely iCaRL, LwF and SupportNet. The experimental results on MNIST dataset achieved 87 % of accuracy and the obtained values on the BreakHis, the LBC and the SIPaKMeD datasets are promising with 92 %, 98 % and 93 % respectively.

Knowledge distillation driven instance segmentation for grading prostate cancer

Prostate cancer (PCa) is one of the deadliest cancers in men, and identifying cancerous tissue patterns at an early stage can assist clinicians in timely treating the PCa spread. Many researchers have developed deep learning systems for mass-screening PCa. These systems, however, are commonly trained with well-annotated datasets in order to produce accurate results. Obtaining such data for training is often time and resource-demanding in clinical settings and can result in compromised screening performance. To address these limitations, we present a novel knowledge distillation-based instance segmentation scheme that allows conventional semantic segmentation models to perform instance-aware segmentation to extract stroma, benign, and the cancerous prostate tissues from the whole slide images (WSI) with incremental few-shot training. The extracted tissues are then used to compute majority and minority Gleason scores, which, afterward, are used in grading the PCa as per the clinical standards. The proposed scheme has been thoroughly tested on two datasets, containing around 10,516 and 11,000 WSI scans, respectively. Across both datasets, the proposed scheme outperforms state-of-the-art methods by 2.01% and 4.45%, respectively, in terms of the mean IoU score for identifying prostate tissues, and 10.73% and 11.42% in terms of F1 score for grading PCa according to the clinical standards. Furthermore, the applicability of the proposed scheme is tested under a blind experiment with a panel of expert pathologists, where it achieved a statistically significant Pearson correlation of 0.9192 and 0.8984 with the clinicians' grading.

Real-time DDoS flood attack monitoring and detection (RT-AMD) model for cloud computing

In recent years, the advent of cloud computing has transformed the field of computing and information technology. It has been enabling customers to rent virtual resources and take advantage of various on-demand services with the lowest costs. Despite the advantages of cloud computing, it faces several threats; an example is a distributed denial of service (DDoS) attack, which is considered among the most serious. This article presents real-time monitoring and detection of DDoS attacks on the cloud using a machine learning approach. Naïve Bayes, K-nearest neighbor, decision tree, and random forest machine learning classifiers have been selected to build a predictive model named "Real-Time DDoS flood Attack Monitoring and Detection RT-AMD." The DDoS-2020 dataset was constructed with 70,020 records to evaluate RT-AMD's accuracy. The DDoS-2020 contains three protocols for network/transport-level, which are TCP, DNS, and ICMP. This article evaluates the proposed model by comparing its accuracy with related works. Our model has shown improvement in the results and reached real-time attack detection using incremental learning. The model achieved 99.38% accuracy for the random forest in real-time on the cloud environment and 99.39% on local testing. The RT-AMD was evaluated on the NSL-KDD dataset as well, in which it achieved 99.30% accuracy in real-time in a cloud environment.

Learning from errors: Exploration of the monitoring learning effect

The present study examined spontaneous detection and repair of naming errors in people with aphasia to advance a theoretical understanding of how monitoring impacts learning in lexical access. Prior work in aphasia has found that spontaneous repair, but not mere detection without repair, of semantic naming errors leads to improved naming on those same items in the future when other factors are accounted for. The present study sought to replicate this finding in a new, larger sample of participants and to examine the critical role of self-generated repair in this monitoring learning effect. Twenty-four participants with chronic aphasia with naming impairment provided naming responses to a 660-item corpus of common, everyday objects at two timepoints. At the first timepoint, a randomly selected subset of trials ended in experimenter-provided corrective feedback. Each naming trial was coded for accuracy, error type, and for any monitoring behavior that occurred, specifically detection with repair (i.e., correction), detection without repair, and no detection. Focusing on semantic errors, the original monitoring learning effect was replicated, with enhanced accuracy at a future timepoint when the first trial with that item involved detection with repair, compared to error trials that were not detected. This enhanced accuracy resulted from learning that arose from the first trial rather than the presence of repair simply signifying easier items. A second analysis compared learning from trials of self-corrected errors to that of trials ending in feedback that were detected but not self-corrected and found enhanced learning after self-generated repair. Implications for theories of lexical access and monitoring are discussed.

Adaptive risk prediction system with incremental and transfer learning

Currently, popular methods for prenatal risk assessment of fetal aneuploidies are based on multivariate probabilistic modelling, that are built on decades of scientific research and large-scale multi-center clinical studies. These static models that are deployed to screening labs are rarely updated or adapted to local population characteristics. In this article, we propose an adaptive risk prediction system or ARPS, which considers these changing characteristics and automatically deploys updated risk models. 8 years of real-life Down syndrome screening data was used to firstly develop a distribution shift detection method that captures significant changes in the patient population and secondly a probabilistic risk modelling system that adapts to new data when these changes are detected. Various candidate systems that utilize transfer -and incremental learning that implement different levels of plasticity were tested. Distribution shift detection using a windowed approach provides a computationally less expensive alternative to fitting models at every data block step while not sacrificing performance. This was possible when utilizing transfer learning. Deploying an ARPS to a lab requires careful consideration of the parameters regarding the distribution shift detection and model updating, as they are affected by lab throughput and the incidence of the screened rare disorder. When this is done, ARPS could be also utilized for other population screening problems. We demonstrate with a large real-life dataset that our best performing novel Incremental-Learning-Population-to-Population-Transfer-Learning design can achieve on par prediction performance without human intervention, when compared to a deployed risk screening algorithm that has been manually updated over several years.

Multimodal Continual Learning with Sonographer Eye-Tracking in Fetal Ultrasound

Deep networks have been shown to achieve impressive accuracy for some medical image analysis tasks where large datasets and annotations are available. However, tasks involving learning over new sets of classes arriving over extended time is a different and difficult challenge due to the tendency of reduction in performance over old classes while adapting to new ones. Controlling such a 'forgetting' is vital for deployed algorithms to evolve with new arrivals of data incrementally. Usually, incremental learning approaches rely on expert knowledge in the form of manual annotations or active feedback. In this paper, we explore the role that other forms of expert knowledge might play in making deep networks in medical image analysis immune to forgetting over extended time. We introduce a novel framework for mitigation of this forgetting effect in deep networks considering the case of combining ultrasound video with point-of-gaze tracked for expert sonographers during model training. This is used along with a novel weighted distillation strategy to reduce the propagation of effects due to class imbalance.

Incremental learning of LSTM framework for sensor fusion in attitude estimation

This paper presents a novel method for attitude estimation of an object in 3D space by incremental learning of the Long-Short Term Memory (LSTM) network. Gyroscope, accelerometer, and magnetometer are few widely used sensors in attitude estimation applications. Traditionally, multi-sensor fusion methods such as the Extended Kalman Filter and Complementary Filter are employed to fuse the measurements from these sensors. However, these methods exhibit limitations in accounting for the uncertainty, unpredictability, and dynamic nature of the motion in real-world situations. In this paper, the inertial sensors data are fed to the LSTM network which are then updated incrementally to incorporate the dynamic changes in motion occurring in the run time. The robustness and efficiency of the proposed framework is demonstrated on the dataset collected from a commercially available inertial measurement unit. The proposed framework offers a significant improvement in the results compared to the traditional method, even in the case of a highly dynamic environment. The LSTM framework-based attitude estimation approach can be deployed on a standard AI-supported processing module for real-time applications.

Dynamic sampling of images from various categories for classification based incremental deep learning in fog computing

Incremental learning evolves deep neural network knowledge over time by learning continuously from new data instead of training a model just once with all data present before the training starts. However, in incremental learning, new samples are always streaming in whereby the model to be trained needs to continuously adapt to new samples. Images are considered to be high dimensional data and thus training deep neural networks on such data is very time-consuming. Fog computing is a paradigm that uses fog devices to carry out computation near data sources to reduce the computational load on the server. Fog computing allows democracy in deep learning by enabling intelligence at the fog devices, however, one of the main challenges is the high communication costs between fog devices and the centralized servers especially in incremental learning where data samples are continuously arriving and need to be transmitted to the server for training. While working with Convolutional Neural Networks (CNN), we demonstrate a novel data sampling algorithm that discards certain training images per class before training even starts which reduces the transmission cost from the fog device to the server and the model training time while maintaining model learning performance both for static and incremental learning. Results show that our proposed method can effectively perform data sampling regardless of the model architecture, dataset, and learning settings.

An incremental learning approach to automatically recognize pulmonary diseases from the multi-vendor chest radiographs

The human respiratory network is a vital system that provides oxygen supply and nourishment to the whole body. Pulmonary diseases can cause severe respiratory problems, leading to sudden death if not treated timely. Many researchers have utilized deep learning systems (in both transfer learning and fine-tuning modes) to diagnose pulmonary disorders using chest X-rays (CXRs). However, such systems require exhaustive training efforts on large-scale (and well-annotated) data to effectively diagnose chest abnormalities (at the inference stage). Furthermore, procuring such large-scale data (in a clinical setting) is often infeasible and impractical, especially for rare diseases. With the recent advances in incremental learning, researchers have periodically tuned deep neural networks to learn different classification tasks with few training examples. Although, such systems can resist catastrophic forgetting, they treat the knowledge representations (which the network learns periodically) independently of each other, and this limits their classification performance. Also, to the best of our knowledge, there is no incremental learning-driven image diagnostic framework (to date) that is specifically designed to screen pulmonary disorders from the CXRs. To address this, we present a novel framework that can learn to screen different chest abnormalities incrementally (via few-shot training). In addition to this, the proposed framework is penalized through an incremental learning loss function that infers Bayesian theory to recognize structural and semantic inter-dependencies between incrementally learned knowledge representations to diagnose the pulmonary diseases effectively (at the inference stage), regardless of the scanner specifications. We tested the proposed framework on five public CXR datasets containing different chest abnormalities, where it achieved an accuracy of 0.8405 and the F1 score of 0.8303, outperforming various state-of-the-art incremental learning schemes. It also achieved a highly competitive performance compared to the conventional fine-tuning (transfer learning) approaches while significantly reducing the training and computational requirements.

Interference effects of phonological similarity in word production arise from competitive incremental learning

In the blocked cyclic naming task, native Mandarin speakers named pictures with disyllabic names in small sets and blocks, with the critical manipulation whether pictures within a block shared an atonal syllable or not. We found the expected facilitation when the overlapping portion of responses was in word-initial position, but we also replicated a recent observation that with 'inconsistent' overlap (shared syllables could be either in first or second word position), form overlap causes interference. Crucially, interference also occurred when phonologically unrelated filler trials or trials which required a nonlinguistic response were interleaved with the critical pictures. The same pattern was found with written responses and orthographic radical overlap. The results are best explained via "competitive incremental learning" between lexical and phonological representations. A computer simulation confirms that this principle generates interference, and that the result is unaffected by filler trials. We conclude that incremental learning constitutes a universal principle in the mapping from semantics to phonology in language production.

Data-driven adaptive modeling method for industrial processes and its application in flotation reagent control

In real industrial processes, new process "excitation" patterns that largely deviate from previously collected training data will appear due to disturbances caused by process inputs. To reduce model mismatch, it is important for a data-driven process model to adapt to new process "excitation" patterns. Although efforts have been devoted to developing adaptive process models to deal with this problem, few studies have attempted to develop an adaptive process model that can incrementally learn new process "excitation" patterns without performance degradation on old patterns. In this study, efforts are devoted to enabling data-driven process models with incremental learning ability. First, a novel incremental learning method is proposed for process model updating. Second, an adaptive neural network process model is developed based on the novel incremental learning method. Third, a nonlinear model predictive control based on the adaptive process model is implemented and applied for flotation reagent control. Experiments based on historical data provide evidence that the newly developed adaptive process model can accommodate new process "excitation" patterns and preserve its performance on old patterns. Furthermore, industry experiments carried out in a real-world lead-zinc froth flotation plant provide industrial evidence and show that the newly designed controller is promising for practical flotation reagent control.

A comprehensive study of class incremental learning algorithms for visual tasks

The ability of artificial agents to increment their capabilities when confronted with new data is an open challenge in artificial intelligence. The main challenge faced in such cases is catastrophic forgetting, i.e., the tendency of neural networks to underfit past data when new ones are ingested. A first group of approaches tackles forgetting by increasing deep model capacity to accommodate new knowledge. A second type of approaches fix the deep model size and introduce a mechanism whose objective is to ensure a good compromise between stability and plasticity of the model. While the first type of algorithms were compared thoroughly, this is not the case for methods which exploit a fixed size model. Here, we focus on the latter, place them in a common conceptual and experimental framework and propose the following contributions: (1) define six desirable properties of incremental learning algorithms and analyze them according to these properties, (2) introduce a unified formalization of the class-incremental learning problem, (3) propose a common evaluation framework which is more thorough than existing ones in terms of number of datasets, size of datasets, size of bounded memory and number of incremental states, (4) investigate the usefulness of herding for past exemplars selection, (5) provide experimental evidence that it is possible to obtain competitive performance without the use of knowledge distillation to tackle catastrophic forgetting and (6) facilitate reproducibility by integrating all tested methods in a common open-source repository. The main experimental finding is that none of the existing algorithms achieves the best results in all evaluated settings. Important differences arise notably if a bounded memory of past classes is allowed or not.

An integrated classification model for incremental learning

Incremental Learning is a particular form of machine learning that enables a model to be modified incrementally, when new data becomes available. In this way, the model can adapt to the new data without the lengthy and time-consuming process required for complete model re-training. However, existing incremental learning methods face two significant problems: 1) noise in the classification sample data, 2) poor accuracy of modern classification algorithms when applied to modern classification problems. In order to deal with these issues, this paper proposes an integrated classification model, known as a Pre-trained Truncated Gradient Confidence-weighted (Pt-TGCW) model. Since the pre-trained model can extract and transform image information into a feature vector, the integrated model also shows its advantages in the field of image classification. Experimental results on ten datasets demonstrate that the proposed method outperform the original counterparts.

Keeping up with the genomes: efficient learning of our increasing knowledge of the tree of life

BACKGROUND

It is a computational challenge for current metagenomic classifiers to keep up with the pace of training data generated from genome sequencing projects, such as the exponentially-growing NCBI RefSeq bacterial genome database. When new reference sequences are added to training data, statically trained classifiers must be rerun on all data, resulting in a highly inefficient process. The rich literature of "incremental learning" addresses the need to update an existing classifier to accommodate new data without sacrificing much accuracy compared to retraining the classifier with all data.

RESULTS

We demonstrate how classification improves over time by incrementally training a classifier on progressive RefSeq snapshots and testing it on: (a) all known current genomes (as a ground truth set) and (b) a real experimental metagenomic gut sample. We demonstrate that as a classifier model's knowledge of genomes grows, classification accuracy increases. The proof-of-concept naïve Bayes implementation, when updated yearly, now runs in 1/4th of the non-incremental time with no accuracy loss.

CONCLUSIONS

It is evident that classification improves by having the most current knowledge at its disposal. Therefore, it is of utmost importance to make classifiers computationally tractable to keep up with the data deluge. The incremental learning classifier can be efficiently updated without the cost of reprocessing nor the access to the existing database and therefore save storage as well as computation resources.

The dual origin of lexical perseverations in aphasia: Residual activation and incremental learning

Lexical perseveration, the inappropriate repetition of a previous response, is common in aphasia. Two underlying mechanisms have been proposed: residual activation and incremental learning. Previous attempts to differentiate the two have relied on experimental paradigms that encourage semantically related errors and analysis techniques designed to detect perseverations over short distances, resulting in a bias towards detecting short-lag, semantically related perseverations that both mechanisms can account for. Two key predictions that differentiate these accounts remain untested: only residual activation can explain short-lag, semantically unrelated perseverations, whereas only incremental learning can explain long-lag, semantically related perseverations. In this paper, we used a large set of picture naming trials and a novel analysis technique to test these key predictions in a multi-session study involving six individuals with aphasia. We found clear evidence for both mechanisms in different individuals, demonstrating that either one is sufficient to cause perseveration. Importantly, perseverations due to residual activation were associated with more severely impaired systems than those due to incremental learning, suggesting that a certain degree of structural and functional integrity was necessary for incremental learning. Finally, the results supported a key prediction of the incremental learning account by showing perseverations over longer lags than have previously been reported.

A novel adaptive deep learning model of Covid-19 with focus on mortality reduction strategies

We employ deep learning to propose an Artificial Neural Network (ANN) based and data stream guided real-time incremental learning algorithm for parameter estimation of a non-intrusive, intelligent, adaptive and online analytical model of Covid-19 disease. Modeling and simulation of such problems pose an additional challenge of continuously evolving training data in which the model parameters change over time depending upon external factors. Our main contribution is that in a scenario of continuously evolving training data, unlike typical deep learning techniques, this non-intrusive algorithm eliminates the need to retrain or rebuild the model from scratch every time a new training data set is received. After validating the model, we use it to study the impact of different strategies for epidemic control. Finally, we propose and simulate a strategy of controlled natural immunization through risk-based population compartmentalization (PC) wherein the population is divided in Low Risk (LR) and High Risk (HR) compartments based on risk factors (like comorbidities and age) and subjected to different disease transmission dynamics by isolating the HR compartment while allowing the LR compartment to develop natural immunity. Upon release from the preventive isolation, the HR compartment finds itself surrounded by enough number of immunized individuals to prevent the spread of infection and thus most of the deaths occurring in this group are avoided.

Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning

Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.

Nonconvulsive epileptic seizure monitoring with incremental learning

Nonconvulsive epileptic seizures (NCSz) and nonconvulsive status epilepticus (NCSE) are two neurological entities associated with increment in morbidity and mortality in critically ill patients. In a previous work, we introduced a method which accurately detected NCSz in EEG data (referred here as 'Batch method'). However, this approach was less effective when the EEG features identified at the beginning of the recording changed over time. Such pattern drift is an issue that causes failures of automated seizure detection methods. This paper presents a support vector machine (SVM)-based incremental learning method for NCSz detection that for the first time addresses the seizure evolution in EEG records from patients with epileptic disorders and from ICU having NCSz. To implement the incremental learning SVM, three methodologies are tested. These approaches differ in the way they reduce the set of potentially available support vectors that are used to build the decision function of the classifier. To evaluate the suitability of the three incremental learning approaches proposed here for NCSz detection, first, a comparative study between the three methods is performed. Secondly, the incremental learning approach with the best performance is compared with the Batch method and three other batch methods from the literature. From this comparison, the incremental learning method based on maximum relevance minimum redundancy (MRMR_IL) obtained the best results. MRMR_IL method proved to be an effective tool for NCSz detection in a real-time setting, achieving sensitivity and accuracy values above 99%.

Implicit incremental natural actor critic algorithm

Natural policy gradient (NPG) methods are promising approaches to finding locally optimal policy parameters. The NPG approach works well in optimizing complex policies with high-dimensional parameters, and the effectiveness of NPG methods has been demonstrated in many fields. However, the incremental estimation of the NPG is computationally unstable owing to its high sensitivity to the step-sizes values, especially to the one used to update the estimate of NPG. In this study, we propose a new incremental and stable algorithm for the NPG estimation. We call the proposed algorithm the implicit incremental natural actor critic (I2NAC), and it is based on the idea of the implicit update. The convergence analysis for I2NAC is provided. Theoretical analysis results indicate the stability of I2NAC and the instability of conventional incremental NPG methods. Numerical experiments were performed, and the results show that I2NAC is less sensitive to the values of the meta-parameters, including the step-size for the NPG update, compared to the existing incremental NPG method.

Transferring and generalizing deep-learning-based neural encoding models across subjects

Recent studies have shown the value of using deep learning models for mapping and characterizing how the brain represents and organizes information for natural vision. However, modeling the relationship between deep learning models and the brain (or encoding models), requires measuring cortical responses to large and diverse sets of natural visual stimuli from single subjects. This requirement limits prior studies to few subjects, making it difficult to generalize findings across subjects or for a population. In this study, we developed new methods to transfer and generalize encoding models across subjects. To train encoding models specific to a target subject, the models trained for other subjects were used as the prior models and were refined efficiently using Bayesian inference with a limited amount of data from the target subject. To train encoding models for a population, the models were progressively trained and updated with incremental data from different subjects. For the proof of principle, we applied these methods to functional magnetic resonance imaging (fMRI) data from three subjects watching tens of hours of naturalistic videos, while a deep residual neural network driven by image recognition was used to model visual cortical processing. Results demonstrate that the methods developed herein provide an efficient and effective strategy to establish both subject-specific and population-wide predictive models of cortical representations of high-dimensional and hierarchical visual features.

Person Re-identification in Identity Regression Space

Most existing person re-identification (re-id) methods are unsuitable for real-world deployment due to two reasons: Unscalability to large population size, and Inadaptability over time. In this work, we present a unified solution to address both problems. Specifically, we propose to construct an identity regression space (IRS) based on embedding different training person identities (classes) and formulate re-id as a regression problem solved by identity regression in the IRS. The IRS approach is characterised by a closed-form solution with high learning efficiency and an inherent incremental learning capability with human-in-the-loop. Extensive experiments on four benchmarking datasets (VIPeR, CUHK01, CUHK03 and Market-1501) show that the IRS model not only outperforms state-of-the-art re-id methods, but also is more scalable to large re-id population size by rapidly updating model and actively selecting informative samples with reduced human labelling effort.

Merging weighted SVMs for parallel incremental learning

Parallel incremental learning is an effective approach for rapidly processing large scale data streams, where parallel and incremental learning are often treated as two separate problems and solved one after another. Incremental learning can be implemented by merging knowledge from incoming data and parallel learning can be performed by merging knowledge from simultaneous learners. We propose to simultaneously solve the two learning problems with a single process of knowledge merging, and we propose parallel incremental wESVM (weighted Extreme Support Vector Machine) to do so. Here, wESVM is reformulated such that knowledge from subsets of training data can be merged via simple matrix addition. As such, the proposed algorithm is able to conduct parallel incremental learning by merging knowledge over data slices arriving at each incremental stage. Both theoretical and experimental studies show the equivalence of the proposed algorithm to batch wESVM in terms of learning effectiveness. In particular, the algorithm demonstrates desired scalability and clear speed advantages to batch retraining.

Assessing a continuum of lexical-semantic knowledge in the second year of life: A multimodal approach

Behavioral dissociations in young children's visual and haptic responses have been taken as evidence that word knowledge is not all-or-none but instead exists on a continuum from absence of knowledge, to partial knowledge, to robust knowledge. This longitudinal study tested a group of 16- to 18-month-olds, 6months after their initial visit, to replicate results of partial understanding as shown by visual-haptic dissociations and to determine whether partial knowledge of word-referent relations can be leveraged for future word recognition. Results show that, like 16-month-olds, 22-month-olds demonstrate behavioral dissociations exhibited by rapid visual reaction times to a named referent but incorrect haptic responses. Furthermore, results suggest that partial word knowledge at one time predicts the degree to which that word will be understood in the future.

Incremental implicit learning of bundles of statistical patterns

Forming an accurate representation of a task environment often takes place incrementally as the information relevant to learning the representation only unfolds over time. This incremental nature of learning poses an important problem: it is usually unclear whether a sequence of stimuli consists of only a single pattern, or multiple patterns that are spliced together. In the former case, the learner can directly use each observed stimulus to continuously revise its representation of the task environment. In the latter case, however, the learner must first parse the sequence of stimuli into different bundles, so as to not conflate the multiple patterns. We created a video-game statistical learning paradigm and investigated (1) whether learners without prior knowledge of the existence of multiple "stimulus bundles" - subsequences of stimuli that define locally coherent statistical patterns - could detect their presence in the input and (2) whether learners are capable of constructing a rich representation that encodes the various statistical patterns associated with bundles. By comparing human learning behavior to the predictions of three computational models, we find evidence that learners can handle both tasks successfully. In addition, we discuss the underlying reasons for why the learning of stimulus bundles occurs even when such behavior may seem irrational.

A Self-Organizing Incremental Neural Network based on local distribution learning

In this paper, we propose an unsupervised incremental learning neural network based on local distribution learning, which is called Local Distribution Self-Organizing Incremental Neural Network (LD-SOINN). The LD-SOINN combines the advantages of incremental learning and matrix learning. It can automatically discover suitable nodes to fit the learning data in an incremental way without a priori knowledge such as the structure of the network. The nodes of the network store rich local information regarding the learning data. The adaptive vigilance parameter guarantees that LD-SOINN is able to add new nodes for new knowledge automatically and the number of nodes will not grow unlimitedly. While the learning process continues, nodes that are close to each other and have similar principal components are merged to obtain a concise local representation, which we call a relaxation data representation. A denoising process based on density is designed to reduce the influence of noise. Experiments show that the LD-SOINN performs well on both artificial and real-word data.

Structured patch model for a unified automatic and interactive segmentation framework

We present a novel interactive segmentation framework incorporating a priori knowledge learned from training data. The knowledge is learned as a structured patch model (StPM) comprising sets of corresponding local patch priors and their pairwise spatial distribution statistics which represent the local shape and appearance along its boundary and the global shape structure, respectively. When successive user annotations are given, the StPM is appropriately adjusted in the target image and used together with the annotations to guide the segmentation. The StPM reduces the dependency on the placement and quantity of user annotations with little increase in complexity since the time-consuming StPM construction is performed offline. Furthermore, a seamless learning system can be established by directly adding the patch priors and the pairwise statistics of segmentation results to the StPM. The proposed method was evaluated on three datasets, respectively, of 2D chest CT, 3D knee MR, and 3D brain MR. The experimental results demonstrate that within an equal amount of time, the proposed interactive segmentation framework outperforms recent state-of-the-art methods in terms of accuracy, while it requires significantly less computing and editing time to obtain results with comparable accuracy.

Active incremental Support Vector Machine for oil and gas pipeline defects prediction system using long range ultrasonic transducers

This work proposes a long range ultrasonic transducers technique in conjunction with an active incremental Support Vector Machine (SVM) classification approach that is used for real-time pipeline defects prediction and condition monitoring. Oil and gas pipeline defects are detected using various techniques. One of the most prevalent techniques is the use of "smart pigs" to travel along the pipeline and detect defects using various types of sensors such as magnetic sensors and eddy-current sensors. A critical short coming of "smart pigs" is the inability to monitor continuously and predict the onset of defects. The emergence of permanently installed long range ultrasonics transducers systems enable continuous monitoring to be achieved. The needs for and the challenges of the proposed technique are presented. The experimental results show that the proposed technique achieves comparable classification accuracy as when batch training is used, while the computational time is decreased, using 56 feature data points acquired from a lab-scale pipeline defect generating experimental rig.

Learning a generative probabilistic grammar of experience: a process-level model of language acquisition

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front-ranging from issues of generativity to the replication of human experimental findings-by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach.

Multilayered neural network with structural lateral inhibition for incremental learning and conceptualization

Distributed connectionist networks have difficulty learning incrementally because the representations in the network overlap. Therefore, it is necessary to reduce the overlaps of representations for incremental learning. At the same time, the representational overlaps give these networks the ability to generalize. In this study, we use a modified multilayered neural network to numerically examine the trade-off between incremental learning and generalization abilities, and then we propose a novel network model with structural lateral inhibitions to reconcile the two abilities. We also analyze the behavior of the proposed model using Formal Concept Analysis, which reveals that the network implements "conceptualization": differentiation and meditation between intensional and extensional representations. This study suggests a new paradigm for the traditional question, whether representations in the brain are distributed or not.

Top-down attention based on object representation and incremental memory for knowledge building and inference

Humans can efficiently perceive arbitrary visual objects based on an incremental learning mechanism with selective attention. This paper proposes a new task specific top-down attention model to locate a target object based on its form and color representation along with a bottom-up saliency based on relativity of primitive visual features and some memory modules. In the proposed model top-down bias signals corresponding to the target form and color features are generated, which draw the preferential attention to the desired object by the proposed selective attention model in concomitance with the bottom-up saliency process. The object form and color representation and memory modules have an incremental learning mechanism together with a proper object feature representation scheme. The proposed model includes a Growing Fuzzy Topology Adaptive Resonance Theory (GFTART) network which plays two important roles in object color and form biased attention; one is to incrementally learn and memorize color and form features of various objects, and the other is to generate a top-down bias signal to localize a target object by focusing on the candidate local areas. Moreover, the GFTART network can be utilized for knowledge inference which enables the perception of new unknown objects on the basis of the object form and color features stored in the memory during training. Experimental results show that the proposed model is successful in focusing on the specified target objects, in addition to the incremental representation and memorization of various objects in natural scenes. In addition, the proposed model properly infers new unknown objects based on the form and color features of previously trained objects.

Scalable system for classification of white blood cells from Leishman stained blood stain images

INTRODUCTION

The White Blood Cell (WBC) differential count yields clinically relevant information about health and disease. Currently, pathologists manually annotate the WBCs, which is time consuming and susceptible to error, due to the tedious nature of the process. This study aims at automation of the Differential Blood Count (DBC) process, so as to increase productivity and eliminate human errors.

MATERIALS AND METHODS

The proposed system takes the peripheral Leishman blood stain images as the input and generates a count for each of the WBC subtypes. The digitized microscopic images are stain normalized for the segmentation, to be consistent over a diverse set of slide images. Active contours are employed for robust segmentation of the WBC nucleus and cytoplasm. The seed points are generated by processing the images in Hue-Saturation-Value (HSV) color space. An efficient method for computing a new feature, 'number of lobes,' for discrimination of WBC subtypes, is introduced in this article. This method is based on the concept of minimization of the compactness of each lobe. The Naive Bayes classifier, with Laplacian correction, provides a fast, efficient, and robust solution to multiclass categorization problems. This classifier is characterized by incremental learning and can also be embedded within the database systems.

RESULTS

An overall accuracy of 92.45% and 92.72% over the training and testing sets has been obtained, respectively.

CONCLUSION

Thus, incremental learning is inducted into the Naive Bayes Classifier, to facilitate fast, robust, and efficient classification, which is evident from the high sensitivity achieved for all the subtypes of WBCs.