Diminishing Batch Normalization

In this article, we propose a generalization of the batch normalization (BN) algorithm, diminishing BN (DBN), where we update the BN parameters in a diminishing moving average way. BN is very effective in accelerating the convergence of a neural network training phase that it has become a common practice. Our proposed DBN algorithm retains the overall structure of the original BN algorithm while introducing a weighted averaging update to some trainable parameters. We provide an analysis of the convergence of the DBN algorithm that converges to a stationary point with respect to the trainable parameters. Our analysis can be easily generalized to the original BN algorithm by setting some parameters to constant. To the best of our knowledge, this analysis is the first of its kind for convergence with BN. We analyze a two-layer model with arbitrary activation functions. Common activation functions, such as ReLU and any smooth activation functions, meet our assumptions. In the numerical experiments, we test the proposed algorithm on complex modern CNN models with stochastic gradients (SGs) and ReLU activation on regression, classification, and image reconstruction tasks. We observe that DBN outperforms the original BN algorithm and benchmark layer normalization (LN) on the MNIST, NI, CIFAR-10, CIFAR-100, and Caltech-UCSD Birds-200-2011 datasets with modern complex CNN models such as Resnet-18 and typical FNN models.

Efficient Architecture Search for Continual Learning

Continual learning with neural networks, which aims to learn a sequence of tasks, is an important learning framework in artificial intelligence (AI). However, it often confronts three challenges: 1) overcome the catastrophic forgetting problem; 2) adapt the current network to new tasks; and 3) control its model complexity. To reach these goals, we propose a novel approach named continual learning with efficient architecture search (CLEAS). CLEAS works closely with neural architecture search (NAS), which leverages reinforcement learning techniques to search for the best neural architecture that fits a new task. In particular, we design a neuron-level NAS controller that decides which old neurons from previous tasks should be reused (knowledge transfer) and which new neurons should be added (to learn new knowledge). Such a fine-grained controller allows finding a very concise architecture that can fit each new task well. Meanwhile, since we do not alter the weights of the reused neurons, we perfectly memorize the knowledge learned from the previous tasks. We evaluate CLEAS on numerous sequential classification tasks, and the results demonstrate that CLEAS outperforms other state-of-the-art alternative methods, achieving higher classification accuracy while using simpler neural architectures.

Open-set recognition of breast cancer treatments

Open-set recognition generalizes a classification task by classifying test samples as one of the known classes from training or "unknown." As novel cancer drug cocktails with improved treatment are continually discovered, classifying patients by treatments can naturally be formulated in terms of an open-set recognition problem. Drawbacks, due to modeling unknown samples during training, arise from straightforward implementations of prior work in healthcare open-set learning. Accordingly, we reframe the problem methodology and apply a recent Gaussian mixture variational autoencoder model, which achieves state-of-the-art results for image datasets, to breast cancer patient data. Not only do we obtain more accurate and robust classification results (14% average F1 increase compared to recent methods), but we also reexamine open-set recognition in terms of deployability to a clinical setting.

A mixed methods analysis of caller-emergency medical dispatcher communication during 9-1-1 calls for out-of-hospital cardiac arrest

OBJECTIVE

Recognition of out-of-hospital cardiac arrest (OHCA) during 9-1-1 calls is critically important, but little is known about how laypersons and emergency medical dispatchers (EMDs) communicate. We sought to describe 9-1-1 calls for OHCA.

METHODS

We performed a mixed-methods, retrospective analysis of 9-1-1 calls for OHCA victims in a large urban emergency medical services (EMS) system using a random sampling of cases containing the term "cardiopulmonary resuscitation" (CPR) in the EMS electronic report. A constant comparison qualitative approach with four independent reviewers continued until thematic saturation was achieved. Quantitative analysis employed computational linguistics. Callers' emotional states were rated using the emotional content and cooperation score (ECCS).

RESULTS

Thematic saturation was achieved after 46 calls. Three "OHCA recognition" themes emerged [ 1) disparate OHCA terms used, 2) OHCA mimics create challenges, 3) EMD questions influence recognition]. Three "CPR facilitation" themes emerged [ 1) directive language may facilitate CPR, 2) specific instructions assist CPR, 3) caller's emotions affect CPR initiation]. Callers were generally "anxious but cooperative." Callers saying "pulse" was associated with OHCA recognition.

CONCLUSION

Communication characteristics appear to influence OHCA recognition and CPR facilitation.

PRACTICE IMPLICATIONS

Dispatch protocols that acknowledge characteristics of callers' communication may improve OHCA recognition and CPR facilitation.

Large-Scale Multi-Document Summarization with Information Extraction and Compression

We develop an abstractive summarization framework independent of labeled data for multiple heterogeneous documents. Unlike existing multi-document summarization methods, our framework processes documents telling different stories instead of documents on the same topic. We also enhance an existing sentence fusion method with a uni-directional language model to prioritize fused sentences with higher sentence probability with the goal of increasing readability. Lastly, we construct a total of twelve dataset variations based on CNN/Daily Mail and the NewsRoom datasets, where each document group contains a large and diverse collection of documents to evaluate the performance of our model in comparison with other baseline systems. Our experiments demonstrate that our framework outperforms current state-of-the-art methods in this more generic setting.

A Simple and Fast Algorithm for L1-Norm Kernel PCA

We present an algorithm for L1-norm kernel PCA and provide a convergence analysis for it. While an optimal solution of L2-norm kernel PCA can be obtained through matrix decomposition, finding that of L1-norm kernel PCA is not trivial due to its non-convexity and non-smoothness. We provide a novel reformulation through which an equivalent, geometrically interpretable problem is obtained. Based on the geometric interpretation of the reformulated problem, we present a "fixed-point" type algorithm that iteratively computes a binary weight for each observation. As the algorithm requires only inner products of data vectors, it is computationally efficient and the kernel trick is applicable. In the convergence analysis, we show that the algorithm converges to a local optimal solution in a finite number of steps. Moreover, we provide a rate of convergence analysis, which has been never done for any L1-norm PCA algorithm, proving that the sequence of objective values converges at a linear rate. In numerical experiments, we show that the algorithm is robust in the presence of entry-wise perturbations and computationally scalable, especially in a large-scale setting. Lastly, we introduce an application to outlier detection where the model based on the proposed algorithm outperforms the benchmark algorithms.

Predicting ICU readmission using grouped physiological and medication trends

BACKGROUND

Patients who are readmitted to an intensive care unit (ICU) usually have a high risk of mortality and an increased length of stay. ICU readmission risk prediction may help physicians to re-evaluate the patient's physical conditions before patients are discharged and avoid preventable readmissions. ICU readmission prediction models are often built based on physiological variables. Intuitively, snapshot measurements, especially the last measurements, are effective predictors that are widely used by researchers. However, methods that only use snapshot measurements neglect predictive information contained in the trends of physiological and medication variables. Mean, maximum or minimum values take multiple time points into account and capture their summary statistics, however, these statistics are not able to catch the detailed picture of temporal trends. In this study, we find strong predictors with ability of capturing detailed temporal trends of variables for 30-day readmission risk and build prediction models with high accuracy.

METHODS

We study physiological measurements and medications from the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) clinical dataset. Time series of each variable are converted into trend graphs with nodes being discretized measurements of each variable. Then we extract important temporal trends by applying frequent subgraph mining on the trend graphs. The frequency of a subgraph is a good cue to find important temporal trends since similar patients often share similar trends regarding their pathophysiological evolution under medical interventions. Important temporal trends are then grouped automatically by non-negative matrix factorization. The grouped trends could be considered as an approximate representation of patients' pathophysiological states and medication profiles. We train a logistic regression model to predict 30-day ICU readmission risk based on snapshot measurements, grouped physiological trends and medication trends.

RESULTS

Our dataset consists of 1170 patients who are alive 30 days after discharge from ICU and have at least 12 h of data. In the dataset, 860 patients were not readmitted and 310 were readmitted, within 30 days after discharge. Our model outperforms all comparison models, and shows an improvement in the area under the receiver operating characteristic curve (AUC) of almost 4% from the best comparison model.

CONCLUSIONS

Grouped physiological and medication trends carry predictive information for ICU readmission risk. In order to build predictive models with higher accuracy, we should add grouped physiological and medication trends as complementary features to snapshot measurements.

Identifying Key Words in 9-1-1 Calls for Stroke: A Mixed Methods Approach

OBJECTIVES

Identifying stroke during a 9-1-1 call is critical to timely prehospital care. However, emergency medical dispatchers (EMDs) recognize stroke in less than half of 9-1-1 calls, potentially due to the words used by callers to communicate stroke signs and symptoms. We hypothesized that callers do not typically use words and phrases considered to be classical descriptors of stroke, such as focal neurologic deficits, but that a mixed-methods approach can identify words and phrases commonly used by 9-1-1 callers to describe acute stroke victims.

METHODS

We performed a mixed-method, retrospective study of 9-1-1 call audio recordings for adult patients with confirmed stroke who were transported by ambulance in a large urban city. Content analysis, a qualitative methodology, and computational linguistics, a quantitative methodology, were used to identify key words and phrases used by 9-1-1 callers to describe acute stroke victims. Because a caller's level of emotional distress contributes to the communication during a 9-1-1 call, the Emotional Content and Cooperation Score was scored by a multidisciplinary team.

RESULTS

A total of 110 9-1-1 calls, received between June and September 2013, were analyzed. EMDs recognized stroke in 48% of calls, and the emotional state of most callers (95%) was calm. In 77% of calls in which EMDs recognized stroke, callers specifically used the word "stroke"; however, the word "stroke" was used in only 38% of calls. Vague, non-specific words and phrases were used to describe stroke victims' symptoms in 55% of calls, and 45% of callers used distractor words and phrases suggestive of non-stroke emergencies. Focal neurologic symptoms were described in 39% of calls. Computational linguistics identified 9 key words that were more commonly used in calls where the EMD identified stroke. These words were concordant with terms identified through qualitative content analysis.

CONCLUSIONS

Most 9-1-1 callers used vague, non-specific, or distractor words and phrases and infrequently provide classic stroke descriptions during 9-1-1 calls for stroke. Both qualitative and quantitative methodologies identified similar key words and phrases associated with accurate EMD stroke recognition. This study suggests that tools incorporating commonly used words and phrases could potentially improve EMD stroke recognition.

A Semi-Supervised Learning Approach to Enhance Health Care Community-Based Question Answering: A Case Study in Alcoholism

Background

Community-based question answering (CQA) sites play an important role in addressing health information needs. However, a significant number of posted questions remain unanswered. Automatically answering the posted questions can provide a useful source of information for Web-based health communities.

Objective

In this study, we developed an algorithm to automatically answer health-related questions based on past questions and answers (QA). We also aimed to understand information embedded within Web-based health content that are good features in identifying valid answers.

Methods

Our proposed algorithm uses information retrieval techniques to identify candidate answers from resolved QA. To rank these candidates, we implemented a semi-supervised leaning algorithm that extracts the best answer to a question. We assessed this approach on a curated corpus from Yahoo! Answers and compared against a rule-based string similarity baseline.

Results

On our dataset, the semi-supervised learning algorithm has an accuracy of 86.2%. Unified medical language system–based (health related) features used in the model enhance the algorithm’s performance by proximately 8%. A reasonably high rate of accuracy is obtained given that the data are considerably noisy. Important features distinguishing a valid answer from an invalid answer include text length, number of stop words contained in a test question, a distance between the test question and other questions in the corpus, and a number of overlapping health-related terms between questions.

Conclusions

Overall, our automated QA system based on historical QA pairs is shown to be effective according to the dataset in this case study. It is developed for general use in the health care domain, which can also be applied to other CQA sites.