A Monte Carlo EM approach for partially observable diffusion processes: theory and applications to neural networks

We present a Monte Carlo approach for training partially observable diffusion processes. We apply the approach to diffusion networks, a stochastic version of continuous recurrent neural networks. The approach is aimed at learning probability distributions of continuous paths, not just expected values. Interestingly, the relevant activation statistics used by the learning rule presented here are inner products in the Hilbert space of square integrable functions. These inner products can be computed using Hebbian operations and do not require backpropagation of error signals. Moreover, standard kernel methods could potentially be applied to compute such inner products. We propose that the main reason that recurrent neural networks have not worked well in engineering applications (e.g., speech recognition) is that they implicitly rely on a very simplistic likelihood model. The diffusion network approach proposed here is much richer and may open new avenues for applications of recurrent neural networks. We present some analysis and simulations to support this view. Very encouraging results were obtained on a visual speech recognition task in which neural networks outperformed hidden Markov models.

Fragment generation and support vector machines for inducing SARs

We present a new approach to the induction of SARs based on the generation of structural fragments and support vector machines (SVMs). It is tailored for bio-chemical databases, where the examples are two-dimensional descriptions of chemical compounds. The fragment generator finds all fragments (i.e. linearly connected atoms) that satisfy user-specified constraints regarding their frequency and generality. In this paper, we are querying for fragments within a minimum and a maximum frequency in the dataset. After fragment generation, we propose to apply SVMs to the problem of inducing SARs from these fragments. We conjecture that the SVMs are particularly useful in this context, as they can deal with a large number of features. Experiments in the domains of carcinogenicity and mutagenicity prediction show that the minimum and the maximum frequency queries for fragments can be answered within a reasonable time, and that the predictive accuracy obtained using these fragments is satisfactory. However, further experiments will have to confirm that this is a viable approach to inducing SARs.

The quantitative evaluation of functional neuroimaging experiments: mutual information learning curves

Learning curves are presented as an unbiased means for evaluating the performance of models for neuroimaging data analysis. The learning curve measures the predictive performance in terms of the generalization or prediction error as a function of the number of independent examples (e.g., subjects) used to determine the parameters in the model. Cross-validation resampling is used to obtain unbiased estimates of a generic multivariate Gaussian classifier, for training set sizes from 2 to 16 subjects. We apply the framework to four different activation experiments, in this case [(15)O]water data sets, although the framework is equally valid for multisubject fMRI studies. We demonstrate how the prediction error can be expressed as the mutual information between the scan and the scan label, measured in units of bits. The mutual information learning curve can be used to evaluate the impact of different methodological choices, e.g., classification label schemes, preprocessing choices. Another application for the learning curve is to examine the model performance using bias/variance considerations enabling the researcher to determine if the model performance is limited by statistical bias or variance. We furthermore present the sensitivity map as a general method for extracting activation maps from statistical models within the probabilistic framework and illustrate relationships between mutual information and pattern reproducibility as derived in the NPAIRS framework described in a companion paper.

Nonlinear fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

The nonlinear discriminant function obtained using a minimum squared error cost function can be shown to be directly related to the nonlinear Fisher discriminant (NFD). With the squared error cost function, the orthogonal least squares (OLS) algorithm can be used to find a parsimonious description of the nonlinear discriminant function. Two simple classification techniques will be introduced and tested on a number of real and artificial data sets. The results show that the new classification technique can often perform favourably compared with other state of the art classification techniques.

Computation of the mid-sagittal plane in 3-D brain images

We present a new method to automatically compute, reorient, and recenter the mid-sagittal plane in anatomical and functional three-dimensional (3-D) brain images. This iterative approach is composed of two steps. At first, given an initial guess of the mid-sagittal plane (generally, the central plane of the image grid), the computation of local similarity measures between the two sides of the head allows to identify homologous anatomical structures or functional areas, by way of a block matching procedure. The output is a set of point-to-point correspondences: the centers of homologous blocks. Subsequently, we define the mid-sagittal plane as the one best superposing the points on one side and their counterparts on the other side by reflective symmetry. Practically, the computation of the parameters characterizing the plane is performed by a least trimmed squares estimation. Then, the estimated plane is aligned with the center of the image grid, and the whole process is iterated until convergence. The robust estimation technique we use allows normal or abnormal asymmetrical structures or areas to be treated as outliers, and the plane to be mainly computed from the underlying gross symmetry of the brain. The algorithm is fast and accurate, even for strongly tilted heads, and even in presence of high acquisition noise and bias field, as shown on a large set of synthetic data. The algorithm has also been visually evaluated on a large set of real magnetic resonance (MR) images. We present a few results on isotropic as well as anisotropic anatomical (MR and computed tomography) and functional (single photon emission computed tomography and positron emission tomography) real images, for normal and pathological subjects.

PAH and metal mixtures in New Orleans soils and sediments

The purpose of this study is to determine the degree of PAH contamination and the association of PAHs with inorganic substances in soils and sediments of New Orleans. Bonnet Carré Spillway (BCS) (n = 5) provides modern baseline data, while urban soil samples (CTY) (n = 27) and sediment samples from Bayou St. John (BSJ) (n = 11) provide experimental data for New Orleans. Soil samples were collected from the top 2.5 cm of the surface, air-dried, and sieved (2 mm). Sediments samples were collected with a Wildco-Ekman bottom dredge, air-dried and finely ground. Accelerated solvent extraction (ASE) was used to release PAHs from the samples and analysis was conducted with gas chromatography-mass spectrometry (GC-MS). Metals were extracted using a 5:1 ratio of 1 mol/L nitric acid (room temperature) for soil and sediment samples, shaken for 2 h, centrifuged (1000 x g for 15 min) and filtered. Metal analysis was done by inductively coupled plasma-atomic emission spectrometry (ICP-AES). Mann-Whitney tests show PAH differences (P < 0.001). Ranking of total PAHs is, BSJ sediments (10.3 mg/kg) > CTY soils (3.7 mg/kg) > BCS alluvium (0.28 mg/kg). The sum of the metals are similar for BSJ sediments (698 mg/kg) and CTY soils (679 mg/kg) and significantly lower for BCS (189 mg/kg). Manganese of these samples is similar for each site. For paired samples, Pearson Product Moment Correlation tests reveal that many PAHs are strongly associated with each other at all locations. For BCS alluvium and BSJ sediments, total PAHs are not significantly associated with total metals. For CTY, most pairs of metals are significantly associated, and total soil PAHs are strongly associated with total soil metals (correlation 0.78, P = 4.9 x 10(-4)). The linear model, total soil PAH = 136.3 + 6.25 (total soil metals) forms the basis for a predicted PAH map of New Orleans. Previous empirical research demonstrates an association between soil lead and children's lead exposure. This study indicates that PAHs are part of the soil mixture of accumulated substances and by-products of industrial society that presents exposure potential in cities.

Regulatory sequence analysis: application to the interpretation of gene expression

Microarray technologies for measuring mRNA abundances in cells allow monitoring of gene expression levels for tens of thousands of genes in parallel. By measuring expression responses across hundreds of different conditions or timepoints a relatively detailed gene expression map starts to emerge. Using cluster analysis techniques, it is possible to identify genes that are consistently coexpressed under several different conditions or treatments. These sets of coexpressed genes can then be compared to existing knowledge about biochemical or signalling pathways, the function of unknown genes can be hypothesised by comparing them to other genes with characterised function, or from trends in expression profiles in general - why cell needs to transcribe or silence the genes during particular treatment. The regulation of genes on the DNA level is largely guided by particular sequence features, the transcription factor binding sites, and other signals encaptured in DNA. By analyzing the regulatory regions of the DNA of the genes consistently coexpressed, we can discover the potential signals hidden in DNA by computational analysis methods. The prerequisite for this kind of analysis is the existence of genomic DNA sequence, knowledge about gene locations, and experimental gene expression measurements for a variety of conditions. This article surveys some of the analysis methods and studies for such a computational discovery approach for yeast Saccharomyces cerevisiae.

Drug design by machine learning: support vector machines for pharmaceutical data analysis

We show that the support vector machine (SVM) classification algorithm, a recent development from the machine learning community, proves its potential for structure-activity relationship analysis. In a benchmark test, the SVM is compared to several machine learning techniques currently used in the field. The classification task involves predicting the inhibition of dihydrofolate reductase by pyrimidines, using data obtained from the UCI machine learning repository. Three artificial neural networks, a radial basis function network, and a C5.0 decision tree are all outperformed by the SVM. The SVM is significantly better than all of these, bar a manually capacity-controlled neural network, which takes considerably longer to train.

Biomarker identification by feature wrappers

Gene expression studies bridge the gap between DNA information and trait information by dissecting biochemical pathways into intermediate components between genotype and phenotype. These studies open new avenues for identifying complex disease genes and biomarkers for disease diagnosis and for assessing drug efficacy and toxicity. However, the majority of analytical methods applied to gene expression data are not efficient for biomarker identification and disease diagnosis. In this paper, we propose a general framework to incorporate feature (gene) selection into pattern recognition in the process to identify biomarkers. Using this framework, we develop three feature wrappers that search through the space of feature subsets using the classification error as measure of goodness for a particular feature subset being "wrapped around": linear discriminant analysis, logistic regression, and support vector machines. To effectively carry out this computationally intensive search process, we employ sequential forward search and sequential forward floating search algorithms. To evaluate the performance of feature selection for biomarker identification we have applied the proposed methods to three data sets. The preliminary results demonstrate that very high classification accuracy can be attained by identified composite classifiers with several biomarkers.

Designing linked micromap plots for states with many counties

This paper describes the design of linked micromap plots for showing county estimates on a state by state basis. The linked micromap template was specifically developed to represent spatially indexed statistical summaries. Each plot shows regional names, spatial patterns and statistical patterns while linking them all together. Thus the design is useful for communicating summaries from a host of health and environmental studies. The specific design challenge in this paper is to create one-page plots for the states with 60 to 120 counties. While the county names and micromaps take up substantial space, the three examples demonstrate that there is sufficient space to represent two variables. The basic design has the potential for showing more. Consequently the new designs are suitable for presenting sophisticated summaries.