Study on outlier detection method of the near infrared spectroscopy analysis by probability metric

Due to the high dimensionality and non-linearity of the near infrared (NIR) spectra data result the difficulty of the outlier measure. This paper proposed a probability based outlier detection method, which adopted the distribution probability of the spectra data to identify outliers at each wavelength by using of copula function. The negative logarithmic function was also used to quantify the overall variation of the joint distribution for the outliers. This method not only enlarges the difference of the spectra between typical samples and outliers, but also can be adapted to multi-type of outliers. Moreover, the jump degree in statistics was introduced for the automated determination of threshold for the outliers, which avoids the threshold setting problem in empirical way and the misjudgment of the outliers. In order to investigate the effectiveness of the algorithm, the recognition of different cases and types of outliers were applied, and compared with the commonly used PCA-Mahalanobis distance, spectral residual (SR) and leverage methods. The experimental results showed that the probability based outlier detection method effectively improved the performance of outlier identification and calibration for NIR analysis.

A Kernel-Based Metric for Balance Assessment

An important goal in causal inference is to achieve balance in the covariates among the treatment groups. In this article, we introduce the concept of distributional balance preserving which requires the distribution of the covariates to be the same in different treatment groups. We also introduce a new balance measure called kernel distance, which is the empirical estimate of the probability metric defined in the reproducing kernel Hilbert spaces. Compared to the traditional balance metrics, the kernel distance measures the difference in the two multivariate distributions instead of the difference in the finite moments of the distributions. Simulation results show that the kernel distance is the best indicator of bias in the estimated casual effect compared to several commonly used balance measures. We then incorporate kernel distance into genetic matching, the state-of-the-art matching procedure and apply the proposed approach to analyze the Early Dieting in Girls study. The study indicates that mothers' overall weight concern increases the likelihood of daughters' early dieting behavior, but the causal effect is not significant.