High-dimensional functional time series forecasting: An application to age-specific mortality rates

Functional data analysis to characterize disease patterns in frequent longitudinal data: application to bacterial vaginal microbiota patterns using weekly Nugent scores and identification of pattern-specific risk factors

BACKGROUND

Technology advancement has allowed more frequent monitoring of biomarkers. The resulting data structure entails more frequent follow-ups compared to traditional longitudinal studies where the number of follow-up is often small. Such data allow explorations of the role of intra-person variability in understanding disease etiology and characterizing disease processes. A specific example was to characterize pathogenesis of bacterial vaginosis (BV) using weekly vaginal microbiota Nugent assay scores collected over 2 years in post-menarcheeal women from Rakai, Uganda, and to identify risk factors for each vaginal microbiota pattern to inform epidemiological and etiological understanding of the pathogenesis of BV.

METHODS

We use a fully data-driven approach to characterize the longitudinal patters of vaginal microbiota by considering the densely sampled Nugent scores to be random functions over time and performing dimension reduction by functional principal components. Extending a current functional data clustering method, we use a hierarchical functional clustering framework considering multiple data features to help identify clinically meaningful patterns of vaginal microbiota fluctuations. Additionally, multinomial logistic regression was used to identify risk factors for each vaginal microbiota pattern to inform epidemiological and etiological understanding of the pathogenesis of BV.

RESULTS

Using weekly Nugent scores over 2 years of 211 sexually active and post-menarcheal women in Rakai, four patterns of vaginal microbiota variation were identified: persistent with a BV state (high Nugent scores), persistent with normal ranged Nugent scores, large fluctuation of Nugent scores which however are predominantly in the BV state; large fluctuation of Nugent scores but predominantly the scores are in the normal state. Higher Nugent score at the start of an interval, younger age group of less than 20 years, unprotected source for bathing water, a woman's partner's being not circumcised, use of injectable/Norplant hormonal contraceptives for family planning were associated with higher odds of persistent BV in women.

CONCLUSION

The hierarchical functional data clustering method can be used for fully data driven unsupervised clustering of densely sampled longitudinal data to identify clinically informative clusters and risk-factors associated with each cluster.

Depth-based reconstruction method for incomplete functional data

The problem of estimating missing fragments of curves from a functional sample has been widely considered in the literature. However, most reconstruction methods rely on estimating the covariance matrix or the components of its eigendecomposition, which may be difficult. In particular, the estimation accuracy might be affected by the complexity of the covariance function, the noise of the discrete observations, and the poor availability of complete discrete functional data. We introduce a non-parametric alternative based on depth measures for partially observed functional data. Our simulations point out that the benchmark methods perform better when the data come from one population, curves are smooth, and there is a large proportion of complete data. However, our approach is superior when considering more complex covariance structures, non-smooth curves, and when the proportion of complete functions is scarce. Moreover, even in the most severe case of having all the functions incomplete, our method provides good estimates; meanwhile, the competitors are unable. The methodology is illustrated with two real data sets: the Spanish daily temperatures observed in different weather stations and the age-specific mortality by prefectures in Japan. They highlight the interpretability potential of the depth-based method.

Combining Interval Time Series Forecasts. A First Step in a Long Way (Research Agenda)

We observe every day a world more complex, uncertain, and riskier than the world of yesterday. Consequently, having accurate forecasts in economics, finance, energy, health, tourism, and so on; is more critical than ever. Moreover, there is an increasing requirement to provide other types of forecasts beyond point ones such as interval forecasts. After more than 50 years of research, there are two consensuses, “combining forecasts reduces the final forecasting error” and “a simple average of several forecasts often outperforms complicated weighting schemes”, which was named “forecast combination puzzle (FCP)”. The introduction of intervalvalued time series (ITS) concepts and several forecasting methods has been proposed in different papers and gives answers to some big data challenges. Hence, one main issue is how to combine several forecasts obtained for one ITS. This paper proposes some combination schemes with a couple or various ITS forecasts. Some of them extend previous crisp combination schemes incorporating as a novelty the use of Theil’s U. The FCP under the ITS forecasts framework will be analyzed in the context of different accuracy measures and some guidelines will be provided. An agenda for future research in the field of combining forecasts obtained for ITS will be outlined.