Quantile regression for functional partially linear model in ultra-high dimensions

Identification of influential weather parameters and seasonal drought prediction in Bangladesh using machine learning algorithm

Droughts pose a severe environmental risk in countries that rely heavily on agriculture, resulting in heightened levels of concern regarding food security and livelihood enhancement. Bangladesh is highly susceptible to environmental hazards, with droughts further exacerbating the precarious situation for its 170 million inhabitants. Therefore, we are endeavouring to highlight the identification of the relative importance of climatic attributes and the estimation of the seasonal intensity and frequency of droughts in Bangladesh. With a period of forty years (1981-2020) of weather data, sophisticated machine learning (ML) methods were employed to classify 35 agroclimatic regions into dry or wet conditions using nine weather parameters, as determined by the Standardized Precipitation Evapotranspiration Index (SPEI). Out of 24 ML algorithms, the four best ML methods, ranger, bagEarth, support vector machine, and random forest (RF) have been identified for the prediction of multi-scale drought indices. The RF classifier and the Boruta algorithms shows that water balance, precipitation, maximum and minimum temperature have a higher influence on drought intensity and occurrence across Bangladesh. The trend of spatio-temporal analysis indicates, drought intensity has decreased over time, but return time has increased. There was significant variation in changing the spatial nature of drought intensity. Spatially, the drought intensity shifted from the northern to central and southern zones of Bangladesh, which had an adverse impact on crop production and the livelihood of rural and urban households. So, this precise study has important implications for the understanding of drought prediction and how to best mitigate its impacts. Additionally, the study emphasizes the need for better collaboration between relevant stakeholders, such as policymakers, researchers, communities, and local actors, to develop effective adaptation strategies and increase monitoring of weather conditions for the meticulous management of droughts in Bangladesh.

Smoothed quantile regression for partially functional linear models in high dimensions

Practitioners of current data analysis are regularly confronted with the situation where the heavy-tailed skewed response is related to both multiple functional predictors and high-dimensional scalar covariates. We propose a new class of partially functional penalized convolution-type smoothed quantile regression to characterize the conditional quantile level between a scalar response and predictors of both functional and scalar types. The new approach overcomes the lack of smoothness and severe convexity of the standard quantile empirical loss, considerably improving the computing efficiency of partially functional quantile regression. We investigate a folded concave penalized estimator for simultaneous variable selection and estimation by the modified local adaptive majorize-minimization (LAMM) algorithm. The functional predictors can be dense or sparse and are approximated by the principal component basis. Under mild conditions, the consistency and oracle properties of the resulting estimators are established. Simulation studies demonstrate a competitive performance against the partially functional standard penalized quantile regression. A real application using Alzheimer's Disease Neuroimaging Initiative data is utilized to illustrate the practicality of the proposed model.

Robust scalar-on-function partial quantile regression

Compared with the conditional mean regression-based scalar-on-function regression model, the scalar-on-function quantile regression is robust to outliers in the response variable. However, it is susceptible to outliers in the functional predictor (called leverage points). This is because the influence function of the regression quantiles is bounded in the response variable but unbounded in the predictor space. The leverage points may alter the eigenstructure of the predictor matrix, leading to poor estimation and prediction results. This study proposes a robust procedure to estimate the model parameters in the scalar-on-function quantile regression method and produce reliable predictions in the presence of both outliers and leverage points. The proposed method is based on a functional partial quantile regression procedure. We propose a weighted partial quantile covariance to obtain functional partial quantile components of the scalar-on-function quantile regression model. After the decomposition, the model parameters are estimated via a weighted loss function, where the robustness is obtained by iteratively reweighting the partial quantile components. The estimation and prediction performance of the proposed method is evaluated by a series of Monte-Carlo experiments and an empirical data example. The results are compared favorably with several existing methods. The method is implemented in an R package robfpqr.

Modeling the evolution of deaths from infectious diseases with functional data models: The case of COVID-19 in Brazil

In this paper, we apply statistical methods for functional data to explore the heterogeneity in the registered number of deaths of COVID-19, over time. The cumulative daily number of deaths in regions across Brazil is treated as continuous curves (functional data). The first stage of the analysis applies clustering methods for functional data to identify and describe potential heterogeneity in the curves and their functional derivatives. The estimated clusters are labeled with different "levels of alert" to identify cities in a possible critical situation. In the second stage of the analysis, we apply a functional quantile regression model for the death curves to explore the associations with functional rates of vaccination and stringency and also with several scalar geographical, socioeconomic and demographic covariates. The proposed model gave a better curve fit at different levels of the cumulative number of deaths when compared to a functional regression model based on ordinary least squares. Our results add to the understanding of the development of COVID-19 death counts.

Extreme quantile estimation for partial functional linear regression models with heavy-tailed distributions

In this article, we propose a novel estimator of extreme conditional quantiles in partial functional linear regression models with heavy-tailed distributions. The conventional quantile regression estimators are often unstable at the extreme tails due to data sparsity, especially for heavy-tailed distributions. We first estimate the slope function and the partially linear coefficient using a functional quantile regression based on functional principal component analysis, which is a robust alternative to the ordinary least squares regression. The extreme conditional quantiles are then estimated by using a new extrapolation technique from extreme value theory. We establish the asymptotic normality of the proposed estimator and illustrate its finite sample performance by simulation studies and an empirical analysis of diffusion tensor imaging data from a cognitive disorder study.

Mapping wind erosion hazard with regression-based machine learning algorithms

Land susceptibility to wind erosion hazard in Isfahan province, Iran, was mapped by testing 16 advanced regression-based machine learning methods: Robust linear regression (RLR), Cforest, Non-convex penalized quantile regression (NCPQR), Neural network with feature extraction (NNFE), Monotone multi-layer perception neural network (MMLPNN), Ridge regression (RR), Boosting generalized linear model (BGLM), Negative binomial generalized linear model (NBGLM), Boosting generalized additive model (BGAM), Spline generalized additive model (SGAM), Spike and slab regression (SSR), Stochastic gradient boosting (SGB), support vector machine (SVM), Relevance vector machine (RVM) and the Cubist and Adaptive network-based fuzzy inference system (ANFIS). Thirteen factors controlling wind erosion were mapped, and multicollinearity among these factors was quantified using the tolerance coefficient (TC) and variance inflation factor (VIF). Model performance was assessed by RMSE, MAE, MBE, and a Taylor diagram using both training and validation datasets. The result showed that five models (MMLPNN, SGAM, Cforest, BGAM and SGB) are capable of delivering a high prediction accuracy for land susceptibility to wind erosion hazard. DEM, precipitation, and vegetation (NDVI) are the most critical factors controlling wind erosion in the study area. Overall, regression-based machine learning models are efficient techniques for mapping land susceptibility to wind erosion hazards.