1
|
Al Mamun MA, Sarker MR, Sarkar MAR, Roy SK, Nihad SAI, McKenzie AM, Hossain MI, Kabir MS. Identification of influential weather parameters and seasonal drought prediction in Bangladesh using machine learning algorithm. Sci Rep 2024; 14:566. [PMID: 38177219 PMCID: PMC10767098 DOI: 10.1038/s41598-023-51111-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 12/30/2023] [Indexed: 01/06/2024] Open
Abstract
Droughts pose a severe environmental risk in countries that rely heavily on agriculture, resulting in heightened levels of concern regarding food security and livelihood enhancement. Bangladesh is highly susceptible to environmental hazards, with droughts further exacerbating the precarious situation for its 170 million inhabitants. Therefore, we are endeavouring to highlight the identification of the relative importance of climatic attributes and the estimation of the seasonal intensity and frequency of droughts in Bangladesh. With a period of forty years (1981-2020) of weather data, sophisticated machine learning (ML) methods were employed to classify 35 agroclimatic regions into dry or wet conditions using nine weather parameters, as determined by the Standardized Precipitation Evapotranspiration Index (SPEI). Out of 24 ML algorithms, the four best ML methods, ranger, bagEarth, support vector machine, and random forest (RF) have been identified for the prediction of multi-scale drought indices. The RF classifier and the Boruta algorithms shows that water balance, precipitation, maximum and minimum temperature have a higher influence on drought intensity and occurrence across Bangladesh. The trend of spatio-temporal analysis indicates, drought intensity has decreased over time, but return time has increased. There was significant variation in changing the spatial nature of drought intensity. Spatially, the drought intensity shifted from the northern to central and southern zones of Bangladesh, which had an adverse impact on crop production and the livelihood of rural and urban households. So, this precise study has important implications for the understanding of drought prediction and how to best mitigate its impacts. Additionally, the study emphasizes the need for better collaboration between relevant stakeholders, such as policymakers, researchers, communities, and local actors, to develop effective adaptation strategies and increase monitoring of weather conditions for the meticulous management of droughts in Bangladesh.
Collapse
Affiliation(s)
- Md Abdullah Al Mamun
- Agricultural Statistics Division, Bangladesh Rice Research Institute, Gazipur, 1701, Bangladesh
| | - Mou Rani Sarker
- Sustainable Impact Platform, International Rice Research Institute, Dhaka, 1213, Bangladesh
| | - Md Abdur Rouf Sarkar
- School of Economics, Zhongnan University of Economics and Law, Wuhan, 430073, China.
- Agricultural Economics Division, Bangladesh Rice Research Institute, Gazipur, 1701, Bangladesh.
| | - Sujit Kumar Roy
- Institute of Water and Flood Management, Bangladesh University of Engineering and Technology, Dhaka, 1000, Bangladesh
| | | | - Andrew M McKenzie
- Department of Agricultural Economics and Agribusiness, The University of Arkansas, Fayetteville, AR, 72701, USA
| | - Md Ismail Hossain
- Agricultural Statistics Division, Bangladesh Rice Research Institute, Gazipur, 1701, Bangladesh
| | | |
Collapse
|
2
|
Wang Z, Bai Y, Härdle WK, Tian M. Smoothed quantile regression for partially functional linear models in high dimensions. Biom J 2023; 65:e2200060. [PMID: 37147793 DOI: 10.1002/bimj.202200060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2022] [Revised: 11/21/2022] [Accepted: 12/11/2022] [Indexed: 05/07/2023]
Abstract
Practitioners of current data analysis are regularly confronted with the situation where the heavy-tailed skewed response is related to both multiple functional predictors and high-dimensional scalar covariates. We propose a new class of partially functional penalized convolution-type smoothed quantile regression to characterize the conditional quantile level between a scalar response and predictors of both functional and scalar types. The new approach overcomes the lack of smoothness and severe convexity of the standard quantile empirical loss, considerably improving the computing efficiency of partially functional quantile regression. We investigate a folded concave penalized estimator for simultaneous variable selection and estimation by the modified local adaptive majorize-minimization (LAMM) algorithm. The functional predictors can be dense or sparse and are approximated by the principal component basis. Under mild conditions, the consistency and oracle properties of the resulting estimators are established. Simulation studies demonstrate a competitive performance against the partially functional standard penalized quantile regression. A real application using Alzheimer's Disease Neuroimaging Initiative data is utilized to illustrate the practicality of the proposed model.
Collapse
Affiliation(s)
- Zhihao Wang
- Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, P. R. China
- School of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, P. R. China
| | - Yongxin Bai
- School of Science, Beijing Information Science and Technology University, Beijing, P. R. China
| | - Wolfgang K Härdle
- School of Business and Economics, Humboldt-Universität Zu Berlin, Berlin, Germany
- Department of Information Management and Finance, National Yang Ming Chiao Tung University (NYCU), Hsinchu City, Taiwan
| | - Maozai Tian
- Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, P. R. China
- School of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi, P. R. China
| |
Collapse
|
3
|
Beyaztas U, Tez M, Lin Shang H. Robust scalar-on-function partial quantile regression. J Appl Stat 2023; 51:1359-1377. [PMID: 38835823 PMCID: PMC11146266 DOI: 10.1080/02664763.2023.2202464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 04/08/2023] [Indexed: 06/06/2024]
Abstract
Compared with the conditional mean regression-based scalar-on-function regression model, the scalar-on-function quantile regression is robust to outliers in the response variable. However, it is susceptible to outliers in the functional predictor (called leverage points). This is because the influence function of the regression quantiles is bounded in the response variable but unbounded in the predictor space. The leverage points may alter the eigenstructure of the predictor matrix, leading to poor estimation and prediction results. This study proposes a robust procedure to estimate the model parameters in the scalar-on-function quantile regression method and produce reliable predictions in the presence of both outliers and leverage points. The proposed method is based on a functional partial quantile regression procedure. We propose a weighted partial quantile covariance to obtain functional partial quantile components of the scalar-on-function quantile regression model. After the decomposition, the model parameters are estimated via a weighted loss function, where the robustness is obtained by iteratively reweighting the partial quantile components. The estimation and prediction performance of the proposed method is evaluated by a series of Monte-Carlo experiments and an empirical data example. The results are compared favorably with several existing methods. The method is implemented in an R package robfpqr.
Collapse
Affiliation(s)
- Ufuk Beyaztas
- Department of Statistics, Marmara University, Kadikoy-Istanbul, Turkey
| | - Mujgan Tez
- Department of Statistics, Marmara University, Kadikoy-Istanbul, Turkey
| | - Han Lin Shang
- Department of Actuarial Studies and Business Analytics, Macquarie University, Sydney, Australia
| |
Collapse
|
4
|
Wu C, Ling N, Vieu P, Liang W. Partially functional linear quantile regression model and variable selection with censoring indicators MAR. J MULTIVARIATE ANAL 2023. [DOI: 10.1016/j.jmva.2023.105189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/07/2023]
|
5
|
Collazos JAA, Dias R, Medeiros MC. Modeling the evolution of deaths from infectious diseases with functional data models: The case of COVID-19 in Brazil. Stat Med 2023; 42:993-1012. [PMID: 36631172 DOI: 10.1002/sim.9654] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Revised: 11/09/2022] [Accepted: 12/29/2022] [Indexed: 01/13/2023]
Abstract
In this paper, we apply statistical methods for functional data to explore the heterogeneity in the registered number of deaths of COVID-19, over time. The cumulative daily number of deaths in regions across Brazil is treated as continuous curves (functional data). The first stage of the analysis applies clustering methods for functional data to identify and describe potential heterogeneity in the curves and their functional derivatives. The estimated clusters are labeled with different "levels of alert" to identify cities in a possible critical situation. In the second stage of the analysis, we apply a functional quantile regression model for the death curves to explore the associations with functional rates of vaccination and stringency and also with several scalar geographical, socioeconomic and demographic covariates. The proposed model gave a better curve fit at different levels of the cumulative number of deaths when compared to a functional regression model based on ordinary least squares. Our results add to the understanding of the development of COVID-19 death counts.
Collapse
Affiliation(s)
- Julian A A Collazos
- Department of Mathematics, New Granada Military University, Bogotá, Colombia
| | - Ronaldo Dias
- Department of Statistics, State University of Campinas, Campinas, Brazil
| | - Marcelo C Medeiros
- Department of Economics, Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil
| |
Collapse
|
6
|
Liu Y, Wang Z, Tian M, Yu K. Estimation and variable selection for generalized functional partially varying coefficient hybrid models. Stat Pap (Berl) 2022. [DOI: 10.1007/s00362-022-01383-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
7
|
Subgroup analysis for high-dimensional functional regression. J MULTIVARIATE ANAL 2022. [DOI: 10.1016/j.jmva.2022.105100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
8
|
Jiang Z, Huang Z, Zhang J. Functional single-index composite quantile regression. METRIKA 2022. [DOI: 10.1007/s00184-022-00887-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
9
|
Bindele HF, Denhere M, Sun W. Generalized signed-rank estimation and selection for the functional linear model. STATISTICS-ABINGDON 2022. [DOI: 10.1080/02331888.2022.2084094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Huybrechts F. Bindele
- Department of Mathematics and Statistics, University of South Alabama, Mobile, AL, USA
| | | | - Wei Sun
- Auburn University, Auburn, AL, USA
| |
Collapse
|
10
|
Zhou Y, Zhang W, Lin H, Lian H. Partially linear functional quantile regression in a reproducing kernel Hilbert space. J Nonparametr Stat 2022. [DOI: 10.1080/10485252.2022.2073354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Affiliation(s)
- Yan Zhou
- College of Mathematics and Statistics, Institute of Statistical Sciences, Shenzhen University, Shenzhen, People's Republic of China
| | - Weiping Zhang
- Department of Statistics, USTC, Hefei, People's Republic of China
| | - Hongmei Lin
- School of Statistics and Information, Shanghai University of International Business and Economics, Shanghai, People's Republic of China
| | - Heng Lian
- Department of Mathematics, City University of Hong Kong, Hong Kong, People's Republic of China
| |
Collapse
|
11
|
Ma H, Liu C, Xu S, Yang J. Subgroup analysis for functional partial linear regression model. CAN J STAT 2022. [DOI: 10.1002/cjs.11696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Haiqiang Ma
- School of Statistics Jiangxi University of Finance and Economics Nanchang Jiangxi 330013 China
| | - Chao Liu
- Department of Statistics and Data Science The Southern University of Science and Technology Shenzhen Guangdong 518055 China
| | - Sheng Xu
- Department of Applied Mathematics The Hong Kong Polytechnic University Kowloon, Hong Kong (SAR) China
| | - Jin Yang
- Biostatistics and Bioinformatics Branch Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health Bethesda MD 20817 U.S.A
| |
Collapse
|
12
|
ZHU H, LI Y, LIU B, YAO W, ZHANG R. Extreme quantile estimation for partial functional linear regression models with heavy-tailed distributions. CAN J STAT 2022; 50:267-286. [PMID: 38239624 PMCID: PMC10795494 DOI: 10.1002/cjs.11653] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Accepted: 03/15/2021] [Indexed: 11/10/2022]
Abstract
In this article, we propose a novel estimator of extreme conditional quantiles in partial functional linear regression models with heavy-tailed distributions. The conventional quantile regression estimators are often unstable at the extreme tails due to data sparsity, especially for heavy-tailed distributions. We first estimate the slope function and the partially linear coefficient using a functional quantile regression based on functional principal component analysis, which is a robust alternative to the ordinary least squares regression. The extreme conditional quantiles are then estimated by using a new extrapolation technique from extreme value theory. We establish the asymptotic normality of the proposed estimator and illustrate its finite sample performance by simulation studies and an empirical analysis of diffusion tensor imaging data from a cognitive disorder study.
Collapse
Affiliation(s)
- Hanbing ZHU
- School of Statistics, Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE, East China Normal University, Shanghai, China
| | - Yehua LI
- Department of Statistics, University of California, Riverside, California, USA
| | - Baisen LIU
- School of Statistics, Dongbei University of Finance and Economics, Dalian, China
| | - Weixin YAO
- Department of Statistics, University of California, Riverside, California, USA
| | - Riquan ZHANG
- School of Statistics, Key Laboratory of Advanced Theory and Application in Statistics and Data Science-MOE, East China Normal University, Shanghai, China
| |
Collapse
|
13
|
|
14
|
|
15
|
Feng S, Zhang M, Tong T. Variable selection for functional linear models with strong heredity constraint. ANN I STAT MATH 2021. [DOI: 10.1007/s10463-021-00798-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
16
|
Li R, Lu W, Zhu Z, Lian H. Optimal prediction of quantile functional linear regression in reproducing kernel Hilbert spaces. J Stat Plan Inference 2021. [DOI: 10.1016/j.jspi.2020.06.010] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
17
|
Xu W, Ding H, Zhang R, Liang H. Estimation and inference in partially functional linear regression with multiple functional covariates. J Stat Plan Inference 2020. [DOI: 10.1016/j.jspi.2020.02.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
18
|
Gholami H, Mohammadifar A, Bui DT, Collins AL. Mapping wind erosion hazard with regression-based machine learning algorithms. Sci Rep 2020; 10:20494. [PMID: 33235269 PMCID: PMC7686346 DOI: 10.1038/s41598-020-77567-0] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2020] [Accepted: 11/10/2020] [Indexed: 11/09/2022] Open
Abstract
Land susceptibility to wind erosion hazard in Isfahan province, Iran, was mapped by testing 16 advanced regression-based machine learning methods: Robust linear regression (RLR), Cforest, Non-convex penalized quantile regression (NCPQR), Neural network with feature extraction (NNFE), Monotone multi-layer perception neural network (MMLPNN), Ridge regression (RR), Boosting generalized linear model (BGLM), Negative binomial generalized linear model (NBGLM), Boosting generalized additive model (BGAM), Spline generalized additive model (SGAM), Spike and slab regression (SSR), Stochastic gradient boosting (SGB), support vector machine (SVM), Relevance vector machine (RVM) and the Cubist and Adaptive network-based fuzzy inference system (ANFIS). Thirteen factors controlling wind erosion were mapped, and multicollinearity among these factors was quantified using the tolerance coefficient (TC) and variance inflation factor (VIF). Model performance was assessed by RMSE, MAE, MBE, and a Taylor diagram using both training and validation datasets. The result showed that five models (MMLPNN, SGAM, Cforest, BGAM and SGB) are capable of delivering a high prediction accuracy for land susceptibility to wind erosion hazard. DEM, precipitation, and vegetation (NDVI) are the most critical factors controlling wind erosion in the study area. Overall, regression-based machine learning models are efficient techniques for mapping land susceptibility to wind erosion hazards.
Collapse
Affiliation(s)
- Hamid Gholami
- Department of Natural Resources Engineering, University of Hormozgan, Bandar-Abbas, Hormozgan, Iran.
| | - Aliakbar Mohammadifar
- Department of Natural Resources Engineering, University of Hormozgan, Bandar-Abbas, Hormozgan, Iran
| | - Dieu Tien Bui
- Institute of Research and Development, Duy Tan University, Da Nang, 550000, Vietnam.
- GIS Group, Department of Business and IT, University of South-Eastern Norway, 3800, Bø i Telemark, Norway.
| | - Adrian L Collins
- Sustainable Agriculture Sciences, Rothamsted/Research, North Wyke, Okehampton, EX20 2SB, Devon, UK
| |
Collapse
|
19
|
Yuan M, Zhang Y. Test for the parametric part in partial functional linear regression based on B-spline. COMMUN STAT-SIMUL C 2019. [DOI: 10.1080/03610918.2019.1667391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Affiliation(s)
- Mingao Yuan
- Department of Statistics, North Dakota State University, Fargo, North Dakota, USA
| | - Yue Zhang
- Department of Mathematics, Indiana University-Purdue University, Indianapolis, Indiana, USA
| |
Collapse
|
20
|
Ferraty F, Kokoszka P, Wang JL, Wu Y. Editorial for the special issue on High-dimensional and functional data analysis. Comput Stat Data Anal 2019. [DOI: 10.1016/j.csda.2018.10.009] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|