1
|
Samadi ME, Mirzaieazar H, Mitsos A, Schuppert A. Noisecut: a python package for noise-tolerant classification of binary data using prior knowledge integration and max-cut solutions. BMC Bioinformatics 2024; 25:155. [PMID: 38641616 PMCID: PMC11031902 DOI: 10.1186/s12859-024-05769-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Accepted: 04/09/2024] [Indexed: 04/21/2024] Open
Abstract
BACKGROUND Classification of binary data arises naturally in many clinical applications, such as patient risk stratification through ICD codes. One of the key practical challenges in data classification using machine learning is to avoid overfitting. Overfitting in supervised learning primarily occurs when a model learns random variations from noisy labels in training data rather than the underlying patterns. While traditional methods such as regularization and early stopping have demonstrated effectiveness in interpolation tasks, addressing overfitting in the classification of binary data, in which predictions always amount to extrapolation, demands extrapolation-enhanced strategies. One such approach is hybrid mechanistic/data-driven modeling, which integrates prior knowledge on input features into the learning process, enhancing the model's ability to extrapolate. RESULTS We present NoiseCut, a Python package for noise-tolerant classification of binary data by employing a hybrid modeling approach that leverages solutions of defined max-cut problems. In a comparative analysis conducted on synthetically generated binary datasets, NoiseCut exhibits better overfitting prevention compared to the early stopping technique employed by different supervised machine learning algorithms. The noise tolerance of NoiseCut stems from a dropout strategy that leverages prior knowledge of input features and is further enhanced by the integration of max-cut problems into the learning process. CONCLUSIONS NoiseCut is a Python package for the implementation of hybrid modeling for the classification of binary data. It facilitates the integration of mechanistic knowledge on the input features into learning from data in a structured manner and proves to be a valuable classification tool when the available training data is noisy and/or limited in size. This advantage is especially prominent in medical and biomedical applications where data scarcity and noise are common challenges. The codebase, illustrations, and documentation for NoiseCut are accessible for download at https://pypi.org/project/noisecut/ . The implementation detailed in this paper corresponds to the version 0.2.1 release of the software.
Collapse
Affiliation(s)
- Moein E Samadi
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| | - Hedieh Mirzaieazar
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| | - Alexander Mitsos
- Process Systems Engineering (AVT.SVT), RWTH Aachen University, Aachen, Germany
| | - Andreas Schuppert
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany.
| |
Collapse
|
2
|
Samadi ME, Guzman-Maldonado J, Nikulina K, Mirzaieazar H, Sharafutdinov K, Fritsch SJ, Schuppert A. A hybrid modeling framework for generalizable and interpretable predictions of ICU mortality across multiple hospitals. Sci Rep 2024; 14:5725. [PMID: 38459085 PMCID: PMC10923850 DOI: 10.1038/s41598-024-55577-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Accepted: 02/26/2024] [Indexed: 03/10/2024] Open
Abstract
The development of reliable mortality risk stratification models is an active research area in computational healthcare. Mortality risk stratification provides a standard to assist physicians in evaluating a patient's condition or prognosis objectively. Particular interest lies in methods that are transparent to clinical interpretation and that retain predictive power once validated across diverse datasets they were not trained on. This study addresses the challenge of consolidating numerous ICD codes for predictive modeling of ICU mortality, employing a hybrid modeling approach that integrates mechanistic, clinical knowledge with mathematical and machine learning models . A tree-structured network connecting independent modules that carry clinical meaning is implemented for interpretability. Our training strategy utilizes graph-theoretic methods for data analysis, aiming to identify the functions of individual black-box modules within the tree-structured network by harnessing solutions from specific max-cut problems. The trained model is then validated on external datasets from different hospitals, demonstrating successful generalization capabilities, particularly in binary-feature datasets where label assessment involves extrapolation.
Collapse
Affiliation(s)
- Moein E Samadi
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany.
| | | | - Kateryna Nikulina
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| | - Hedieh Mirzaieazar
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| | | | - Sebastian Johannes Fritsch
- Department of Intensive Care Medicine, University Hospital RWTH Aachen, Aachen, Germany
- Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany
- Center for Advanced Simulation and Analytics (CASA), Forschungszentrum Jülich, Jülich, Germany
| | - Andreas Schuppert
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
3
|
E. Samadi M, Kiefer S, Fritsch SJ, Bickenbach J, Schuppert A. A training strategy for hybrid models to break the curse of dimensionality. PLoS One 2022; 17:e0274569. [PMID: 36107916 PMCID: PMC9477345 DOI: 10.1371/journal.pone.0274569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Accepted: 08/30/2022] [Indexed: 11/18/2022] Open
Abstract
Mechanistic/data-driven hybrid modeling is a key approach when the mechanistic details of the processes at hand are not sufficiently well understood, but also inferring a model purely from data is too complex. By the integration of first principles into a data-driven approach, hybrid modeling promises a feasible data demand alongside extrapolation. In this work, we introduce a learning strategy for tree-structured hybrid models to perform a binary classification task. Given a set of binary labeled data, the challenge is to use them to develop a model that accurately assesses labels of new unlabeled data. Our strategy employs graph-theoretic methods to analyze the data and deduce a function that maps input features to output labels. Our focus here is on data sets represented by binary features in which the label assessment of unlabeled data points is always extrapolation. Our strategy shows the existence of small sets of data points within given binary data for which knowing the labels allows for extrapolation to the entire valid input space. An implementation of our strategy yields a notable reduction of training-data demand in a binary classification task compared with different supervised machine learning algorithms. As an application, we have fitted a tree-structured hybrid model to the vital status of a cohort of COVID-19 patients requiring intensive-care unit treatment and mechanical ventilation. Our learning strategy yields the existence of patient cohorts for whom knowing the vital status enables extrapolation to the entire valid input space of the developed hybrid model.
Collapse
Affiliation(s)
- Moein E. Samadi
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
- Joint Research Center for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
- * E-mail:
| | - Sandra Kiefer
- Max Planck Institute for Software Systems, Saarland Informatics Campus, Saarbrücken, Germany
| | - Sebastian Johaness Fritsch
- Department of Intensive Care Medicine, University Hospital RWTH Aachen, Aachen, Germany
- Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany
| | - Johannes Bickenbach
- Department of Intensive Care Medicine, University Hospital RWTH Aachen, Aachen, Germany
| | - Andreas Schuppert
- Institute for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
- Joint Research Center for Computational Biomedicine, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
4
|
Shi Y, Lau AKH, Ng E, Ho HC, Bilal M. A Multiscale Land Use Regression Approach for Estimating Intraurban Spatial Variability of PM 2.5 Concentration by Integrating Multisource Datasets. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 19:321. [PMID: 35010580 PMCID: PMC8751171 DOI: 10.3390/ijerph19010321] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Revised: 12/24/2021] [Accepted: 12/28/2021] [Indexed: 06/14/2023]
Abstract
Poor air quality has been a major urban environmental issue in large high-density cities all over the world, and particularly in Asia, where the multiscale complex of pollution dispersal creates a high-level spatial variability of exposure level. Investigating such multiscale complexity and fine-scale spatial variability is challenging. In this study, we aim to tackle the challenge by focusing on PM2.5 (particulate matter with an aerodynamic diameter less than 2.5 µm,) which is one of the most concerning air pollutants. We use the widely adopted land use regression (LUR) modeling technique as the fundamental method to integrate air quality data, satellite data, meteorological data, and spatial data from multiple sources. Unlike most LUR and Aerosol Optical Depth (AOD)-PM2.5 studies, the modeling process was conducted independently at city and neighborhood scales. Correspondingly, predictor variables at the two scales were treated separately. At the city scale, the model developed in the present study obtains better prediction performance in the AOD-PM2.5 relationship when compared with previous studies (R2¯ from 0.72 to 0.80). At the neighborhood scale, point-based building morphological indices and road network centrality metrics were found to be fit-for-purpose indicators of PM2.5 spatial estimation. The resultant PM2.5 map was produced by combining the models from the two scales, which offers a geospatial estimation of small-scale intraurban variability.
Collapse
Affiliation(s)
- Yuan Shi
- Institute of Future Cities (IOFC), The Chinese University of Hong Kong, Hong Kong, China
| | - Alexis Kai-Hon Lau
- Division of Environment and Sustainability, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China;
- Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
- Institute for the Environment, The Hong Kong University of Science & Technology, Clear Water Bay, Kowloon, Hong Kong, China
| | - Edward Ng
- Institute of Future Cities (IOFC), The Chinese University of Hong Kong, Hong Kong, China
- School of Architecture, The Chinese University of Hong Kong, Hong Kong, China;
- Institute of Environment, Energy and Sustainability (IEES), The Chinese University of Hong Kong, Hong Kong, China
| | - Hung-Chak Ho
- Department of Urban Planning and Design, The University of Hong Kong, Hong Kong, China;
| | - Muhammad Bilal
- Lab of Environmental Remote Sensing (LERS), School of Marine Sciences, Nanjing University of Information Science and Technology, Nanjing 210044, China;
| |
Collapse
|
6
|
Soranno PA, Cheruvelil KS, Liu B, Wang Q, Tan PN, Zhou J, King KBS, McCullough IM, Stachelek J, Bartley M, Filstrup CT, Hanks EM, Lapierre JF, Lottig NR, Schliep EM, Wagner T, Webster KE. Ecological prediction at macroscales using big data: Does sampling design matter? ECOLOGICAL APPLICATIONS : A PUBLICATION OF THE ECOLOGICAL SOCIETY OF AMERICA 2020; 30:e02123. [PMID: 32160362 DOI: 10.1002/eap.2123] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/09/2019] [Revised: 12/13/2019] [Accepted: 01/06/2020] [Indexed: 06/10/2023]
Abstract
Although ecosystems respond to global change at regional to continental scales (i.e., macroscales), model predictions of ecosystem responses often rely on data from targeted monitoring of a small proportion of sampled ecosystems within a particular geographic area. In this study, we examined how the sampling strategy used to collect data for such models influences predictive performance. We subsampled a large and spatially extensive data set to investigate how macroscale sampling strategy affects prediction of ecosystem characteristics in 6,784 lakes across a 1.8-million-km2 area. We estimated model predictive performance for different subsets of the data set to mimic three common sampling strategies for collecting observations of ecosystem characteristics: random sampling design, stratified random sampling design, and targeted sampling. We found that sampling strategy influenced model predictive performance such that (1) stratified random sampling designs did not improve predictive performance compared to simple random sampling designs and (2) although one of the scenarios that mimicked targeted (non-random) sampling had the poorest performing predictive models, the other targeted sampling scenarios resulted in models with similar predictive performance to that of the random sampling scenarios. Our results suggest that although potential biases in data sets from some forms of targeted sampling may limit predictive performance, compiling existing spatially extensive data sets can result in models with good predictive performance that may inform a wide range of science questions and policy goals related to global change.
Collapse
Affiliation(s)
- Patricia A Soranno
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
| | - Kendra Spence Cheruvelil
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
- Lyman Briggs College, Michigan State University, 919 East Shaw Lane, East Lansing, Michigan, 48825, USA
| | - Boyang Liu
- Department of Computer Science and Engineering, Michigan State University, 428 South Shaw Lane, East Lansing, Michigan, 48824, USA
| | - Qi Wang
- Department of Computer Science and Engineering, Michigan State University, 428 South Shaw Lane, East Lansing, Michigan, 48824, USA
| | - Pang-Ning Tan
- Department of Computer Science and Engineering, Michigan State University, 428 South Shaw Lane, East Lansing, Michigan, 48824, USA
| | - Jiayu Zhou
- Department of Computer Science and Engineering, Michigan State University, 428 South Shaw Lane, East Lansing, Michigan, 48824, USA
| | - Katelyn B S King
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
| | - Ian M McCullough
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
| | - Joseph Stachelek
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
| | - Meridith Bartley
- Department of Statistics, The Pennsylvania State University, 324 Thomas Building, University Park, Pennsylvania, 16802, USA
| | - Christopher T Filstrup
- Natural Resources Research Institute, University of Minnesota Duluth, 5013 Miller Trunk Highway, Duluth, Minnesota, 55811, USA
| | - Ephraim M Hanks
- Department of Statistics, The Pennsylvania State University, 324 Thomas Building, University Park, Pennsylvania, 16802, USA
| | - Jean-François Lapierre
- Sciences Biologiques, Universite de Montreal, Pavillon Marie-Victorin, CP 6128, succursale Centre-Ville, Montreal, Quebec, H3C 3J7, Canada
| | - Noah R Lottig
- Center for Limnology Trout Lake Station, University of Wisconsin Madison, Boulder Junction, Wisconsin, 54512, USA
| | - Erin M Schliep
- Department of Statistics, University of Missouri, 146 Middlebush Hall, Columbia, Missouri, 65211, USA
| | - Tyler Wagner
- U.S. Geological Survey, Pennsylvania Cooperative Fish and Wildlife Research Unit, Pennsylvania State University, Forest Resources Building, University Park, Pennsylvania, 16802, USA
| | - Katherine E Webster
- Department of Fisheries and Wildlife, Michigan State University, 480 Wilson Road, East Lansing, Michigan, 48824, USA
| |
Collapse
|