1
|
Ostojic D, Lalousis PA, Donohoe G, Morris DW. The challenges of using machine learning models in psychiatric research and clinical practice. Eur Neuropsychopharmacol 2024; 88:53-65. [PMID: 39232341 DOI: 10.1016/j.euroneuro.2024.08.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 08/06/2024] [Accepted: 08/12/2024] [Indexed: 09/06/2024]
Abstract
To understand the complex nature of heterogeneous psychiatric disorders, scientists and clinicians are required to employ a wide range of clinical, endophenotypic, neuroimaging, genomic, and environmental data to understand the biological mechanisms of psychiatric illness before this knowledge is applied into clinical setting. Machine learning (ML) is an automated process that can detect patterns from large multidimensional datasets and can supersede conventional statistical methods as it can detect both linear and non-linear relationships. Due to this advantage, ML has potential to enhance our understanding, improve diagnosis, prognosis and treatment of psychiatric disorders. The current review provides an in-depth examination of, and offers practical guidance for, the challenges encountered in the application of ML models in psychiatric research and clinical practice. These challenges include the curse of dimensionality, data quality, the 'black box' problem, hyperparameter tuning, external validation, class imbalance, and data representativeness. These challenges are particularly critical in the context of psychiatry as it is expected that researchers will encounter them during the stages of ML model development and deployment. We detail practical solutions and best practices to effectively mitigate the outlined challenges. These recommendations have the potential to improve reliability and interpretability of ML models in psychiatry.
Collapse
Affiliation(s)
- Dijana Ostojic
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Paris Alexandros Lalousis
- Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, United Kingdom; Section for Precision Psychiatry, Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University Munich, Munich, Germany
| | - Gary Donohoe
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland
| | - Derek W Morris
- School of Biological and Chemical Sciences and School of Psychology, Centre for Neuroimaging, Cognition and Genomics (NICOG), University of Galway, Ireland.
| |
Collapse
|
2
|
Nathvani R, D V, Clark SN, Alli AS, Muller E, Coste H, Bennett JE, Nimo J, Moses JB, Baah S, Hughes A, Suel E, Metzler AB, Rashid T, Brauer M, Baumgartner J, Owusu G, Agyei-Mensah S, Arku RE, Ezzati M. Beyond here and now: Evaluating pollution estimation across space and time from street view images with deep learning. THE SCIENCE OF THE TOTAL ENVIRONMENT 2023; 903:166168. [PMID: 37586538 PMCID: PMC7615099 DOI: 10.1016/j.scitotenv.2023.166168] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 08/07/2023] [Accepted: 08/07/2023] [Indexed: 08/18/2023]
Abstract
Advances in computer vision, driven by deep learning, allows for the inference of environmental pollution and its potential sources from images. The spatial and temporal generalisability of image-based pollution models is crucial in their real-world application, but is currently understudied, particularly in low-income countries where infrastructure for measuring the complex patterns of pollution is limited and modelling may therefore provide the most utility. We employed convolutional neural networks (CNNs) for two complementary classification models, in both an end-to-end approach and as an interpretable feature extractor (object detection), to estimate spatially and temporally resolved fine particulate matter (PM2.5) and noise levels in Accra, Ghana. Data used for training the models were from a unique dataset of over 1.6 million images collected over 15 months at 145 representative locations across the city, paired with air and noise measurements. Both end-to-end CNN and object-based approaches surpassed null model benchmarks for predicting PM2.5 and noise at single locations, but performance deteriorated when applied to other locations. Model accuracy diminished when tested on images from locations unseen during training, but improved by sampling a greater number of locations during model training, even if the total quantity of data was reduced. The end-to-end models used characteristics of images associated with atmospheric visibility for predicting PM2.5, and specific objects such as vehicles and people for noise. The results demonstrate the potential and challenges of image-based, spatiotemporal air pollution and noise estimation, and that robust, environmental modelling with images requires integration with traditional sensor networks.
Collapse
Affiliation(s)
- Ricky Nathvani
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK.
| | - Vishwanath D
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - Sierra N Clark
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - Abosede S Alli
- Department of Environmental Health Sciences, School of Public Health and Health Sciences, University of Massachusetts, Amherst, USA
| | - Emily Muller
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - Henri Coste
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - James E Bennett
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - James Nimo
- Department of Physics, University of Ghana, Accra, Ghana
| | | | - Solomon Baah
- Department of Physics, University of Ghana, Accra, Ghana
| | - Allison Hughes
- Department of Physics, University of Ghana, Accra, Ghana
| | - Esra Suel
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK; Centre for Advanced Spatial Analysis, University College London, London, UK
| | - Antje Barbara Metzler
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - Theo Rashid
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK
| | - Michael Brauer
- School of Population and Public Health, University of British Columbia, Vancouver, Canada
| | - Jill Baumgartner
- Institute for Health and Social Policy, McGill University, Montreal, Canada; Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada
| | - George Owusu
- Institute of Statistical, Social & Economic Research, University of Ghana, Accra, Ghana
| | - Samuel Agyei-Mensah
- Department of Geography and Resource Development, University of Ghana, Accra, Ghana
| | - Raphael E Arku
- Department of Environmental Health Sciences, School of Public Health and Health Sciences, University of Massachusetts, Amherst, USA
| | - Majid Ezzati
- Department of Epidemiology and Biostatistics, School of Public Health, Imperial College London, London, UK; MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK; Regional Institute for Population Studies, University of Ghana, Accra, Ghana
| |
Collapse
|
3
|
Yego NKK, Nkurunziza J, Kasozi J. Predicting health insurance uptake in Kenya using Random Forest: An analysis of socio-economic and demographic factors. PLoS One 2023; 18:e0294166. [PMID: 38032867 PMCID: PMC10688734 DOI: 10.1371/journal.pone.0294166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/27/2023] [Indexed: 12/02/2023] Open
Abstract
Universal Health Coverage (UHC) is a global objective aimed at providing equitable access to essential and cost-effective healthcare services, irrespective of individuals' financial circumstances. Despite efforts to promote UHC through health insurance programs, the uptake in Kenya remains low. This study aimed to explore the factors influencing health insurance uptake and offer insights for effective policy development and outreach programs. The study utilized machine learning techniques on data from the 2021 FinAccess Survey. Among the models examined, the Random Forest model demonstrated the highest performance with notable metrics, including a high Kappa score of 0.9273, Recall score of 0.9640, F1 score of 0.9636, and Accuracy of 0.9636. The study identified several crucial predictors of health insurance uptake, ranked in ascending order of importance by the optimal model, including poverty vulnerability, social security usage, income, education, and marital status. The results suggest that affordability is a significant barrier to health insurance uptake. The study highlights the need to address affordability challenges and implement targeted interventions to improve health insurance uptake in Kenya, thereby advancing progress towards achieving Universal Health Coverage (UHC) and ensuring universal access to quality healthcare services.
Collapse
Affiliation(s)
- Nelson Kimeli Kemboi Yego
- African Center of Excellence in Data Science, University of Rwanda, Kigali, Rwanda
- Department of Mathematics and Computer Science, Moi University, Kenya
| | - Joseph Nkurunziza
- African Center of Excellence in Data Science, University of Rwanda, Kigali, Rwanda
- School of Economics, University of Rwanda, Kigali, Rwanda
| | - Juma Kasozi
- African Center of Excellence in Data Science, University of Rwanda, Kigali, Rwanda
- Department of Mathematics, Makerere University, Kampala, Uganda
| |
Collapse
|
4
|
Sperrin M, Riley RD, Collins GS, Martin GP. Targeted validation: validating clinical prediction models in their intended population and setting. Diagn Progn Res 2022; 6:24. [PMID: 36550534 PMCID: PMC9773429 DOI: 10.1186/s41512-022-00136-8] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 11/14/2022] [Indexed: 12/24/2022] Open
Abstract
Clinical prediction models must be appropriately validated before they can be used. While validation studies are sometimes carefully designed to match an intended population/setting of the model, it is common for validation studies to take place with arbitrary datasets, chosen for convenience rather than relevance. We call estimating how well a model performs within the intended population/setting "targeted validation". Use of this term sharpens the focus on the intended use of a model, which may increase the applicability of developed models, avoid misleading conclusions, and reduce research waste. It also exposes that external validation may not be required when the intended population for the model matches the population used to develop the model; here, a robust internal validation may be sufficient, especially if the development dataset was large.
Collapse
Affiliation(s)
- Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK.
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| |
Collapse
|
5
|
Bento N, Rebelo J, Barandas M, Carreiro AV, Campagner A, Cabitza F, Gamboa H. Comparing Handcrafted Features and Deep Neural Representations for Domain Generalization in Human Activity Recognition. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22197324. [PMID: 36236427 PMCID: PMC9572241 DOI: 10.3390/s22197324] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 09/21/2022] [Accepted: 09/23/2022] [Indexed: 06/02/2023]
Abstract
Human Activity Recognition (HAR) has been studied extensively, yet current approaches are not capable of generalizing across different domains (i.e., subjects, devices, or datasets) with acceptable performance. This lack of generalization hinders the applicability of these models in real-world environments. As deep neural networks are becoming increasingly popular in recent work, there is a need for an explicit comparison between handcrafted and deep representations in Out-of-Distribution (OOD) settings. This paper compares both approaches in multiple domains using homogenized public datasets. First, we compare several metrics to validate three different OOD settings. In our main experiments, we then verify that even though deep learning initially outperforms models with handcrafted features, the situation is reversed as the distance from the training distribution increases. These findings support the hypothesis that handcrafted features may generalize better across specific domains.
Collapse
Affiliation(s)
- Nuno Bento
- Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
| | - Joana Rebelo
- Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
| | - Marília Barandas
- Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
- Laboratório de Instrumentação, Engenharia Biomédica e Física da Radiação (LIBPhys–UNL), Departamento de Física, Faculdade de Ciências e Tecnologia (FCT), Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
| | - André V. Carreiro
- Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
| | - Andrea Campagner
- Dipartimento di Informatica, Sistemistica e Comunicazione, Università degli Studi di Milano-Bicocca, 20126 Milan, Italy
| | - Federico Cabitza
- Dipartimento di Informatica, Sistemistica e Comunicazione, Università degli Studi di Milano-Bicocca, 20126 Milan, Italy
- IRCCS Istituto Ortopedico Galeazzi, 20161 Milan, Italy
| | - Hugo Gamboa
- Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
- Laboratório de Instrumentação, Engenharia Biomédica e Física da Radiação (LIBPhys–UNL), Departamento de Física, Faculdade de Ciências e Tecnologia (FCT), Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
| |
Collapse
|
6
|
Elhefnawy M, Ragab A, Ouali MS. Polygon generation and video-to-video translation for time-series prediction. JOURNAL OF INTELLIGENT MANUFACTURING 2022; 34:261-279. [PMID: 36618340 PMCID: PMC9813064 DOI: 10.1007/s10845-022-02003-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Accepted: 07/29/2022] [Indexed: 06/17/2023]
Abstract
This paper proposes an innovative method for time-series prediction in energy-intensive industrial systems characterized by highly dynamic non-linear operations. The proposed method can capture the true distributions of the inputs and outputs of such systems and map these distributions using polygon generation and video-to-video translation techniques. More specifically, the time-series data are represented as polygon streams (videos), then the video-to-video translation is used to transform the input polygon streams into the output ones. This transformation is tuned based on a model trustworthiness metric for optimal video synthesis. Finally, an image processing procedure is used for mapping the output polygon streams back to time-series outputs. The proposed method is based on cycle-consistent generative adversarial networks as an unsupervised approach. This does not need the heavy involvement of the human expert who devotes much effort to labeling the complex industrial data. The performance of the proposed method was validated successfully using a challenging industrial dataset collected from a complex heat exchanger network in a Canadian pulp mill. The results obtained using the proposed method demonstrate better performance than other comparable time-series prediction models. This allows process operators to accurately monitor process key performance indicators (KPIs) and to achieve a more energy-efficient operation.
Collapse
Affiliation(s)
- Mohamed Elhefnawy
- Department of Mathematics and Industrial Engineering, Polytechnique Montréal, 2500 Chemin de Polytechnique, Montréal, QC H3T 1J4 Canada
- CanmetENERGY, 1615 Lionel-Boulet Blvd., P.O. Box 4800, Varennes, QC J3X 1P7 Canada
| | - Ahmed Ragab
- Department of Mathematics and Industrial Engineering, Polytechnique Montréal, 2500 Chemin de Polytechnique, Montréal, QC H3T 1J4 Canada
- CanmetENERGY, 1615 Lionel-Boulet Blvd., P.O. Box 4800, Varennes, QC J3X 1P7 Canada
- Faculty of Electronic Engineering, Menoufia University, Menouf, 32952 Egypt
| | - Mohamed-Salah Ouali
- Department of Mathematics and Industrial Engineering, Polytechnique Montréal, 2500 Chemin de Polytechnique, Montréal, QC H3T 1J4 Canada
| |
Collapse
|
7
|
Keskes N, Fakhfakh S, Kanoun O, Derbel N. Representativeness consideration in the selection of classification algorithms for the ECG signal quality assessment. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103686] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
8
|
Zhu J, Li H, Jing ZZ, Zheng W, Luo YR, Chen SX, Guo F. Robust host source tracking building on the divergent and non-stochastic assembly of gut microbiomes in wild and farmed large yellow croaker. MICROBIOME 2022; 10:18. [PMID: 35081990 PMCID: PMC8790850 DOI: 10.1186/s40168-021-01214-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Accepted: 12/12/2021] [Indexed: 05/08/2023]
Abstract
BACKGROUND Given the lack of genetic background, the source tracking unknown individuals of fish species with both farmed and wild populations often cannot be robustly achieved. The gut microbiome, which is shaped by both deterministic and stochastic processes, can serve as a molecular marker of fish host source tracking, particularly as an alternative to the yet-to-be-established host genetic marker. A candidate for testing the feasibility is the large yellow croaker, Larimichthys crocea, which is carnivorous and ranks the top mariculture fish in China. Wild resource of this fish was depleted decades ago and might have potential problematic estimation because of escaping of farmed individuals. RESULTS The rectums of wild (n = 212) and farmed (n = 79) croakers from multiple batches were collected for the profiling of their gut bacterial communities. The farmed individuals had a higher alpha diversity and lower bacterial load than the wild individuals. The gut microbiota of the two sources exhibited divergence and high inter-batch variation, as featured by the dominance of Psychrobacter spp. in the wild group. Predicted functional capacity of the gut microbiome and representative isolates showed differences in terms of host source. This difference can be linked to the potential diet divergence between farmed and wild fishes. The non-stochastic distribution pattern of the core gut microbiota of the wild and farmed individuals supports the feasibility of microbiota-based host source tracking via the machine learning algorithm. A random forest classifier based on the divergence and non-stochastic assembly of the gut microbiome was robust in terms of host source tracking the individuals from all batches of croaker, including a newly introduced batch. CONCLUSIONS Our study revealed the divergence of gut microbiota and related functional profiles between wild and farmed croakers. For the first time, with representative datasets and non-stochastic patterns, we have verified that gut microbiota can be robustly applied to the tracking of host source even in carnivorous fish. Video abstract.
Collapse
Affiliation(s)
- Jun Zhu
- State Key Laboratory of Cellular Stress Biology, School of Life Sciences, Xiamen University, Xiamen, China
- Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai, China
| | - Hao Li
- State Key Laboratory of Cellular Stress Biology, School of Life Sciences, Xiamen University, Xiamen, China
| | - Ze Zhou Jing
- State-Province Joint Engineering Laboratory of Marine Bioproducts and Technology, College of Ocean and Earth Sciences, Xiamen University, Xiamen, China
| | - Wei Zheng
- State Key Laboratory of Cellular Stress Biology, School of Life Sciences, Xiamen University, Xiamen, China
| | - Yuan Rong Luo
- Key Laboratory of the Ministry of Education for Coastal and Wetland Ecosystems, Xiamen University, Xiamen, China
| | - Shi Xi Chen
- State-Province Joint Engineering Laboratory of Marine Bioproducts and Technology, College of Ocean and Earth Sciences, Xiamen University, Xiamen, China.
| | - Feng Guo
- State Key Laboratory of Cellular Stress Biology, School of Life Sciences, Xiamen University, Xiamen, China.
- Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai, China.
| |
Collapse
|
9
|
Cabitza F, Campagner A, Soares F, García de Guadiana-Romualdo L, Challa F, Sulejmani A, Seghezzi M, Carobene A. The importance of being external. methodological insights for the external validation of machine learning models in medicine. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 208:106288. [PMID: 34352688 DOI: 10.1016/j.cmpb.2021.106288] [Citation(s) in RCA: 70] [Impact Index Per Article: 23.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 07/09/2021] [Indexed: 06/13/2023]
Abstract
UNLABELLED Background and Objective Medical machine learning (ML) models tend to perform better on data from the same cohort than on new data, often due to overfitting, or co-variate shifts. For these reasons, external validation (EV) is a necessary practice in the evaluation of medical ML. However, there is still a gap in the literature on how to interpret EV results and hence assess the robustness of ML models. METHODS We fill this gap by proposing a meta-validation method, to assess the soundness of EV procedures. In doing so, we complement the usual way to assess EV by considering both dataset cardinality, and the similarity of the EV dataset with respect to the training set. We then investigate how the notions of cardinality and similarity can be used to inform on the reliability of a validation procedure, by integrating them into two summative data visualizations. RESULTS We illustrate our methodology by applying it to the validation of a state-of-the-art COVID-19 diagnostic model on 8 EV sets, collected across 3 different continents. The model performance was moderately impacted by data similarity (Pearson ρ = 0.38, p< 0.001). In the EV, the validated model reported good AUC (average: 0.84), acceptable calibration (average: 0.17) and utility (average: 0.50). The validation datasets were adequate in terms of dataset cardinality and similarity, thus suggesting the soundness of the results. We also provide a qualitative guideline to evaluate the reliability of validation procedures, and we discuss the importance of proper external validation in light of the obtained results. CONCLUSIONS In this paper, we propose a novel, lean methodology to: 1) study how the similarity between training and validation sets impacts the generalizability of a ML model; 2) assess the soundness of EV evaluations along three complementary performance dimensions: discrimination, utility and calibration; 3) draw conclusions on the robustness of the model under validation. We applied this methodology to a state-of-the-art model for the diagnosis of COVID-19 from routine blood tests, and showed how to interpret the results in light of the presented framework.
Collapse
Affiliation(s)
- Federico Cabitza
- University of Milano-Bicocca, Viale Sarca 336, Milano, 20126, Italy.
| | - Andrea Campagner
- University of Milano-Bicocca, Viale Sarca 336, Milano, 20126, Italy
| | - Felipe Soares
- Department of Industrial Engineering - Universidade Federal do Rio Grande do Sul. Porto Alegre, Brazil
| | | | - Feyissa Challa
- National Reference Laboratory for Clinical Chemistry, Ethiopian Public Health Institute, Addis Ababa, Ethiopia
| | - Adela Sulejmani
- Laboratorio di chimica clinica, Ospedale di Desio e Monza, ASST-Monza, Dipartimento di medicina e chirurgia, Universit di Milano-Bicocca, Monza, Italy
| | - Michela Seghezzi
- Laboratorio di chimica clinica, Ospedale Papa Giovanni XXIII, Bergamo, Italy
| | - Anna Carobene
- Laboratory Medicine, IRCCS San Raffaele Scientific Institute, Milan, Italy
| |
Collapse
|
10
|
Cabitza F, Campagner A. The need to separate the wheat from the chaff in medical informatics: Introducing a comprehensive checklist for the (self)-assessment of medical AI studies. Int J Med Inform 2021; 153:104510. [PMID: 34108105 DOI: 10.1016/j.ijmedinf.2021.104510] [Citation(s) in RCA: 116] [Impact Index Per Article: 38.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Revised: 05/26/2021] [Accepted: 05/27/2021] [Indexed: 12/23/2022]
Abstract
This editorial aims to contribute to the current debate about the quality of studies that apply machine learning (ML) methodologies to medical data to extract value from them and provide clinicians with viable and useful tools supporting everyday care practices. We propose a practical checklist to help authors to self assess the quality of their contribution and to help reviewers to recognize and appreciate high-quality medical ML studies by distinguishing them from the mere application of ML techniques to medical data.
Collapse
Affiliation(s)
- Federico Cabitza
- DISCo, University of Milano-Bicocca, viale Sarca 336, Milano 20126, Italy.
| | - Andrea Campagner
- DISCo, University of Milano-Bicocca, viale Sarca 336, Milano 20126, Italy
| |
Collapse
|