1
|
Babar M, Qureshi B, Koubaa A. Investigating the impact of data heterogeneity on the performance of federated learning algorithm using medical imaging. PLoS One 2024; 19:e0302539. [PMID: 38748657 PMCID: PMC11095741 DOI: 10.1371/journal.pone.0302539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Accepted: 04/09/2024] [Indexed: 05/19/2024] Open
Abstract
In recent years, Federated Learning (FL) has gained traction as a privacy-centric approach in medical imaging. This study explores the challenges posed by data heterogeneity on FL algorithms, using the COVIDx CXR-3 dataset as a case study. We contrast the performance of the Federated Averaging (FedAvg) algorithm on non-identically and independently distributed (non-IID) data against identically and independently distributed (IID) data. Our findings reveal a notable performance decline with increased data heterogeneity, emphasizing the need for innovative strategies to enhance FL in diverse environments. This research contributes to the practical implementation of FL, extending beyond theoretical concepts and addressing the nuances in medical imaging applications. This research uncovers the inherent challenges in FL due to data diversity. It sets the stage for future advancements in FL strategies to effectively manage data heterogeneity, especially in sensitive fields like healthcare.
Collapse
Affiliation(s)
- Muhammad Babar
- Robotics and Internet of Things Lab, Prince Sultan University, Riyadh, Saudi Arabia
| | - Basit Qureshi
- College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia
| | - Anis Koubaa
- Robotics and Internet of Things Lab, Prince Sultan University, Riyadh, Saudi Arabia
| |
Collapse
|
2
|
Soltan AAS, Thakur A, Yang J, Chauhan A, D'Cruz LG, Dickson P, Soltan MA, Thickett DR, Eyre DW, Zhu T, Clifton DA. A scalable federated learning solution for secondary care using low-cost microcomputing: privacy-preserving development and evaluation of a COVID-19 screening test in UK hospitals. Lancet Digit Health 2024; 6:e93-e104. [PMID: 38278619 DOI: 10.1016/s2589-7500(23)00226-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 10/17/2023] [Accepted: 10/30/2023] [Indexed: 01/28/2024]
Abstract
BACKGROUND Multicentre training could reduce biases in medical artificial intelligence (AI); however, ethical, legal, and technical considerations can constrain the ability of hospitals to share data. Federated learning enables institutions to participate in algorithm development while retaining custody of their data but uptake in hospitals has been limited, possibly as deployment requires specialist software and technical expertise at each site. We previously developed an artificial intelligence-driven screening test for COVID-19 in emergency departments, known as CURIAL-Lab, which uses vital signs and blood tests that are routinely available within 1 h of a patient's arrival. Here we aimed to federate our COVID-19 screening test by developing an easy-to-use embedded system-which we introduce as full-stack federated learning-to train and evaluate machine learning models across four UK hospital groups without centralising patient data. METHODS We supplied a Raspberry Pi 4 Model B preloaded with our federated learning software pipeline to four National Health Service (NHS) hospital groups in the UK: Oxford University Hospitals NHS Foundation Trust (OUH; through the locally linked research University, University of Oxford), University Hospitals Birmingham NHS Foundation Trust (UHB), Bedfordshire Hospitals NHS Foundation Trust (BH), and Portsmouth Hospitals University NHS Trust (PUH). OUH, PUH, and UHB participated in federated training, training a deep neural network and logistic regressor over 150 rounds to form and calibrate a global model to predict COVID-19 status, using clinical data from patients admitted before the pandemic (COVID-19-negative) and testing positive for COVID-19 during the first wave of the pandemic. We conducted a federated evaluation of the global model for admissions during the second wave of the pandemic at OUH, PUH, and externally at BH. For OUH and PUH, we additionally performed local fine-tuning of the global model using the sites' individual training data, forming a site-tuned model, and evaluated the resultant model for admissions during the second wave of the pandemic. This study included data collected between Dec 1, 2018, and March 1, 2021; the exact date ranges used varied by site. The primary outcome was overall model performance, measured as the area under the receiver operating characteristic curve (AUROC). Removable micro secure digital (microSD) storage was destroyed on study completion. FINDINGS Clinical data from 130 941 patients (1772 COVID-19-positive), routinely collected across three hospital groups (OUH, PUH, and UHB), were included in federated training. The evaluation step included data from 32 986 patients (3549 COVID-19-positive) attending OUH, PUH, or BH during the second wave of the pandemic. Federated training of a global deep neural network classifier improved upon performance of models trained locally in terms of AUROC by a mean of 27·6% (SD 2·2): AUROC increased from 0·574 (95% CI 0·560-0·589) at OUH and 0·622 (0·608-0·637) at PUH using the locally trained models to 0·872 (0·862-0·882) at OUH and 0·876 (0·865-0·886) at PUH using the federated global model. Performance improvement was smaller for a logistic regression model, with a mean increase in AUROC of 13·9% (0·5%). During federated external evaluation at BH, AUROC for the global deep neural network model was 0·917 (0·893-0·942), with 89·7% sensitivity (83·6-93·6) and 76·6% specificity (73·9-79·1). Site-specific tuning of the global model did not significantly improve performance (change in AUROC <0·01). INTERPRETATION We developed an embedded system for federated learning, using microcomputing to optimise for ease of deployment. We deployed full-stack federated learning across four UK hospital groups to develop a COVID-19 screening test without centralising patient data. Federation improved model performance, and the resultant global models were generalisable. Full-stack federated learning could enable hospitals to contribute to AI development at low cost and without specialist technical expertise at each site. FUNDING The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.
Collapse
Affiliation(s)
- Andrew A S Soltan
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK; Department of Oncology, University of Oxford, Oxford, UK; Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK; Big Data Institute, Nuffield Department of Population Health, University of Oxford, Oxford, UK; Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK.
| | - Anshul Thakur
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Jenny Yang
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Anoop Chauhan
- Portsmouth Hospitals University NHS Trust, Portsmouth, UK
| | - Leon G D'Cruz
- Portsmouth Hospitals University NHS Trust, Portsmouth, UK
| | | | - Marina A Soltan
- The Queen Elizabeth Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
| | - David R Thickett
- The Queen Elizabeth Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
| | - David W Eyre
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK; Big Data Institute, Nuffield Department of Population Health, University of Oxford, Oxford, UK; NIHR Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, University of Oxford and Public Health England, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford, UK
| | - Tingting Zhu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - David A Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford, UK; Oxford-Suzhou Centre for Advanced Research, Suzhou, China
| |
Collapse
|
3
|
Sharma S, Guleria K. A comprehensive review on federated learning based models for healthcare applications. Artif Intell Med 2023; 146:102691. [PMID: 38042608 DOI: 10.1016/j.artmed.2023.102691] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 10/22/2023] [Accepted: 10/22/2023] [Indexed: 12/04/2023]
Abstract
A disease is an abnormal condition that negatively impacts the functioning of the human body. Pathology determines the causes behind the disease and identifies its development mechanism and functional consequences. Each disease has different identification methods, including X-ray scans for pneumonia, covid-19, and lung cancer, whereas biopsy and CT-scan can identify the presence of skin cancer and Alzheimer's disease, respectively. Early disease detection leads to effective treatment and avoids abiding complications. Deep learning has provided a vast number of applications in medical sectors resulting in accurate and reliable early disease predictions. These models are utilized in the healthcare industry to provide supplementary assistance to doctors in identifying the presence of diseases. Majorly, these models are trained through secondary data sources since healthcare institutions refrain from sharing patients' private data to ensure confidentiality, which limits the effectiveness of deep learning models due to the requirement of extensive datasets for training to achieve optimal results. Federated learning deals with the data in such a way that it doesn't exploit the privacy of a patient's data. In this work, a wide variety of disease detection models trained through federated learning have been rigorously reviewed. This meta-analysis provides an in-depth review of the federated learning architectures, federated learning types, hyperparameters, dataset utilization details, aggregation techniques, performance measures, and augmentation methods applied in the existing models during the development phase. The review also highlights various open challenges associated with the disease detection models trained through federated learning for future research.
Collapse
Affiliation(s)
- Shagun Sharma
- Chitkara University Institute of Engineering & Technology, Chitkara University, Rajpura 140401, Punjab, India
| | - Kalpna Guleria
- Chitkara University Institute of Engineering & Technology, Chitkara University, Rajpura 140401, Punjab, India.
| |
Collapse
|
4
|
Deng H, Hu J, Sharma R, Mo M, Ren Y. NVAS: A non-interactive verifiable federated learning aggregation scheme for COVID-19 based on game theory. COMPUTER COMMUNICATIONS 2023; 206:1-9. [PMID: 37139177 PMCID: PMC10140058 DOI: 10.1016/j.comcom.2023.04.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 03/21/2023] [Accepted: 04/22/2023] [Indexed: 05/05/2023]
Abstract
The continued spread of COVID-19 seriously endangers the physical and mental health of people in all countries. It is an important method to establish inter agency COVID-19 detection and prevention system based on game theory through wireless communication and artificial intelligence. Federated learning (FL) as a privacy preserving machine learning framework has received extensive attention. From the perspective of game theory, FL can be regarded as a process in which multiple participants play games against each other to maximize their own interests. This requires that the user's data is not leaked during the training process. However, existing studies have proved that the privacy protection capability of FL is insufficient. In addition, the existing way of realizing privacy protection through multiple rounds of communication between participants increases the burden of wireless communication. To this end, this paper considers the security model of FL based on game theory, and proposes our scheme, NVAS, a non-interactive verifiable privacy-preserving FL aggregation scheme in wireless communication environments. The NVAS can protect user privacy during FL training without unnecessary interaction between participants, which can better motivate more participants to join and provide high-quality training data. Furthermore, we designed a concise and efficient verification algorithm to ensure the correctness of model aggregation. Finally, the security and feasibility of the scheme are analyzed.
Collapse
Affiliation(s)
- Haitao Deng
- Engineering Research Center of Digital Forensics of Ministry of Education, School of Computer, Nanjing University of Information Science and Technology, No. 219, Ningliu Road, Nanjing, Jiangsu, 210044, China
| | - Jing Hu
- Department of Rehabilitation Medicine, Children's Hospital of Nanjing Medical University, Nanjing, Jiangsu, 210008, China
| | - Rohit Sharma
- Department of Electronics and Communication Engineering, Faculty of Engineering and Technology, SRM Institute of Science and Technology, Ghaziabad, Uttar Pradesh, India
| | - Mingsen Mo
- Engineering Research Center of Digital Forensics of Ministry of Education, School of Computer, Nanjing University of Information Science and Technology, No. 219, Ningliu Road, Nanjing, Jiangsu, 210044, China
| | - Yongjun Ren
- Engineering Research Center of Digital Forensics of Ministry of Education, School of Computer, Nanjing University of Information Science and Technology, No. 219, Ningliu Road, Nanjing, Jiangsu, 210044, China
| |
Collapse
|
5
|
Hanser T. Federated learning for molecular discovery. Curr Opin Struct Biol 2023; 79:102545. [PMID: 36804704 DOI: 10.1016/j.sbi.2023.102545] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 01/06/2023] [Accepted: 01/13/2023] [Indexed: 02/18/2023]
Abstract
Federated Learning enables machine learning across multiple sources of data and alleviates the risk of leaking private information between partners thereby encouraging knowledge sharing and collaborative modelling. Hence, Federated Learning opens the ways to a new generation of improved models. Domains involving molecular informatics, like Drug Discovery, are progressively adopting Federated Learning; this review describes the main projects and applications of Federated Learning for molecular discovery with a special focus on their benefits and the remaining challenges. All the studies demonstrate a real benefit of Federated Learning, namely the improvement of the performance of models as well as their applicability domain thanks to knowledge aggregation. The selected publications also reveal several remaining challenges to be addressed to fully exploit Federated Learning.
Collapse
Affiliation(s)
- Thierry Hanser
- Lhasa Limited, Granary Wharf House. 2 Canal Wharf. LS11 5PS Leeds United Kingdom.
| |
Collapse
|
6
|
Malik H, Naeem A, Naqvi RA, Loh WK. DMFL_Net: A Federated Learning-Based Framework for the Classification of COVID-19 from Multiple Chest Diseases Using X-rays. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23020743. [PMID: 36679541 PMCID: PMC9864925 DOI: 10.3390/s23020743] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 01/02/2023] [Accepted: 01/03/2023] [Indexed: 05/14/2023]
Abstract
Coronavirus Disease 2019 (COVID-19) is still a threat to global health and safety, and it is anticipated that deep learning (DL) will be the most effective way of detecting COVID-19 and other chest diseases such as lung cancer (LC), tuberculosis (TB), pneumothorax (PneuTh), and pneumonia (Pneu). However, data sharing across hospitals is hampered by patients' right to privacy, leading to unexpected results from deep neural network (DNN) models. Federated learning (FL) is a game-changing concept since it allows clients to train models together without sharing their source data with anybody else. Few studies, however, focus on improving the model's accuracy and stability, whereas most existing FL-based COVID-19 detection techniques aim to maximize secondary objectives such as latency, energy usage, and privacy. In this work, we design a novel model named decision-making-based federated learning network (DMFL_Net) for medical diagnostic image analysis to distinguish COVID-19 from four distinct chest disorders including LC, TB, PneuTh, and Pneu. The DMFL_Net model that has been suggested gathers data from a variety of hospitals, constructs the model using the DenseNet-169, and produces accurate predictions from information that is kept secure and only released to authorized individuals. Extensive experiments were carried out with chest X-rays (CXR), and the performance of the proposed model was compared with two transfer learning (TL) models, i.e., VGG-19 and VGG-16 in terms of accuracy (ACC), precision (PRE), recall (REC), specificity (SPF), and F1-measure. Additionally, the DMFL_Net model is also compared with the default FL configurations. The proposed DMFL_Net + DenseNet-169 model achieves an accuracy of 98.45% and outperforms other approaches in classifying COVID-19 from four chest diseases and successfully protects the privacy of the data among diverse clients.
Collapse
Affiliation(s)
- Hassaan Malik
- Department of Computer Science, University of Management and Technology, Lahore 54000, Pakistan
| | - Ahmad Naeem
- Department of Computer Science, University of Management and Technology, Lahore 54000, Pakistan
| | - Rizwan Ali Naqvi
- Department of Unmanned Vehicle Engineering, Sejong University, Seoul 05006, Republic of Korea
- Correspondence: (R.A.N.); (W.-K.L.)
| | - Woong-Kee Loh
- School of Computing, Gachon University, Seongnam 13120, Republic of Korea
- Correspondence: (R.A.N.); (W.-K.L.)
| |
Collapse
|
7
|
Alvi SBK, Nayyer MZ, Jamal MH, Raza I, de la Torre Diez I, Velasco CLR, Brenosa JM, Ashraf I. A lightweight deep learning approach for COVID-19 detection using X-ray images with edge federation. Digit Health 2023; 9:20552076231203604. [PMID: 37799499 PMCID: PMC10548806 DOI: 10.1177/20552076231203604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 09/08/2023] [Indexed: 10/07/2023] Open
Abstract
Objective This study aims to develop a lightweight convolutional neural network-based edge federated learning architecture for COVID-19 detection using X-ray images, aiming to minimize computational cost, latency, and bandwidth requirements while preserving patient privacy. Method The proposed method uses an edge federated learning architecture to optimize task allocation and execution. Unlike in traditional edge networks where requests from fixed nodes are handled by nearby edge devices or remote clouds, the proposed model uses an intelligent broker within the federation to assess member edge cloudlets' parameters, such as resources and hop count, to make optimal decisions for task offloading. This approach enhances performance and privacy by placing tasks in closer proximity to the user. DenseNet is used for model training, with a depth of 60 and 357,482 parameters. This resource-aware distributed approach optimizes computing resource utilization within the edge-federated learning architecture. Results The experimental results demonstrate significant improvements in various performance metrics. The proposed method reduces training time by 53.1%, optimizes CPU and memory utilization by 17.5% and 33.6%, and maintains accurate COVID-19 detection capabilities without compromising the F1 score, demonstrating the efficiency and effectiveness of the lightweight convolutional neural network-based edge federated learning architecture. Conclusion Existing studies predominantly concentrate on either privacy and accuracy or load balancing and energy optimization, with limited emphasis on training time. The proposed approach offers a comprehensive performance-centric solution that simultaneously addresses privacy, load balancing, and energy optimization while reducing training time, providing a more holistic and balanced solution for optimal system performance.
Collapse
Affiliation(s)
- Sohaib Bin Khalid Alvi
- Department of Computer Science, GIFT University, Gujranwala, Pakistan
- Department of Information Technology, University of the Punjab Gujranwala Campus, Gujranwala, Pakistan
| | | | | | - Imran Raza
- Department of Computer Science, COMSATS University Islamabad, Lahore, Pakistan
| | - Isabel de la Torre Diez
- Department of Signal Theory and Communications and Telematic Engineering. Unviersity of Valladolid, Valladolid, Spain
| | - Carmen Lili Rodriguez Velasco
- Universidad Europea del Atlántico. Isabel Torres 21, Santander, Spain
- Universidad Internacional Iberoamericana Campeche, México
- Fundación Universitaria Internacional de Colombia Bogotá, Colombia
| | - Jose Manuel Brenosa
- Universidad Europea del Atlántico. Isabel Torres 21, Santander, Spain
- Universidad Internacional Iberoamericana Arecibo, Puerto Rico, USA
- Universidade Internacional do Cuanza. Cuito, Bié, Angola
| | - Imran Ashraf
- Department of Information and Communication Engineering, Yeungnam University, Gyeongsan, Korea
| |
Collapse
|