1
|
Veras Magalhães G, L. de S. Santos R, H. S. Vogado L, Cardoso de Paiva A, de Alcântara dos Santos Neto P. XRaySwinGen: Automatic medical reporting for X-ray exams with multimodal model. Heliyon 2024; 10:e27516. [PMID: 38560155 PMCID: PMC10979158 DOI: 10.1016/j.heliyon.2024.e27516] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 02/29/2024] [Accepted: 03/01/2024] [Indexed: 04/04/2024] Open
Abstract
The importance of radiology in modern medicine is acknowledged for its non-invasive diagnostic capabilities, yet the manual formulation of unstructured medical reports poses time constraints and error risks. This study addresses the common limitation of Artificial Intelligence applications in medical image captioning, which typically focus on classification problems, lacking detailed information about the patient's condition. Despite advancements in AI-generated medical reports that incorporate descriptive details from X-ray images, which are essential for comprehensive reports, the challenge persists. The proposed solution involves a multimodal model utilizing Computer Vision for image representation and Natural Language Processing for textual report generation. A notable contribution is the innovative use of the Swin Transformer as the image encoder, enabling hierarchical mapping and enhanced model perception without a surge in parameters or computational costs. The model incorporates GPT-2 as the textual decoder, integrating cross-attention layers and bilingual training with datasets in Portuguese PT-BR and English. Promising results are noted in the proposed database with ROUGE-L 0.748, METEOR 0.741, and NIH CHEST X-ray with ROUGE-L 0.404 and METEOR 0.393.
Collapse
Affiliation(s)
| | | | - Luis H. S. Vogado
- Departamento de Computação, Universidade Federal do Piauí, Teresina, Brazil
| | | | | |
Collapse
|
2
|
Guo K, Cheng J, Li K, Wang L, Lv Y, Cao D. Diagnosis and detection of pneumonia using weak-label based on X-ray images: a multi-center study. BMC Med Imaging 2023; 23:209. [PMID: 38087255 PMCID: PMC10717871 DOI: 10.1186/s12880-023-01174-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Accepted: 12/05/2023] [Indexed: 12/18/2023] Open
Abstract
PURPOSE Development and assessment the deep learning weakly supervised algorithm for the classification and detection pneumonia via X-ray. METHODS This retrospective study analyzed two publicly available dataset that contain X-ray images of pneumonia cases and normal cases. The first dataset from Guangzhou Women and Children's Medical Center. It contains a total of 5,856 X-ray images, which are divided into training, validation, and test sets with 8:1:1 ratio for algorithm training and testing. The deep learning algorithm ResNet34 was employed to build diagnostic model. And the second public dataset were collated by researchers from Qatar University and the University of Dhaka along with collaborators from Pakistan and Malaysia and some medical doctors. A total of 1,300 images of COVID-19 positive cases, 1,300 normal images and 1,300 images of viral pneumonia for external validation. Class activation map (CAM) were used to location the pneumonia lesions. RESULTS The ResNet34 model for pneumonia detection achieved an AUC of 0.9949 [0.9910-0.9981] (with an accuracy of 98.29% a sensitivity of 99.29% and a specificity of 95.57%) in the test dataset. And for external validation dataset, the model obtained an AUC of 0.9835[0.9806-0.9864] (with an accuracy of 94.62%, a sensitivity of 92.35% and a specificity of 99.15%). Moreover, the CAM can accurately locate the pneumonia area. CONCLUSION The deep learning algorithm can accurately detect pneumonia and locate the pneumonia area based on weak supervision information, which can provide potential value for helping radiologists to improve their accuracy of detection pneumonia patients through X-ray images.
Collapse
Affiliation(s)
- Kairou Guo
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China
| | - Jiangbo Cheng
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China
| | - Kaiyuan Li
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China
| | - Lanhui Wang
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China
| | - Yadong Lv
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China
| | - Desen Cao
- Department of Biomedical Engineering, Chinese PLA General Hospital, Beijing, 100853, P.R. China.
| |
Collapse
|
3
|
Khan MA, Khan A, Alhaisoni M, Alqahtani A, Alsubai S, Alharbi M, Malik NA, Damaševičius R. Multimodal brain tumor detection and classification using deep saliency map and improved dragonfly optimization algorithm. INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOLOGY 2023; 33:572-587. [DOI: 10.1002/ima.22831] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Accepted: 11/18/2022] [Indexed: 08/25/2024]
Abstract
AbstractIn the last decade, there has been a significant increase in medical cases involving brain tumors. Brain tumor is the tenth most common type of tumor, affecting millions of people. However, if it is detected early, the cure rate can increase. Computer vision researchers are working to develop sophisticated techniques for detecting and classifying brain tumors. MRI scans are primarily used for tumor analysis. We proposed an automated system for brain tumor detection and classification using a saliency map and deep learning feature optimization in this paper. The proposed framework was implemented in stages. In the initial phase of the proposed framework, a fusion‐based contrast enhancement technique is proposed. In the following phase, a tumor segmentation technique based on saliency maps is proposed, which is then mapped on original images based on active contour. Following that, a pre‐trained CNN model named EfficientNetB0 is fine‐tuned and trained in two ways: on enhanced images and on tumor localization images. Deep transfer learning is used to train both models, and features are extracted from the average pooling layer. The deep learning features are then fused using an improved fusion approach known as Entropy Serial Fusion. The best features are chosen in the final step using an improved dragonfly optimization algorithm. Finally, the best features are classified using an extreme learning machine (ELM). The experimental process is conducted on three publically available datasets and achieved an improved accuracy of 95.14, 94.89, and 95.94%, respectively. The comparison with several neural nets shows the improvement of proposed framework.
Collapse
Affiliation(s)
| | - Awais Khan
- Department of Computer Science HITEC University Taxila Pakistan
| | - Majed Alhaisoni
- Computer Sciences Department, College of Computer and Information Sciences Princess Nourah bint Abdulrahman University Riyadh Saudi Arabia
| | - Abdullah Alqahtani
- College of Computer Engineering and Sciences Prince Sattam bin Abdulaziz University Al‐Kharj Saudi Arabia
| | - Shtwai Alsubai
- College of Computer Engineering and Sciences Prince Sattam bin Abdulaziz University Al‐Kharj Saudi Arabia
| | - Meshal Alharbi
- Department of Computer Science, College of Computer Engineering and Sciences Prince Sattam Bin Abdulaziz University Al‐Kharj Saudi Arabia
| | - Nazir Ahmed Malik
- Cyber Reconnaissance and Combat Lab Bahria University Islamabad Islamabad Pakistan
| | | |
Collapse
|
4
|
Si X, Yang Z, Zhang X, Sun Y, Jin W, Wang L, Yin S, Ming D. Patient-independent seizure detection based on long-term iEEG and a novel lightweight CNN. J Neural Eng 2023; 20. [PMID: 36626831 DOI: 10.1088/1741-2552/acb1d9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 01/10/2023] [Indexed: 01/11/2023]
Abstract
Objective.Patient-dependent seizure detection based on intracranial electroencephalography (iEEG) has made significant progress. However, due to the difference in the locations and number of iEEG electrodes used for each patient, patient-independent seizure detection based on iEEG has not been carried out. Additionally, current seizure detection algorithms based on deep learning have outperformed traditional machine learning algorithms in many performance metrics. However, they still have shortcomings of large memory footprints and slow inference speed.Approach.To solve the above problems of the current study, we propose a novel lightweight convolutional neural network model combining the Convolutional Block Attention Module (CBAM). Its performance for patient-independent seizure detection is evaluated on two long-term continuous iEEG datasets: SWEC-ETHZ and TJU-HH. Finally, we reproduce four other patient-independent methods to compare with our method and calculate the memory footprints and inference speed for all methods.Main results.Our method achieves 83.81% sensitivity (SEN) and 85.4% specificity (SPE) on the SWEC-ETHZ dataset and 86.63% SEN and 92.21% SPE on the TJU-HH dataset. In particular, it takes only 11 ms to infer 10 min iEEG (128 channels), and its memory footprint is only 22 kB. Compared to baseline methods, our method not only achieves better patient-independent seizure detection performance but also has a smaller memory footprint and faster inference speed.Significance.To our knowledge, this is the first iEEG-based patient-independent seizure detection study. This facilitates the application of seizure detection algorithms to the future clinic.
Collapse
Affiliation(s)
- Xiaopeng Si
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Zhuobin Yang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Xingjian Zhang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Yulin Sun
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Weipeng Jin
- Department of Neurosurgery, Huanhu Hospital, Tianjin University, Tianjin 300072, People's Republic of China
| | - Le Wang
- Department of Neurosurgery, Huanhu Hospital, Tianjin University, Tianjin 300072, People's Republic of China
| | - Shaoya Yin
- Department of Neurosurgery, Huanhu Hospital, Tianjin University, Tianjin 300072, People's Republic of China
| | - Dong Ming
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China.,Tianjin Key Laboratory of Brain Science and Neural Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|
5
|
Bożko A, Ambroziak L. Influence of Insufficient Dataset Augmentation on IoU and Detection Threshold in CNN Training for Object Detection on Aerial Images. SENSORS (BASEL, SWITZERLAND) 2022; 22:9080. [PMID: 36501781 PMCID: PMC9740240 DOI: 10.3390/s22239080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 11/18/2022] [Accepted: 11/21/2022] [Indexed: 06/17/2023]
Abstract
The objects and events detection tasks are being performed progressively often by robotic systems like unmanned aerial vehicles (UAV) or unmanned surface vehicles (USV). Autonomous operations and intelligent sensing are becoming standard in numerous scenarios such as supervision or even search and rescue (SAR) missions. The low cost of autonomous vehicles, vision sensors and portable computers allows the incorporation of the deep learning, mainly convolutional neural networks (CNN) in these solutions. Many systems meant for custom purposes rely on insufficient training datasets, what may cause a decrease of effectiveness. Moreover, the system's accuracy is usually dependent on the returned bounding boxes highlighting the supposed targets. In desktop applications, precise localisation might not be particularly relevant; however, in real situations, with low visibility and non-optimal camera orientation, it becomes crucial. One of the solutions for dataset enhancement is its augmentation. The presented work is an attempt to evaluate the influence of the training images augmentation on the detection parameters important for the effectiveness of neural networks in the context of object detection. In this research, network appraisal relies on the detection confidence and bounding box prediction accuracy (IoU). All the applied image modifications were simple pattern and colour alterations. The obtained results imply that there is a measurable impact of the augmentation process on the localisation accuracy. It was concluded that a positive or negative influence is related to the complexity and variability of the objects classes.
Collapse
|
6
|
Ragab M, Choudhry H, H. Asseri A, Binyamin SS, Al-Rabia MW. Enhanced Gravitational Search Optimization with Hybrid Deep Learning Model for COVID-19 Diagnosis on Epidemiology Data. Healthcare (Basel) 2022; 10:healthcare10071339. [PMID: 35885865 PMCID: PMC9317045 DOI: 10.3390/healthcare10071339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 07/13/2022] [Accepted: 07/14/2022] [Indexed: 11/16/2022] Open
Abstract
Effective screening provides efficient and quick diagnoses of COVID-19 and could alleviate related problems in the health care system. A prediction model that combines multiple features to assess contamination risks was established in the hope of supporting healthcare workers worldwide in triaging patients, particularly in situations with limited health care resources. Furthermore, a lack of diagnosis kits and asymptomatic cases can lead to missed or delayed diagnoses, exposing visitors, medical staff, and patients to 2019-nCoV contamination. Non-clinical techniques including data mining, expert systems, machine learning, and other artificial intelligence technologies have a crucial role to play in containment and diagnosis in the COVID-19 outbreak. This study developed Enhanced Gravitational Search Optimization with a Hybrid Deep Learning Model (EGSO-HDLM) for COVID-19 diagnoses using epidemiology data. The major aim of designing the EGSO-HDLM model was the identification and classification of COVID-19 using epidemiology data. In order to examine the epidemiology data, the EGSO-HDLM model employed a hybrid convolutional neural network with a gated recurrent unit based fusion (HCNN-GRUF) model. In addition, the hyperparameter optimization of the HCNN-GRUF model was improved by the use of the EGSO algorithm, which was derived by including the concepts of cat map and the traditional GSO algorithm. The design of the EGSO algorithm helps in reducing the ergodic problem, avoiding premature convergence, and enhancing algorithm efficiency. To demonstrate the better performance of the EGSO-HDLM model, experimental validation on a benchmark dataset was performed. The simulation results ensured the enhanced performance of the EGSO-HDLM model over recent approaches.
Collapse
Affiliation(s)
- Mahmoud Ragab
- Information Technology Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
- Centre for Artificial Intelligence in Precision Medicines, King Abdulaziz University, Jeddah 21589, Saudi Arabia; (H.C.); (A.H.A.)
- Mathematics Department, Faculty of Science, Al-Azhar University, Nasr City, Cairo 11884, Egypt
- Correspondence:
| | - Hani Choudhry
- Centre for Artificial Intelligence in Precision Medicines, King Abdulaziz University, Jeddah 21589, Saudi Arabia; (H.C.); (A.H.A.)
- Biochemistry Department, Faculty of Science, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Amer H. Asseri
- Centre for Artificial Intelligence in Precision Medicines, King Abdulaziz University, Jeddah 21589, Saudi Arabia; (H.C.); (A.H.A.)
- Biochemistry Department, Faculty of Science, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| | - Sami Saeed Binyamin
- Computer and Information Technology Department, The Applied College, King Abdulaziz University, Jeddah 21589, Saudi Arabia;
| | - Mohammed W. Al-Rabia
- Department of Medical Microbiology and Parasitolog, Faculty of Medicine, King Abdulaziz University, Jeddah 21589, Saudi Arabia;
- Health Promotion Center, King Abdulaziz University, Jeddah 21589, Saudi Arabia
| |
Collapse
|
7
|
Do Vaccination Attitudes Mediate the Link between Critical Consciousness and COVID-19 Vaccination Behaviour? A Cross-Sectional Study. SUSTAINABILITY 2022. [DOI: 10.3390/su14137623] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Critical consciousness development represents a building block in the formation of health-related attitudes and behaviours. One of the most studied health-related behaviours in the previous year is COVID-19 vaccination behaviour. The present study examines critical consciousness, COVID-19 vaccination attitudes and vaccination behaviour in a consistent sample of young adults (N = 1185). Participants were residents of Romania, aged 18 to 41 years. The Critical Consciousness Scale—Short Form (CCS-S) and the Vaccination Attitudes Examination (VAX) Scale were used. The results demonstrate the mediating role of two types of vaccination attitude—lack of confidence in the promoted vaccine benefits and worries about unforeseen effects—in the relationship between critical reflection and vaccination behaviour. Communication is fundamental in an initiative aimed at changing behaviour. When shaping a health-related campaign, communication and dialogue must be horizontal and characterised by empathy and mutual recognition to determine favourable health-related behaviour.
Collapse
|
8
|
Di Martino F, Orciuoli F. A computational framework to support the treatment of bedsores during COVID-19 diffusion. JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING 2022:1-11. [PMID: 35669338 PMCID: PMC9135601 DOI: 10.1007/s12652-022-03886-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 04/28/2022] [Indexed: 06/15/2023]
Abstract
The treatment of pressure ulcers, also known as bedsores, is a complex process that requires to employ specialized field workforce assisting patients in their houses. In the period of COVID-19 or during any other non-trivial emergency, reaching the patients in their own house is impossible. Therefore, as well as in the other sectors, the adoption of digital technologies is invoked to solve, or at least mitigate, the problem. In particular, during the COVID-19, the social distances should be maintained in order to decrease the risk of contagion. The Project Health Management Systems proposes a complete framework, based on Deep Learning, Augmented Reality. Pattern Matching, Image Segmentation and Edge Detection approaches, to support the treatment of bedsores without increasing the risk of contagion, i.e., improving the remote aiding of specialized operators and physicians and involving inexperienced familiars in the process.
Collapse
Affiliation(s)
- Ferdinando Di Martino
- Dip.to di Architettura, Università degli Studi di Napoli Federico II, Via Toledo 402, Napoli, Italy
| | - Francesco Orciuoli
- Dip.to Scienze Aziendali - Management & Innovation Systems, Università degli Studi di Salerno, Via Giovanni Paolo II, 132 Fisciano, SA Italy
| |
Collapse
|
9
|
Recognition of Leaf Disease Using Hybrid Convolutional Neural Network by Applying Feature Reduction. SENSORS 2022; 22:s22020575. [PMID: 35062534 PMCID: PMC8779777 DOI: 10.3390/s22020575] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Revised: 01/06/2022] [Accepted: 01/11/2022] [Indexed: 01/27/2023]
Abstract
Agriculture is crucial to the economic prosperity and development of India. Plant diseases can have a devastating influence towards food safety and a considerable loss in the production of agricultural products. Disease identification on the plant is essential for long-term agriculture sustainability. Manually monitoring plant diseases is difficult due to time limitations and the diversity of diseases. In the realm of agricultural inputs, automatic characterization of plant diseases is widely required. Based on performance out of all image-processing methods, is better suited for solving this task. This work investigates plant diseases in grapevines. Leaf blight, Black rot, stable, and Black measles are the four types of diseases found in grape plants. Several earlier research proposals using machine learning algorithms were created to detect one or two diseases in grape plant leaves; no one offers a complete detection of all four diseases. The photos are taken from the plant village dataset in order to use transfer learning to retrain the EfficientNet B7 deep architecture. Following the transfer learning, the collected features are down-sampled using a Logistic Regression technique. Finally, the most discriminant traits are identified with the highest constant accuracy of 98.7% using state-of-the-art classifiers after 92 epochs. Based on the simulation findings, an appropriate classifier for this application is also suggested. The proposed technique’s effectiveness is confirmed by a fair comparison to existing procedures.
Collapse
|