1
|
Fabijan A, Zawadzka-Fabijan A, Fabijan R, Zakrzewski K, Nowosławska E, Polis B. Artificial Intelligence in Medical Imaging: Analyzing the Performance of ChatGPT and Microsoft Bing in Scoliosis Detection and Cobb Angle Assessment. Diagnostics (Basel) 2024; 14:773. [PMID: 38611686 PMCID: PMC11011528 DOI: 10.3390/diagnostics14070773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 03/24/2024] [Accepted: 04/04/2024] [Indexed: 04/14/2024] Open
Abstract
Open-source artificial intelligence models (OSAIM) find free applications in various industries, including information technology and medicine. Their clinical potential, especially in supporting diagnosis and therapy, is the subject of increasingly intensive research. Due to the growing interest in artificial intelligence (AI) for diagnostic purposes, we conducted a study evaluating the capabilities of AI models, including ChatGPT and Microsoft Bing, in the diagnosis of single-curve scoliosis based on posturographic radiological images. Two independent neurosurgeons assessed the degree of spinal deformation, selecting 23 cases of severe single-curve scoliosis. Each posturographic image was separately implemented onto each of the mentioned platforms using a set of formulated questions, starting from 'What do you see in the image?' and ending with a request to determine the Cobb angle. In the responses, we focused on how these AI models identify and interpret spinal deformations and how accurately they recognize the direction and type of scoliosis as well as vertebral rotation. The Intraclass Correlation Coefficient (ICC) with a 'two-way' model was used to assess the consistency of Cobb angle measurements, and its confidence intervals were determined using the F test. Differences in Cobb angle measurements between human assessments and the AI ChatGPT model were analyzed using metrics such as RMSEA, MSE, MPE, MAE, RMSLE, and MAPE, allowing for a comprehensive assessment of AI model performance from various statistical perspectives. The ChatGPT model achieved 100% effectiveness in detecting scoliosis in X-ray images, while the Bing model did not detect any scoliosis. However, ChatGPT had limited effectiveness (43.5%) in assessing Cobb angles, showing significant inaccuracy and discrepancy compared to human assessments. This model also had limited accuracy in determining the direction of spinal curvature, classifying the type of scoliosis, and detecting vertebral rotation. Overall, although ChatGPT demonstrated potential in detecting scoliosis, its abilities in assessing Cobb angles and other parameters were limited and inconsistent with expert assessments. These results underscore the need for comprehensive improvement of AI algorithms, including broader training with diverse X-ray images and advanced image processing techniques, before they can be considered as auxiliary in diagnosing scoliosis by specialists.
Collapse
Affiliation(s)
- Artur Fabijan
- Department of Neurosurgery, Polish-Mother’s Memorial Hospital Research Institute, 93-338 Lodz, Poland; (K.Z.); (E.N.); (B.P.)
| | - Agnieszka Zawadzka-Fabijan
- Department of Rehabilitation Medicine, Faculty of Health Sciences, Medical University of Lodz, 90-419 Lodz, Poland;
| | | | - Krzysztof Zakrzewski
- Department of Neurosurgery, Polish-Mother’s Memorial Hospital Research Institute, 93-338 Lodz, Poland; (K.Z.); (E.N.); (B.P.)
| | - Emilia Nowosławska
- Department of Neurosurgery, Polish-Mother’s Memorial Hospital Research Institute, 93-338 Lodz, Poland; (K.Z.); (E.N.); (B.P.)
| | - Bartosz Polis
- Department of Neurosurgery, Polish-Mother’s Memorial Hospital Research Institute, 93-338 Lodz, Poland; (K.Z.); (E.N.); (B.P.)
| |
Collapse
|
2
|
Yu KL, Tseng YS, Yang HC, Liu CJ, Kuo PC, Lee MR, Huang CT, Kuo LC, Wang JY, Ho CC, Shih JY, Yu CJ. Deep learning with test-time augmentation for radial endobronchial ultrasound image differentiation: a multicentre verification study. BMJ Open Respir Res 2023; 10:e001602. [PMID: 37532473 PMCID: PMC10401203 DOI: 10.1136/bmjresp-2022-001602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 06/23/2023] [Indexed: 08/04/2023] Open
Abstract
PURPOSE Despite the importance of radial endobronchial ultrasound (rEBUS) in transbronchial biopsy, researchers have yet to apply artificial intelligence to the analysis of rEBUS images. MATERIALS AND METHODS This study developed a convolutional neural network (CNN) to differentiate between malignant and benign tumours in rEBUS images. This study retrospectively collected rEBUS images from medical centres in Taiwan, including 769 from National Taiwan University Hospital Hsin-Chu Branch, Hsinchu Hospital for model training (615 images) and internal validation (154 images) as well as 300 from National Taiwan University Hospital (NTUH-TPE) and 92 images were obtained from National Taiwan University Hospital Hsin-Chu Branch, Biomedical Park Hospital (NTUH-BIO) for external validation. Further assessments of the model were performed using image augmentation in the training phase and test-time augmentation (TTA). RESULTS Using the internal validation dataset, the results were as follows: area under the curve (AUC) (0.88 (95% CI 0.83 to 0.92)), sensitivity (0.80 (95% CI 0.73 to 0.88)), specificity (0.75 (95% CI 0.66 to 0.83)). Using the NTUH-TPE external validation dataset, the results were as follows: AUC (0.76 (95% CI 0.71 to 0.80)), sensitivity (0.58 (95% CI 0.50 to 0.65)), specificity (0.92 (95% CI 0.88 to 0.97)). Using the NTUH-BIO external validation dataset, the results were as follows: AUC (0.72 (95% CI 0.64 to 0.82)), sensitivity (0.71 (95% CI 0.55 to 0.86)), specificity (0.76 (95% CI 0.64 to 0.87)). After fine-tuning, the AUC values for the external validation cohorts were as follows: NTUH-TPE (0.78) and NTUH-BIO (0.82). Our findings also demonstrated the feasibility of the model in differentiating between lung cancer subtypes, as indicated by the following AUC values: adenocarcinoma (0.70; 95% CI 0.64 to 0.76), squamous cell carcinoma (0.64; 95% CI 0.54 to 0.74) and small cell lung cancer (0.52; 95% CI 0.32 to 0.72). CONCLUSIONS Our results demonstrate the feasibility of the proposed CNN-based algorithm in differentiating between malignant and benign lesions in rEBUS images.
Collapse
Affiliation(s)
- Kai-Lun Yu
- Department of Internal Medicine, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu, Taiwan
- Graduate Institute of Clinical Medicine, National Taiwan University College of Medicine, Taipei, Taiwan
| | - Yi-Shiuan Tseng
- Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan
| | - Han-Ching Yang
- Department of Internal Medicine, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu, Taiwan
| | - Chia-Jung Liu
- Department of Internal Medicine, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu, Taiwan
| | - Po-Chih Kuo
- Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan
| | - Meng-Rui Lee
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Chun-Ta Huang
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Lu-Cheng Kuo
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Jann-Yuan Wang
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Chao-Chi Ho
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Jin-Yuan Shih
- Graduate Institute of Clinical Medicine, National Taiwan University College of Medicine, Taipei, Taiwan
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| | - Chong-Jen Yu
- Department of Internal Medicine, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu, Taiwan
- Department of Internal Medicine, National Taiwan University Hospital, Taipei, Taiwan
| |
Collapse
|
3
|
Zhu Z, Wang S, Zhang Y. ROENet: A ResNet-Based Output Ensemble for Malaria Parasite Classification. ELECTRONICS 2022; 11:2040. [PMID: 36567678 PMCID: PMC7613984 DOI: 10.3390/electronics11132040] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Background (1)People may be infected with an insect-borne disease (malaria) through the blood input of malaria-infected people or the bite of Anopheles mosquitoes. Doctors need a lot of time and energy to diagnose malaria, and sometimes the results are not ideal. Many researchers use CNN to classify malaria images. However, we believe that the classification performance of malaria parasites can be improved. Methods (2)In this paper, we propose a novel method (ROENet) to automatically classify malaria parasite on the blood smear. The backbone of ROENet is the pretrained ResNet-18. We use randomized neural networks (RNNs) as the classifier in our proposed model. Three RNNs are used in ROENet, which are random vector functional link (RVFL), Schmidt neural network (SNN), and extreme learning machine (ELM). To improve the performance of ROENet, the results of ROENet are the ensemble outputs from three RNNs. Results (3)We evaluate the proposed ROENet by five-fold cross-validation. The specificity, F1 score, sensitivity, and accuracy are 96.68 ± 3.81%, 95.69 ± 2.65%, 94.79 ± 3.71%, and 95.73 ± 2.63%, respectively. Conclusions (4)The proposed ROENet is compared with other state-of-the-art methods and provides the best results of these methods.
Collapse
Affiliation(s)
- Ziquan Zhu
- School of Computing and Mathematical Sciences, University of Leicester, East Midlands, Leicester, LE1 7RH, UK
| | - ShuiHua Wang
- School of Computing and Mathematical Sciences, University of Leicester, East Midlands, Leicester, LE1 7RH, UK
- Correspondence: (S.-H.W.); (Y.-D.Z.)
| | - YuDong Zhang
- School of Computing and Mathematical Sciences, University of Leicester, East Midlands, Leicester, LE1 7RH, UK
- Correspondence: (S.-H.W.); (Y.-D.Z.)
| |
Collapse
|