1
|
Xu C, Li X, Zhang X, Wu R, Zhou Y, Zhao Q, Zhang Y, Geng S, Gu Y, Hong S. Cardiac murmur grading and risk analysis of cardiac diseases based on adaptable heterogeneous-modality multi-task learning. Health Inf Sci Syst 2024; 12:2. [PMID: 38045019 PMCID: PMC10692066 DOI: 10.1007/s13755-023-00249-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Accepted: 09/20/2023] [Indexed: 12/05/2023] Open
Abstract
Cardiovascular disease (CVDs) has become one of the leading causes of death, posing a significant threat to human life. The development of reliable Artificial Intelligence (AI) assisted diagnosis algorithms for cardiac sounds is of great significance for early detection and treatment of CVDs. However, there is scarce research in this field. Existing research mainly faces three major challenges: (1) They mainly limited to murmur classification and cannot achieve murmur grading, but attempting both classification and grading may lead to negative effects between different multi-tasks. (2) They mostly pay attention to unstructured cardiac sound modality and do not consider the structured demographic modality, as it is difficult to balance the influence of heterogeneous modalities. (3) Deep learning methods lack interpretability, which makes it challenging to apply them clinically. To tackle these challenges, we propose a method for cardiac murmur grading and cardiac risk analysis based on heterogeneous modality adaptive multi-task learning. Specifically, a Hierarchical Multi-Task learning-based cardiac murmur detection and grading method (HMT) is proposed to prevent negative interference between different tasks. In addition, a cardiac risk analysis method based on Heterogeneous Multi-modal feature impact Adaptation (HMA) is also proposed, which transforms unstructured modality into structured modality representation, and utilizes an adaptive mode weight learning mechanism to balance the impact between unstructured modality and structured modality, thus enhancing the performance of cardiac risk prediction. Finally, we propose a multi-task interpretability learning module that incorporates an important evaluation using random masks. This module utilizes SHAP graphs to visualize crucial murmur segments in cardiac sound and employs a multi-factor risk decoupling model based on nomograms. And then we gain insights into the cardiac disease risk in both pre-decoupled multi-modality and post-decoupled single-modality scenarios, thus providing a solid foundation for AI assisted cardiac murmur grading and risk analysis. Experimental results on a large real-world CirCor DigiScope PCG dataset demonstrate that the proposed method outperforms the state-of-the-art (SOTA) method in murmur detection, grading, and cardiac risk analysis, while also providing valuable diagnostic evidence.
Collapse
Affiliation(s)
- Chenyang Xu
- Department of Computer Science, Tianjin University of Technology, Tianjin, China
| | - Xin Li
- Department of Rehabilitation Medicine, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Xinyue Zhang
- Department of Computer Science, Tianjin University of Technology, Tianjin, China
| | - Ruilin Wu
- Department of Computer Science, Tianjin University of Technology, Tianjin, China
| | - Yuxi Zhou
- Department of Computer Science, Tianjin University of Technology, Tianjin, China
- DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China
| | - Qinghao Zhao
- Department of Cardiology, Peking University People’s Hospital, Beijing, China
| | - Yong Zhang
- DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China
| | | | - Yue Gu
- Department of Computer Science, Tianjin University of Technology, Tianjin, China
| | - Shenda Hong
- National Institute of Health Data Science, Peking University, Beijing, China
- Institute of Medical Technology, Peking University, Beijing, China
| |
Collapse
|
2
|
Sobotka D, Herold A, Perkonigg M, Beer L, Bastati N, Sablatnig A, Ba-Ssalamah A, Langs G. Improving Vessel Segmentation with Multi-Task Learning and Auxiliary Data Available Only During Model Training. Comput Med Imaging Graph 2024; 114:102369. [PMID: 38518411 DOI: 10.1016/j.compmedimag.2024.102369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 03/13/2024] [Accepted: 03/13/2024] [Indexed: 03/24/2024]
Abstract
Liver vessel segmentation in magnetic resonance imaging data is important for the computational analysis of vascular remodeling, associated with a wide spectrum of diffuse liver diseases. Existing approaches rely on contrast enhanced imaging data, but the necessary dedicated imaging sequences are not uniformly acquired. Images without contrast enhancement are acquired more frequently, but vessel segmentation is challenging, and requires large-scale annotated data. We propose a multi-task learning framework to segment vessels in liver MRI without contrast. It exploits auxiliary contrast enhanced MRI data available only during training to reduce the need for annotated training examples. Our approach draws on paired native and contrast enhanced data with and without vessel annotations for model training. Results show that auxiliary data improves the accuracy of vessel segmentation, even if they are not available during inference. The advantage is most pronounced if only few annotations are available for training, since the feature representation benefits from the shared task structure. A validation of this approach to augment a model for brain tumor segmentation confirms its benefits across different domains. An auxiliary informative imaging modality can augment expert annotations even if it is only available during training.
Collapse
Affiliation(s)
- Daniel Sobotka
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Alexander Herold
- Division of General and Paediatric Radiology, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Matthias Perkonigg
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria; Department of Medical Statistics, Informatics and Health Economics, Medical University of Innsbruck, Innsbruck, Austria
| | - Lucian Beer
- Division of General and Paediatric Radiology, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Nina Bastati
- Division of General and Paediatric Radiology, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Alina Sablatnig
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Ahmed Ba-Ssalamah
- Division of General and Paediatric Radiology, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria
| | - Georg Langs
- Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria.
| |
Collapse
|
3
|
Fu K, Li H, Shi X. CTF-former: A novel simplified multi-task learning strategy for simultaneous multivariate chaotic time series prediction. Neural Netw 2024; 174:106234. [PMID: 38521015 DOI: 10.1016/j.neunet.2024.106234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 02/22/2024] [Accepted: 03/11/2024] [Indexed: 03/25/2024]
Abstract
Multivariate chaotic time series prediction is a challenging task, especially when multiple variables are predicted simultaneously. For multiple related prediction tasks typically require multiple models, however, multiple models are difficult to keep synchronization, making immediate communication between predicted values challenging. Although multi-task learning can be applied to this problem, the principles of allocation and layout options between shared and specific representations are ambiguous. To address this issue, a novel simplified multi-task learning method was proposed for the precise implementation of simultaneous multiple chaotic time series prediction tasks. The scheme proposed consists of a cross-convolution operator designed to capture variable correlations and sequence correlations, and an attention module proposed to capture the information embedded in the sequence structure. In the attention module, a non-linear transformation was implemented with convolution, and its local receptive field and the global dependency of the attention mechanism achieve complementarity. In addition, an attention weight calculation was devised that takes into account not only the synergy of time and frequency domain features, but also the fusion of series and channel information. Notably the scheme proposed a purely simplified design principle of multi-task learning by reducing the specific network to single neuron. The precision of the proposed solution and its potential for engineering applications were verified with the Lorenz system and power consumption. The mean absolute error of the proposed method was reduced by an average of 82.9% in the Lorenz system and 19.83% in power consumption compared to the Gated Recurrent Unit.
Collapse
Affiliation(s)
- Ke Fu
- School of Mechanical Engineering & Automation, Northeastern University, Shenyang 110819, China
| | - He Li
- School of Mechanical Engineering & Automation, Northeastern University, Shenyang 110819, China.
| | - Xiaotian Shi
- School of Mechanical Engineering & Automation, Northeastern University, Shenyang 110819, China
| |
Collapse
|
4
|
Bui DC, Song B, Kim K, Kwak JT. DAX-Net: A dual-branch dual-task adaptive cross-weight feature fusion network for robust multi-class cancer classification in pathology images. Comput Methods Programs Biomed 2024; 248:108112. [PMID: 38479146 DOI: 10.1016/j.cmpb.2024.108112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Revised: 02/15/2024] [Accepted: 03/01/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND AND OBJECTIVE Multi-class cancer classification has been extensively studied in digital and computational pathology due to its importance in clinical decision-making. Numerous computational tools have been proposed for various types of cancer classification. Many of them are built based on convolutional neural networks. Recently, Transformer-style networks have shown to be effective for cancer classification. Herein, we present a hybrid design that leverages both convolutional neural networks and transformer architecture to obtain superior performance in cancer classification. METHODS We propose a dual-branch dual-task adaptive cross-weight feature fusion network, called DAX-Net, which exploits heterogeneous feature representations from the convolutional neural network and Transformer network, adaptively combines them to boost their representation power, and conducts cancer classification as categorical classification and ordinal classification. For an efficient and effective optimization of the proposed model, we introduce two loss functions that are tailored to the two classification tasks. RESULTS To evaluate the proposed method, we employed colorectal and prostate cancer datasets, of which each contains both in-domain and out-of-domain test sets. For colorectal cancer, the proposed method obtained an accuracy of 88.4%, a quadratic kappa score of 0.945, and an F1 score of 0.831 for the in-domain test set, and 84.4%, 0.910, and 0.768 for the out-of-domain test set. For prostate cancer, it achieved an accuracy of 71.6%, a kappa score of 0.635, and an F1 score of 0.655 for the in-domain test set, 79.2% accuracy, 0.721 kappa score, and 0.686 F1 score for the first out-of-domain test set, and 58.1% accuracy, 0.564 kappa score, and 0.493 F1 score for the second out-of-domain test set. It is worth noting that the performance of the proposed method outperformed other competitors by significant margins, in particular, with respect to the out-of-domain test sets. CONCLUSIONS The experimental results demonstrate that the proposed method is not only accurate but also robust to varying conditions of the test sets in comparison to several, related methods. These results suggest that the proposed method can facilitate automated cancer classification in various clinical settings.
Collapse
Affiliation(s)
- Doanh C Bui
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Boram Song
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Kyungeun Kim
- Department of Pathology, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, 03181, Republic of Korea
| | - Jin Tae Kwak
- School of Electrical Engineering, Korea University, Seoul, 02841, Republic of Korea.
| |
Collapse
|
5
|
He Q, Yang Q, Su H, Wang Y. Multi-task learning for segmentation and classification of breast tumors from ultrasound images. Comput Biol Med 2024; 173:108319. [PMID: 38513394 DOI: 10.1016/j.compbiomed.2024.108319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 03/03/2024] [Accepted: 03/12/2024] [Indexed: 03/23/2024]
Abstract
Segmentation and classification of breast tumors are critical components of breast ultrasound (BUS) computer-aided diagnosis (CAD), which significantly improves the diagnostic accuracy of breast cancer. However, the characteristics of tumor regions in BUS images, such as non-uniform intensity distributions, ambiguous or missing boundaries, and varying tumor shapes and sizes, pose significant challenges to automated segmentation and classification solutions. Many previous studies have proposed multi-task learning methods to jointly tackle tumor segmentation and classification by sharing the features extracted by the encoder. Unfortunately, this often introduces redundant or misleading information, which hinders effective feature exploitation and adversely affects performance. To address this issue, we present ACSNet, a novel multi-task learning network designed to optimize tumor segmentation and classification in BUS images. The segmentation network incorporates a novel gate unit to allow optimal transfer of valuable contextual information from the encoder to the decoder. In addition, we develop the Deformable Spatial Attention Module (DSAModule) to improve segmentation accuracy by overcoming the limitations of conventional convolution in dealing with morphological variations of tumors. In the classification branch, multi-scale feature extraction and channel attention mechanisms are integrated to discriminate between benign and malignant breast tumors. Experiments on two publicly available BUS datasets demonstrate that ACSNet not only outperforms mainstream multi-task learning methods for both breast tumor segmentation and classification tasks, but also achieves state-of-the-art results for BUS tumor segmentation. Code and models are available at https://github.com/qqhe-frank/BUS-segmentation-and-classification.git.
Collapse
Affiliation(s)
- Qiqi He
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China; School of Life Science and Technology, Xidian University, Xi'an, China
| | - Qiuju Yang
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China.
| | - Hang Su
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| | - Yixuan Wang
- School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China
| |
Collapse
|
6
|
Hou G, Jian Y, Zhao Q, Quan X, Zhang H. Language model based on deep learning network for biomedical named entity recognition. Methods 2024; 226:71-77. [PMID: 38641084 DOI: 10.1016/j.ymeth.2024.04.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 12/22/2023] [Accepted: 04/16/2024] [Indexed: 04/21/2024] Open
Abstract
Biomedical Named Entity Recognition (BioNER) is one of the most basic tasks in biomedical text mining, which aims to automatically identify and classify biomedical entities in text. Recently, deep learning-based methods have been applied to Biomedical Named Entity Recognition and have shown encouraging results. However, many biological entities are polysemous and ambiguous, which is one of the main obstacles to the task of biomedical named entity recognition. Deep learning methods require large amounts of training data, so the lack of data also affect the performance of model recognition. To solve the problem of polysemous words and insufficient data, for the task of biomedical named entity recognition, we propose a multi-task learning framework fused with language model based on the BiLSTM-CRF architecture. Our model uses a language model to design a differential encoding of the context, which could obtain dynamic word vectors to distinguish words in different datasets. Moreover, we use a multi-task learning method to collectively share the dynamic word vector of different types of entities to improve the recognition performance of each type of entity. Experimental results show that our model reduces the false positives caused by polysemous words through differentiated coding, and improves the performance of each subtask by sharing information between different entity data. Compared with other state-of-the art methods, our model achieved superior results in four typical training sets, and achieved the best results in F1 values.
Collapse
Affiliation(s)
- Guan Hou
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Yuhao Jian
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Qingqing Zhao
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Xiongwen Quan
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Han Zhang
- College of Artificial Intelligence, Nankai University, Tianjin, China.
| |
Collapse
|
7
|
Zhang Y, Yang X, Cheng Y, Wu X, Sun X, Hou R, Wang H. Fruit freshness detection based on multi-task convolutional neural network. Curr Res Food Sci 2024; 8:100733. [PMID: 38655189 PMCID: PMC11035072 DOI: 10.1016/j.crfs.2024.100733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Revised: 03/20/2024] [Accepted: 04/04/2024] [Indexed: 04/26/2024] Open
Abstract
Background Fruit freshness detection by computer vision is essential for many agricultural applications, e.g., automatic harvesting and supply chain monitoring. This paper proposes to use the multi-task learning (MTL) paradigm to build a deep convolutional neural work for fruit freshness detection. Results We design an MTL model that optimizes the freshness detection (T1) and fruit type classification (T2) tasks in parallel. The model uses a shared CNN (convolutional neural network) subnet and two FC (fully connected) task heads. The shared CNN acts as a feature extraction module and feeds the two task heads with common semantic features. Based on an open fruit image dataset, we conducted a comparative study of MTL and single-task learning (STL) paradigms. The STL models use the same CNN subnet with only one specific task head. In the MTL scenario, the T1 and T2 mean accuracies on the test set are 93.24% and 88.66%, respectively. Meanwhile, for STL, the two accuracies are 92.50% and 87.22%. Statistical tests report significant differences between MTL and STL on T1 and T2 test accuracies. We further investigated the extracted feature vectors (semantic embeddings) from the two STL models. The vectors have an averaged 0.7 cosine similarity on the entire dataset, with most values lying in the 0.6-0.8 range. This indicates a between-task correlation and justifies the effectiveness of the proposed MTL approach. Conclusion This study proves that MTL exploits the mutual correlation between two or more relevant tasks and can maximally share their underlying feature extraction process. we envision this approach to be extended to other domains that involve multiple interconnected tasks.
Collapse
Affiliation(s)
- Yinsheng Zhang
- Zhejiang Food and Drug Quality & Safety Engineering Research Institute, Zhejiang Gongshang University, Hangzhou, 310018, China
| | - Xudong Yang
- School of Management and E-Business, Zhejiang Gongshang University, Hangzhou, 310018, China
| | - Yongbo Cheng
- School of Management Science and Engineering, Nanjing University of Finance and Economics, Nanjing, 210023, China
| | - Xiaojun Wu
- Institute of Science and Technology, Jiangnan University, Wuxi, 214122, China
| | - Xiulan Sun
- School of Food Science and Technology, Jiangnan University, Wuxi, 214122, China
| | - Ruiqi Hou
- Zhejiang Food and Drug Quality & Safety Engineering Research Institute, Zhejiang Gongshang University, Hangzhou, 310018, China
| | - Haiyan Wang
- Zhejiang Food and Drug Quality & Safety Engineering Research Institute, Zhejiang Gongshang University, Hangzhou, 310018, China
| |
Collapse
|
8
|
Conze PH, Andrade-Miranda G, Le Meur Y, Cornec-Le Gall E, Rousseau F. Dual-task kidney MR segmentation with transformers in autosomal-dominant polycystic kidney disease. Comput Med Imaging Graph 2024; 113:102349. [PMID: 38330635 DOI: 10.1016/j.compmedimag.2024.102349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Revised: 01/21/2024] [Accepted: 02/01/2024] [Indexed: 02/10/2024]
Abstract
Autosomal-dominant polycystic kidney disease is a prevalent genetic disorder characterized by the development of renal cysts, leading to kidney enlargement and renal failure. Accurate measurement of total kidney volume through polycystic kidney segmentation is crucial to assess disease severity, predict progression and evaluate treatment effects. Traditional manual segmentation suffers from intra- and inter-expert variability, prompting the exploration of automated approaches. In recent years, convolutional neural networks have been employed for polycystic kidney segmentation from magnetic resonance images. However, the use of Transformer-based models, which have shown remarkable performance in a wide range of computer vision and medical image analysis tasks, remains unexplored in this area. With their self-attention mechanism, Transformers excel in capturing global context information, which is crucial for accurate organ delineations. In this paper, we evaluate and compare various convolutional-based, Transformers-based, and hybrid convolutional/Transformers-based networks for polycystic kidney segmentation. Additionally, we propose a dual-task learning scheme, where a common feature extractor is followed by per-kidney decoders, towards better generalizability and efficiency. We extensively evaluate various architectures and learning schemes on a heterogeneous magnetic resonance imaging dataset collected from 112 patients with polycystic kidney disease. Our results highlight the effectiveness of Transformer-based models for polycystic kidney segmentation and the relevancy of exploiting dual-task learning to improve segmentation accuracy and mitigate data scarcity issues. A promising ability in accurately delineating polycystic kidneys is especially shown in the presence of heterogeneous cyst distributions and adjacent cyst-containing organs. This work contribute to the advancement of reliable delineation methods in nephrology, paving the way for a broad spectrum of clinical applications.
Collapse
Affiliation(s)
- Pierre-Henri Conze
- IMT Atlantique, LaTIM UMR 1101, Technopôle Brest-Iroise, 29238 Brest, France; LaTIM UMR 1101, Inserm, IBRBS, 22 rue Camille Desmoulins, 29200 Brest, France.
| | | | - Yannick Le Meur
- Department of Nephrology, University Hospital of Brest, bd Tanguy Prigent, 29200 Brest, France; LBAI UMR 1227, Inserm, 9 rue Félix le Dantec, 29200 Brest, France
| | - Emilie Cornec-Le Gall
- Department of Nephrology, University Hospital of Brest, bd Tanguy Prigent, 29200 Brest, France; UMR 1078, Inserm, IBRBS, 22 rue Camille Desmoulins, 29238 Brest, France
| | - François Rousseau
- IMT Atlantique, LaTIM UMR 1101, Technopôle Brest-Iroise, 29238 Brest, France; LaTIM UMR 1101, Inserm, IBRBS, 22 rue Camille Desmoulins, 29200 Brest, France
| |
Collapse
|
9
|
Huang C, Wang Z, Yuan G, Xiong Z, Hu J, Tong Y. PKSEA-Net: A prior knowledge supervised edge-aware multi-task network for retinal arteriolar morphometry. Comput Biol Med 2024; 172:108255. [PMID: 38461696 DOI: 10.1016/j.compbiomed.2024.108255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 02/06/2024] [Accepted: 03/06/2024] [Indexed: 03/12/2024]
Abstract
Retinal fundus images serve as a non-invasive modality to obtain information pertaining to retinal vessels through fundus photography, thereby offering insights into cardiovascular and cerebrovascular diseases. Retinal arteriolar morphometry has emerged as the most convenient and fundamental clinical methodology in the realm of patient screening and diagnosis. Nevertheless, the analysis of retinal arterioles is challenging attributable to imaging noise, stochastic fuzzy characteristics, and blurred boundaries proximal to blood vessels. In response to these limitations, we introduce an innovative methodology, named PKSEA-Net, which aims to improve segmentation accuracy by enhancing the perception of edge information in retinal fundus images. PKSEA-Net employs the universal architecture PVT-v2 as the encoder, complemented by a novel decoder architecture consisting of an Edge-Aware Block (EAB) and a Pyramid Feature Fusion Module (PFFM). The EAB block incorporates prior knowledge for supervision and multi-query for multi-task learning, with supervision information derived from an enhanced Full Width at Half Maximum (FWHM) algorithm and gradient map. Moreover, PFFM efficiently integrates multi-scale features through a novel attention fusion method. Additionally, we have collected a Retinal Cross-Sectional Vessel (RCSV) dataset derived from approximately 200 patients in Quzhou People's Hospital to serve as the benchmark dataset. Comparative evaluations with several state-of-the-art (SOTA) networks confirm that PKSEA-Net achieves exceptional experimental performance, thereby establishing its status as a SOTA approach for precise boundary delineation and retinal vessel segmentation.
Collapse
Affiliation(s)
- Chongjun Huang
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China; Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang 324000, China
| | - Zhuoran Wang
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China; Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang 324000, China
| | - Guohui Yuan
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China; Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang 324000, China
| | - Zhiming Xiong
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China; Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang 324000, China
| | - Jing Hu
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China; Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou, Zhejiang 324000, China
| | - Yuhua Tong
- The Quzhou Affiliated Hospital of Wenzhou Medical University, Quzhou People's Hospital, Quzhou, Zhejiang 324000, China.
| |
Collapse
|
10
|
Wu Y, Li J, Wang X, Zhang Z, Zhao S. DECIDE: A decoupled semantic and boundary learning network for precise osteosarcoma segmentation by integrating multi-modality MRI. Comput Biol Med 2024; 174:108308. [PMID: 38581998 DOI: 10.1016/j.compbiomed.2024.108308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/17/2024] [Accepted: 03/12/2024] [Indexed: 04/08/2024]
Abstract
Automated Osteosarcoma Segmentation in Multi-modality MRI (AOSMM) holds clinical significance for effective tumor evaluation and treatment planning. However, the precision of AOSMM is challenged by the diverse characteristics of multi-modality MRI and the inherent heterogeneity and boundary ambiguity of osteosarcoma. While numerous methods have made significant strides in automated osteosarcoma segmentation, they primarily focused on the use of a single MRI modality and overlooked the potential benefits of integrating complementary information from other MRI modalities. Furthermore, they did not adequately model the long-range dependencies of complex tumor features, which may lead to insufficiently discriminative feature representations. To this end, we propose a decoupled semantic and boundary learning network (DECIDE) to achieve precise AOSMM with three functional modules. The Multi-modality Feature Fusion and Recalibration (MFR) module adaptively fuses and recalibrates multi-modality features by exploiting their channel-wise dependencies to compute low-rank attention weights for effectively aggregating useful information from different MRI modalities, which promotes complementary learning between multi-modality MRI and enables a more comprehensive tumor characterization. The Lesion Attention Enhancement (LAE) module employs spatial and channel attention mechanisms to capture global contextual dependencies over local features, significantly enhancing the discriminability and representational capacity of intricate tumor features. The Boundary Context Aggregation (BCA) module further enhances semantic representations by utilizing boundary information for effective context aggregation while also ensuring intra-class consistency in cases of boundary ambiguity. Substantial experiments demonstrate that DECIDE achieves exceptional performance in osteosarcoma segmentation, surpassing state-of-the-art methods in terms of accuracy and stability.
Collapse
Affiliation(s)
- Yinhao Wu
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China
| | - Jianqi Li
- The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, 510080, China
| | - Xinxin Wang
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China
| | - Zhaohui Zhang
- The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, 510080, China.
| | - Shen Zhao
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China.
| |
Collapse
|
11
|
Park SJ, Yang S, Kim JM, Kang JH, Kim JE, Huh KH, Lee SS, Yi WJ, Heo MS. Automatic and robust estimation of sex and chronological age from panoramic radiographs using a multi-task deep learning network: a study on a South Korean population. Int J Legal Med 2024:10.1007/s00414-024-03204-4. [PMID: 38467754 DOI: 10.1007/s00414-024-03204-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 02/27/2024] [Indexed: 03/13/2024]
Abstract
Sex and chronological age estimation are crucial in forensic investigations and research on individual identification. Although manual methods for sex and age estimation have been proposed, these processes are labor-intensive, time-consuming, and error-prone. The purpose of this study was to estimate sex and chronological age from panoramic radiographs automatically and robustly using a multi-task deep learning network (ForensicNet). ForensicNet consists of a backbone and both sex and age attention branches to learn anatomical context features of sex and chronological age from panoramic radiographs and enables the multi-task estimation of sex and chronological age in an end-to-end manner. To mitigate bias in the data distribution, our dataset was built using 13,200 images with 100 images for each sex and age range of 15-80 years. The ForensicNet with EfficientNet-B3 exhibited superior estimation performance with mean absolute errors of 2.93 ± 2.61 years and a coefficient of determination of 0.957 for chronological age, and achieved accuracy, specificity, and sensitivity values of 0.992, 0.993, and 0.990, respectively, for sex prediction. The network demonstrated that the proposed sex and age attention branches with a convolutional block attention module significantly improved the estimation performance for both sex and chronological age from panoramic radiographs of elderly patients. Consequently, we expect that ForensicNet will contribute to the automatic and accurate estimation of both sex and chronological age from panoramic radiographs.
Collapse
Affiliation(s)
- Se-Jin Park
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea
| | - Su Yang
- Department of Applied Bioengineering, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, 03080, South Korea
| | - Jun-Min Kim
- Department of Electronics and Information Engineering, Hansung University, Seoul, 03080, South Korea
| | - Ju-Hee Kang
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea
| | - Jo-Eun Kim
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea
| | - Kyung-Hoe Huh
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea
| | - Sam-Sun Lee
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea
| | - Won-Jin Yi
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea.
- Department of Applied Bioengineering, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, 03080, South Korea.
- Department of Oral and Maxillofacial Radiology, School of Dentistry, Seoul National University, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea.
| | - Min-Suk Heo
- Department of Oral and Maxillofacial Radiology and Dental Research Institute, School of Dentistry, Seoul National University, Seoul, 03080, South Korea.
- Department of Oral and Maxillofacial Radiology, School of Dentistry, Seoul National University, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Korea.
| |
Collapse
|
12
|
Xu X, Qi Z, Han X, Wang Y, Yu M, Geng Z. Combined-task deep network based on LassoNet feature selection for predicting the comorbidities of acute coronary syndrome. Comput Biol Med 2024; 170:107992. [PMID: 38242014 DOI: 10.1016/j.compbiomed.2024.107992] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 01/03/2024] [Accepted: 01/13/2024] [Indexed: 01/21/2024]
Abstract
Acute coronary syndrome (ACS) is a multifaceted cardiovascular condition frequently accompanied by multiple comorbidities, which can have significant implications for patient outcomes and treatment approaches. Precisely predicting these comorbidities is crucial for providing personalized care and making well-informed clinical decisions. However, there is a shortage of research investigating the identification of risk factors associated with ACS comorbidities and accurately predicting their likelihood of occurrence beyond heart failure. In this study, an approach called Combined-task Deep Network based on LassoNet feature selection (CDNL) is presented for predicting ACS comorbidities, including hypertension, diabetes, hyperlipidemia, and heart failure. In order to identify crucial biomarkers associated with ACS comorbidities, the proposed framework first incorporates LassoNet, which extends Lasso regression to the deep network by adding a skip (residual) layer. Additionally, a correlation score calculation method across tasks is introduced based on measuring the overlap of identified biomarkers and their assigned importance. This method enables the development of an optimal combined-task prediction model for each ACS comorbidity, addressing the challenge of limited representations in traditional multi-task learning. Our evaluation, conducted through a meticulous cross-sectional study at a tertiary hospital in China, involved a dataset of 2941 samples with 42 clinical features. The results demonstrate that CDNL facilitates the identification of significant biomarkers and achieves an average improvement in AUC of 4.93% and 8.58% compared to deep learning multi-layer neural network (DNN) and SVM, respectively. Additionally, it shows an average improvement of 2.64% and 1.92% compared to two state-of-the-art multi-task models.
Collapse
Affiliation(s)
- Xiaolu Xu
- School of Computer and Artificial Intelligence, Liaoning Normal University, Dalian 116029, China
| | - Zitong Qi
- Department of Statistics, University of Washington, Seattle, WA 98195, USA
| | - Xiumei Han
- College of Artificial Intelligence, Dalian Maritime University, Dalian 116026, China
| | - Yuxing Wang
- Department of Cardiology, Second Affiliated Hospital of Dalian Medical University, Dalian 116023, China
| | - Ming Yu
- Department of Cardiology, Second Affiliated Hospital of Dalian Medical University, Dalian 116023, China
| | - Zhaohong Geng
- Department of Cardiology, Second Affiliated Hospital of Dalian Medical University, Dalian 116023, China.
| |
Collapse
|
13
|
Zhang K, Lin PC, Pan J, Shao R, Xu PX, Cao R, Wu CG, Crookes D, Hua L, Wang L. DeepmdQCT: A multitask network with domain invariant features and comprehensive attention mechanism for quantitative computer tomography diagnosis of osteoporosis. Comput Biol Med 2024; 170:107916. [PMID: 38237237 DOI: 10.1016/j.compbiomed.2023.107916] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 12/18/2023] [Accepted: 12/29/2023] [Indexed: 02/28/2024]
Abstract
In the medical field, the application of machine learning technology in the automatic diagnosis and monitoring of osteoporosis often faces challenges related to domain adaptation in drug therapy research. The existing neural networks used for the diagnosis of osteoporosis may experience a decrease in model performance when applied to new data domains due to changes in radiation dose and equipment. To address this issue, in this study, we propose a new method for multi domain diagnostic and quantitative computed tomography (QCT) images, called DeepmdQCT. This method adopts a domain invariant feature strategy and integrates a comprehensive attention mechanism to guide the fusion of global and local features, effectively improving the diagnostic performance of multi domain CT images. We conducted experimental evaluations on a self-created OQCT dataset, and the results showed that for dose domain images, the average accuracy reached 91%, while for device domain images, the accuracy reached 90.5%. our method successfully estimated bone density values, with a fit of 0.95 to the gold standard. Our method not only achieved high accuracy in CT images in the dose and equipment fields, but also successfully estimated key bone density values, which is crucial for evaluating the effectiveness of osteoporosis drug treatment. In addition, we validated the effectiveness of our architecture in feature extraction using three publicly available datasets. We also encourage the application of the DeepmdQCT method to a wider range of medical image analysis fields to improve the performance of multi-domain images.
Collapse
Affiliation(s)
- Kun Zhang
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China; Nantong Key Laboratory of Intelligent Control and Intelligent Computing, Nantong, Jiangsu, 226001, China; Nantong Key Laboratory of Intelligent Medicine Innovation and Transformation, Nantong, Jiangsu, 226001, China
| | - Peng-Cheng Lin
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China
| | - Jing Pan
- Department of Radiology, Affiliated Hospital 2 of Nantong University, Nantong, Jiangsu, 226001, China
| | - Rui Shao
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China
| | - Pei-Xia Xu
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China
| | - Rui Cao
- Department of Radiology, Affiliated Hospital 2 of Nantong University, Nantong, Jiangsu, 226001, China
| | - Cheng-Gang Wu
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China
| | - Danny Crookes
- School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, BT7 1NN, UK
| | - Liang Hua
- School of Electrical Engineering, Nantong University, Nantong, Jiangsu, 226001, China.
| | - Lin Wang
- Department of Radiology, Affiliated Hospital 2 of Nantong University, Nantong, Jiangsu, 226001, China.
| |
Collapse
|
14
|
Modi S, Kasmiran KA, Mohd Sharef N, Sharum MY. Extracting adverse drug events from clinical Notes: A systematic review of approaches used. J Biomed Inform 2024; 151:104603. [PMID: 38331081 DOI: 10.1016/j.jbi.2024.104603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/10/2024]
Abstract
BACKGROUND An adverse drug event (ADE) is any unfavorable effect that occurs due to the use of a drug. Extracting ADEs from unstructured clinical notes is essential to biomedical text extraction research because it helps with pharmacovigilance and patient medication studies. OBJECTIVE From the considerable amount of clinical narrative text, natural language processing (NLP) researchers have developed methods for extracting ADEs and their related attributes. This work presents a systematic review of current methods. METHODOLOGY Two biomedical databases have been searched from June 2022 until December 2023 for relevant publications regarding this review, namely the databases PubMed and Medline. Similarly, we searched the multi-disciplinary databases IEEE Xplore, Scopus, ScienceDirect, and the ACL Anthology. We adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement guidelines and recommendations for reporting systematic reviews in conducting this review. Initially, we obtained 5,537 articles from the search results from the various databases between 2015 and 2023. Based on predefined inclusion and exclusion criteria for article selection, 100 publications have undergone full-text review, of which we consider 82 for our analysis. RESULTS We determined the general pattern for extracting ADEs from clinical notes, with named entity recognition (NER) and relation extraction (RE) being the dual tasks considered. Researchers that tackled both NER and RE simultaneously have approached ADE extraction as a "pipeline extraction" problem (n = 22), as a "joint task extraction" problem (n = 7), and as a "multi-task learning" problem (n = 6), while others have tackled only NER (n = 27) or RE (n = 20). We further grouped the reviews based on the approaches for data extraction, namely rule-based (n = 8), machine learning (n = 11), deep learning (n = 32), comparison of two or more approaches (n = 11), hybrid (n = 12) and large language models (n = 8). The most used datasets are MADE 1.0, TAC 2017 and n2c2 2018. CONCLUSION Extracting ADEs is crucial, especially for pharmacovigilance studies and patient medications. This survey showcases advances in ADE extraction research, approaches, datasets, and state-of-the-art performance in them. Challenges and future research directions are highlighted. We hope this review will guide researchers in gaining background knowledge and developing more innovative ways to address the challenges.
Collapse
Affiliation(s)
- Salisu Modi
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia; Department of Computer Science, Sokoto State University, Sokoto, Nigeria.
| | - Khairul Azhar Kasmiran
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Nurfadhlina Mohd Sharef
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| | - Mohd Yunus Sharum
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Selangor, Malaysia.
| |
Collapse
|
15
|
Tan Z, Madzin H, Norafida B, ChongShuang Y, Sun W, Nie T, Cai F. DeepPulmoTB: A benchmark dataset for multi-task learning of tuberculosis lesions in lung computerized tomography (CT). Heliyon 2024; 10:e25490. [PMID: 38370224 PMCID: PMC10869762 DOI: 10.1016/j.heliyon.2024.e25490] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 01/29/2024] [Indexed: 02/20/2024] Open
Abstract
Tuberculosis (TB) remains a significant global health challenge, characterized by high incidence and mortality rates on a global scale. With the rapid advancement of computer-aided diagnosis (CAD) tools in recent years, CAD has assumed an increasingly crucial role in supporting TB diagnosis. Nonetheless, the development of CAD for TB diagnosis heavily relies on well-annotated computerized tomography (CT) datasets. Currently, the available annotations in TB CT datasets are still limited, which in turn restricts the development of CAD tools for TB diagnosis to some extent. To address this limitation, we introduce DeepPulmoTB, a CT multi-task learning dataset explicitly designed for TB diagnosis. To demonstrate the advantages of DeepPulmoTB, we propose a novel multi-task learning model, DeepPulmoTBNet (DPTBNet), for the joint segmentation and classification of lesion tissues in CT images. The architecture of DPTBNet comprises two subnets: SwinUnetR for the segmentation task, and a lightweight multi-scale network for the classification task. Furthermore, to enhance the model's capacity to capture TB lesion features, we introduce an improved iterative optimization algorithm that refines feature maps by integrating probability maps obtained in previous iterations. Extensive experiments validate the effectiveness of DPTBNet and the practicality of the DeepPulmoTB dataset.
Collapse
Affiliation(s)
- Zhuoyi Tan
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, 43400, Malaysia
| | - Hizmawati Madzin
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, 43400, Malaysia
| | - Bahari Norafida
- Department of Radiology, Universit Putra Malaysia, 43400 Serdang, Selangor, Malaysia
| | - Yang ChongShuang
- Department of Radiology, Universit Putra Malaysia, 43400 Serdang, Selangor, Malaysia
| | - Wei Sun
- Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, 43400, Malaysia
| | - Tianyu Nie
- College of Computer Science, Chongqing University, Chongqing 400030, China
| | - Fengzhou Cai
- Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK
| |
Collapse
|
16
|
Zhang X, Gao L, Wang Z, Yu Y, Zhang Y, Hong J. Improved neural network with multi-task learning for Alzheimer's disease classification. Heliyon 2024; 10:e26405. [PMID: 38434063 PMCID: PMC10906290 DOI: 10.1016/j.heliyon.2024.e26405] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 01/13/2024] [Accepted: 02/13/2024] [Indexed: 03/05/2024] Open
Abstract
Alzheimer's disease(AD) poses a significant challenge due to its widespread prevalence and the lack of effective treatments, highlighting the urgent need for early detection. This research introduces an enhanced neural network, named ADnet, which is based on the VGG16 model, to detect Alzheimer's disease using two-dimensional MRI slices. ADNet incorporates several key improvements: it replaces traditional convolution with depthwise separable convolution to reduce model parameters, replaces the ReLU activation function with ELU to address potential issues with exploding gradients, and integrates the SE(Squeeze-and-Excitation) module to enhance feature extraction efficiency. In addition to the primary task of MRI feature extraction, ADnet is simultaneously trained on two auxiliary tasks: clinical dementia score regression and mental state score regression. Experimental results demonstrate that compared to the baseline VGG16, ADNet achieves a 4.18% accuracy improvement for AD vs. CN classification and a 6% improvement for MCI vs. CN classification. These findings highlight the effectiveness of ADnet in classifying Alzheimer's disease, providing crucial support for early diagnosis and intervention by medical professionals. The proposed enhancements represent advancements in neural network architecture and training strategies for improved AD classification.
Collapse
Affiliation(s)
- Xin Zhang
- School of Electronic and Information Engineering, Wuyi University, Jiangmen, 529000, China
| | - Le Gao
- School of Electronic and Information Engineering, Wuyi University, Jiangmen, 529000, China
| | - Zhimin Wang
- School of Electronic and Information Engineering, Wuyi University, Jiangmen, 529000, China
| | - Yong Yu
- School of Computer Science, Shaanxi Normal University, Xi'an, 710062, China
| | - Yudong Zhang
- School of Computing and Mathematic Sciences, University of Leicester, Leicester, LE17RH, UK
| | - Jin Hong
- School of Information Engineering, Nanchang University, Nanchang, 330031, China
| |
Collapse
|
17
|
Cao K, Zhang T, Huang J. Advanced hybrid LSTM-transformer architecture for real-time multi-task prediction in engineering systems. Sci Rep 2024; 14:4890. [PMID: 38418510 DOI: 10.1038/s41598-024-55483-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 02/23/2024] [Indexed: 03/01/2024] Open
Abstract
In the field of engineering systems-particularly in underground drilling and green stormwater management-real-time predictions are vital for enhancing operational performance, ensuring safety, and increasing efficiency. Addressing this niche, our study introduces a novel LSTM-transformer hybrid architecture, uniquely specialized for multi-task real-time predictions. Building on advancements in attention mechanisms and sequence modeling, our model integrates the core strengths of LSTM and Transformer architectures, offering a superior alternative to traditional predictive models. Further enriched with online learning, our architecture dynamically adapts to variable operational conditions and continuously incorporates new field data. Utilizing knowledge distillation techniques, we efficiently transfer insights from larger, pretrained networks, thereby achieving high predictive accuracy without sacrificing computational resources. Rigorous experiments on sector-specific engineering datasets validate the robustness and effectiveness of our approach. Notably, our model exhibits clear advantages over existing methods in terms of predictive accuracy, real-time adaptability, and computational efficiency. This work contributes a pioneering predictive framework for targeted engineering applications, offering actionable insights into.
Collapse
Affiliation(s)
- Kangjie Cao
- Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China, No. 27 Zhongguancun South Avenue, Beijing, 100081, China
- School of Information and Engineering, Minzu University of China, Beijing, 100081, China
| | - Ting Zhang
- Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China, No. 27 Zhongguancun South Avenue, Beijing, 100081, China.
- School of Information and Engineering, Minzu University of China, Beijing, 100081, China.
| | - Jueqiao Huang
- School of Information and Engineering, Minzu University of China, Beijing, 100081, China
| |
Collapse
|
18
|
Fan L, Gong X, Zheng C, Li J. Data pyramid structure for optimizing EUS-based GISTs diagnosis in multi-center analysis with missing label. Comput Biol Med 2024; 169:107897. [PMID: 38171262 DOI: 10.1016/j.compbiomed.2023.107897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 12/04/2023] [Accepted: 12/23/2023] [Indexed: 01/05/2024]
Abstract
This study introduces the Data Pyramid Structure (DPS) to address data sparsity and missing labels in medical image analysis. The DPS optimizes multi-task learning and enables sustainable expansion of multi-center data analysis. Specifically, It facilitates attribute prediction and malignant tumor diagnosis tasks by implementing a segmentation and aggregation strategy on data with absent attribute labels. To leverage multi-center data, we propose the Unified Ensemble Learning Framework (UELF) and the Unified Federated Learning Framework (UFLF), which incorporate strategies for data transfer and incremental learning in scenarios with missing labels. The proposed method was evaluated on a challenging EUS patient dataset from five centers, achieving promising diagnostic performance. The average accuracy was 0.984 with an AUC of 0.927 for multi-center analysis, surpassing state-of-the-art approaches. The interpretability of the predictions further highlights the potential clinical relevance of our method.
Collapse
Affiliation(s)
- Lin Fan
- School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, Sichuan 611756, China; Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province, China; Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, China; National Engineering Laboratory of Integrated Transportation Big Data Application Technology, China
| | - Xun Gong
- School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, Sichuan 611756, China; Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province, China; Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, China; National Engineering Laboratory of Integrated Transportation Big Data Application Technology, China.
| | - Cenyang Zheng
- School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, Sichuan 611756, China; Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province, China; Engineering Research Center of Sustainable Urban Intelligent Transportation, Ministry of Education, China; National Engineering Laboratory of Integrated Transportation Big Data Application Technology, China
| | - Jiao Li
- Department of Gastroenterology, The Third People's Hospital of Chendu, Affiliated Hospital of Southwest Jiaotong University, Chengdu 610031, China
| |
Collapse
|
19
|
Liu W, Lin C, Chen D, Niu L, Zhang R, Pi Z. Shape-margin knowledge augmented network for thyroid nodule segmentation and diagnosis. Comput Methods Programs Biomed 2024; 244:107999. [PMID: 38194766 DOI: 10.1016/j.cmpb.2023.107999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/11/2023] [Accepted: 12/26/2023] [Indexed: 01/11/2024]
Abstract
BACKGROUND AND OBJECTIVE Thyroid nodule segmentation is a crucial step in the diagnostic procedure of physicians and computer-aided diagnosis systems. However, prevailing studies often treat segmentation and diagnosis as independent tasks, overlooking the intrinsic relationship between these processes. The sequencial steps of these independent tasks in computer-aided diagnosis systems may lead to the accumulation of errors. Therefore, it is worth combining them as a whole by exploring the relationship between thyroid nodule segmentation and diagnosis. According to the diagnostic procedure of thyroid imaging reporting and data system (TI-RADS), the assessment of shape and margin characteristics is the prerequisite for radiologists to discriminate benign and malignant thyroid nodules. Inspired by TI-RADS, this study aims to integrate these tasks into a cohesive process, leveraging the insights from TI-RADS, thereby enhancing the accuracy and interpretability of thyroid nodule analysis. METHODS Specifically, this paper proposes a shape-margin knowledge augmented network (SkaNet) for simultaneous thyroid nodule segmentation and diagnosis. Due to the visual feature similarities between segmentation and diagnosis, SkaNet shares visual features in the feature extraction stage and then utilizes a dual-branch architecture to perform thyroid nodule segmentation and diagnosis tasks respectively. In the shared feature extraction, the combination of convolutional feature maps and self-attention maps allows to exploitation of both local information and global patterns in thyroid nodule images. To enhance effective discriminative features, an exponential mixture module is introduced, combining convolutional feature maps and self-attention maps through exponential weighting. Then, SkaNet is jointly optimized by a knowledge augmented multi-task loss function with a constraint penalty term. The constraint penalty term embeds shape and margin characteristics through numerical computations, establishing a vital relationship between thyroid nodule diagnosis results and segmentation masks. RESULTS We evaluate the proposed approach on a public thyroid ultrasound dataset (DDTI) and a locally collected thyroid ultrasound dataset. The experimental results reveal the value of our contributions and demonstrate that our approach can yield significant improvements compared with state-of-the-art counterparts. CONCLUSIONS SkaNet highlights the potential of combining thyroid nodule segmentation and diagnosis with knowledge augmented learning into a unified framework, which captures the key shape and margin characteristics for discriminating benign and malignant thyroid nodules. Our findings suggest promising insights for advancing computer-aided diagnosis joint with segmentation.
Collapse
Affiliation(s)
- Weihua Liu
- School of Medical Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China; AthenaEyesCO., LTD., Building 14, No. 39 Jianshan Road, Changsha, 410205, Hunan, China.
| | - Chaochao Lin
- School of Computer Science and Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China.
| | - Duanduan Chen
- School of Medical Technology, Beijing Institute of Technology, 5 Zhongguancun South Street, Haidian, 100081, Beijing, China.
| | - Lijuan Niu
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Nanli, Panjiayuan, Chaoyang, 100021, Beijing, China.
| | - Rui Zhang
- Department of Ultrasound, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No. 17 Nanli, Panjiayuan, Chaoyang, 100021, Beijing, China.
| | - Zhaoqiong Pi
- Xiangya School of Medicine, Central South University, No. 172, Tongzipo Road, Changsha, 410083, Hunan, China.
| |
Collapse
|
20
|
Hu J, Tang B, Lyu N, He Y, Xiong Y. CMBEE: A constraint-based multi-task learning framework for biomedical event extraction. J Biomed Inform 2024; 150:104599. [PMID: 38272433 DOI: 10.1016/j.jbi.2024.104599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 12/31/2023] [Accepted: 01/19/2024] [Indexed: 01/27/2024]
Abstract
OBJECTIVE Event extraction plays a crucial role in natural language processing. However, in the biomedical domain, the presence of nested events adds complexity to event extraction compared to single events, and these events usually have strong semantic relationships and constraints. Previous approaches ignored the binding connections between these complex nested events. This study aims to develop a unified framework based on event constraint information that jointly extract biomedical event triggers and arguments and enhance the performance of nested biomedical event extraction. MATERIAL AND METHODS We propose a multi-task learning framework based on constraint information called CMBEE for the task of biomedical event extraction. The N-tuple form of event patterns is used to represent the constrained information, which is integrated into role detection and event type classification tasks. The framework use attention mechanism and gating mechanism to explore the fusion of multiple tuple information, as well as local and global constrained information fusion methods to dig further into the connections between events. RESULTS Experimental results demonstrate that our proposed method achieves the highest F1 score on a multilevel event extraction biomedical (MLEE) corpus and performs favorably on the biomedical natural language processing shared task 2013 Genia event corpus (GE 13). CONCLUSIONS The experimental results indicate that modeling event patterns and constraints for multi-event extraction tasks is effective for complex biomedical event extraction. The fusion strategy proposed in this study, which incorporates different constraint information, helps to better express semantic information.
Collapse
Affiliation(s)
- Jingyue Hu
- Department of Computer Science, Harbin Institute of Technology, Shenzhen, 518055, China
| | - Buzhou Tang
- Department of Computer Science, Harbin Institute of Technology, Shenzhen, 518055, China; Peng Cheng Laboratory, Shenzhen, China.
| | - Nan Lyu
- Beijing Key Laboratory of Mental Disorders, National Clinical Research Center for Mental Disorders & National Center for Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, 100088, China
| | - Yuxin He
- Department of Computer Science, Harbin Institute of Technology, Shenzhen, 518055, China
| | - Ying Xiong
- Department of Computer Science, Harbin Institute of Technology, Shenzhen, 518055, China
| |
Collapse
|
21
|
Liu H, Peng W, Dai W, Lin J, Fu X, Liu L, Liu L, Yu N. Improving anti-cancer drug response prediction using multi-task learning on graph convolutional networks. Methods 2024; 222:41-50. [PMID: 38157919 DOI: 10.1016/j.ymeth.2023.11.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 09/19/2023] [Accepted: 11/19/2023] [Indexed: 01/03/2024] Open
Abstract
Predicting the therapeutic effect of anti-cancer drugs on tumors based on the characteristics of tumors and patients is one of the important contents of precision oncology. Existing computational methods regard the drug response prediction problem as a classification or regression task. However, few of them consider leveraging the relationship between the two tasks. In this work, we propose a Multi-task Interaction Graph Convolutional Network (MTIGCN) for anti-cancer drug response prediction. MTIGCN first utilizes an graph convolutional network-based model to produce embeddings for both cell lines and drugs. After that, the model employs multi-task learning to predict anti-cancer drug response, which involves training the model on three different tasks simultaneously: the main task of the drug sensitive or resistant classification task and the two auxiliary tasks of regression prediction and similarity network reconstruction. By sharing parameters and optimizing the losses of different tasks simultaneously, MTIGCN enhances the feature representation and reduces overfitting. The results of the experiments on two in vitro datasets demonstrated that MTIGCN outperformed seven state-of-the-art baseline methods. Moreover, the well-trained model on the in vitro dataset GDSC exhibited good performance when applied to predict drug responses in in vivo datasets PDX and TCGA. The case study confirmed the model's ability to discover unknown drug responses in cell lines.
Collapse
Affiliation(s)
- Hancheng Liu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China
| | - Wei Peng
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China; Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, Kunming 650050, China.
| | - Wei Dai
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China; Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, Kunming 650050, China.
| | - Jiangzhen Lin
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China
| | - Xiaodong Fu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China; Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, Kunming 650050, China
| | - Li Liu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China; Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, Kunming 650050, China.
| | - Lijun Liu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650050, China; Computer Technology Application Key Lab of Yunnan Province, Kunming University of Science and Technology, Kunming 650050, China
| | - Ning Yu
- State University of New York, The College at Brockport, Department of Computing Sciences, 350 New Campus Drive, Brockport NY 14422.
| |
Collapse
|
22
|
Hervella ÁS, Rouco J, Novo J, Ortega M. Multi-Adaptive Optimization for multi-task learning with deep neural networks. Neural Netw 2024; 170:254-265. [PMID: 37995547 DOI: 10.1016/j.neunet.2023.11.038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 11/16/2023] [Accepted: 11/16/2023] [Indexed: 11/25/2023]
Abstract
Multi-task learning is a promising paradigm to leverage task interrelations during the training of deep neural networks. A key challenge in the training of multi-task networks is to adequately balance the complementary supervisory signals of multiple tasks. In that regard, although several task-balancing approaches have been proposed, they are usually limited by the use of per-task weighting schemes and do not completely address the uneven contribution of the different tasks to the network training. In contrast to classical approaches, we propose a novel Multi-Adaptive Optimization (MAO) strategy that dynamically adjusts the contribution of each task to the training of each individual parameter in the network. This automatically produces a balanced learning across tasks and across parameters, throughout the whole training and for any number of tasks. To validate our proposal, we perform comparative experiments on real-world datasets for computer vision, considering different experimental settings. These experiments allow us to analyze the performance obtained in several multi-task scenarios along with the learning balance across tasks, network layers and training steps. The results demonstrate that MAO outperforms previous task-balancing alternatives. Additionally, the performed analyses provide insights that allow us to comprehend the advantages of this novel approach for multi-task learning.
Collapse
Affiliation(s)
- Álvaro S Hervella
- Centro de Investigación CITIC, Universidade da Coruña, A Coruña, Spain; VARPA Research Group, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, A Coruña, Spain.
| | - José Rouco
- Centro de Investigación CITIC, Universidade da Coruña, A Coruña, Spain; VARPA Research Group, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, A Coruña, Spain
| | - Jorge Novo
- Centro de Investigación CITIC, Universidade da Coruña, A Coruña, Spain; VARPA Research Group, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, A Coruña, Spain
| | - Marcos Ortega
- Centro de Investigación CITIC, Universidade da Coruña, A Coruña, Spain; VARPA Research Group, Instituto de Investigación Biomédica de A Coruña (INIBIC), Universidade da Coruña, A Coruña, Spain
| |
Collapse
|
23
|
Hu D, Liu B, Cheng L, Guo R, Wang J, Lu X, Wu N. A Deep Multi-Task Network to Learn Tumor Pathological Representations for Lymph Node Metastasis Prediction. Stud Health Technol Inform 2024; 310:906-910. [PMID: 38269940 DOI: 10.3233/shti231096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2024]
Abstract
Lymph node metastasis is of paramount importance for patient treatment decision-making, prognosis evaluation, and clinical trial enrollment. However, accurate preoperative diagnosis remains challenging. In this study, we proposed a multi-task network to learn the primary tumor pathological features using the pT stage prediction task and leverage these features to facilitate lymph node metastasis prediction. We conducted experiments using electronic medical record data from 681 patients with non-small cell lung cancer. The proposed method achieved a 0.768 area under the receiver operating characteristic curve (AUC) value with a 0.073 standard deviation (SD) and a 0.448 average precision (AP) value with a 0.113 SD for lymph node metastasis prediction, which significantly outperformed the baseline models. Based on the results, we can conclude that the proposed multi-task method can effectively learn representations about tumor pathological conditions to support lymph node metastasis prediction.
Collapse
Affiliation(s)
- Danqing Hu
- Research Center for Intelligent Computing Software, Zhejiang Lab
| | - Bing Liu
- Department of Thoracic Surgery II, Peking University Cancer Hospital & Institute
| | - Lechao Cheng
- Research Center for Intelligent Computing Software, Zhejiang Lab
| | - Rui Guo
- Research Center for Intelligent Computing Software, Zhejiang Lab
| | - Jin Wang
- Research Center for Intelligent Computing Software, Zhejiang Lab
| | - Xudong Lu
- College of Biomedical Engineering and Instrumental Science, Zhejiang University
| | - Nan Wu
- Department of Thoracic Surgery II, Peking University Cancer Hospital & Institute
| |
Collapse
|
24
|
Shi Z, Lu H, Sun X, Guo J, Ke Y, Wang K, Qin F. Virtual group photography based on scene and person perception. Heliyon 2024; 10:e23568. [PMID: 38187338 PMCID: PMC10770450 DOI: 10.1016/j.heliyon.2023.e23568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 10/10/2023] [Accepted: 12/06/2023] [Indexed: 01/09/2024] Open
Abstract
Group photos have become indispensable in various gathering scenarios, such as family reunions, friends' gatherings, competitions, conferences, store openings, and school graduation ceremonies. The researchers tried automatically adding people who could not participate in the group photo. However, the current research on generating the pose or position of the person by context prediction of the group photo ignores the individual attributes (such as height and body shape) of the target person and does not consider the pose and boundary of the person at the same time. To address these issues, we propose a virtual group photography model that combines the global context of a group photo and the individual attributes of the target person. The model is divided into two stages. The first stage is to predict the person's position, pose, and boundary in the new group photo based on the context of the input group photo and the person's characteristics. The second stage generates new group photos based on the first stage's pose and boundary results. The experimental results show that our method can significantly improve the harmony and authenticity of the synthesis of people in group photos and synthesize the characters that should exist in the group photo, which is very suitable for the field of group photos.
Collapse
Affiliation(s)
- Zhenquan Shi
- School of Software Engineering, Tiangong University, Tianjin, China
| | - Hongming Lu
- School of Computer Science and Technology, Tiangong University, Tianjin, China
- Fitow (Tianjin) Detection Technology Co. Ltd, China
| | - Xinyue Sun
- School of Computer Science and Technology, Tiangong University, Tianjin, China
| | - Jing Guo
- School of Software Engineering, Tiangong University, Tianjin, China
| | - Yongzhen Ke
- School of Computer Science and Technology, Tiangong University, Tianjin, China
- National Demonstration Center for Experimental Engineering Training Education(Tiangong University), Tianjin, China
| | - Kai Wang
- School of Computer Science and Technology, Tiangong University, Tianjin, China
| | - Fan Qin
- Business School, Nankai University, Tianjin, China
| |
Collapse
|
25
|
Zhang S, Luo X, Mai B. Multi-task machine learning models for simultaneous prediction of tissue-to-blood partition coefficients of chemicals in mammals. Environ Res 2024; 241:117603. [PMID: 37939805 DOI: 10.1016/j.envres.2023.117603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 10/25/2023] [Accepted: 11/04/2023] [Indexed: 11/10/2023]
Abstract
Tissue-to-blood partition coefficients (Ptb) are crucial for assessing the distribution of chemicals in organisms. Given the lack of experimental data and laborious nature of experimental methods, there is an urgent need to develop efficient predictive models. With the help of machine learning algorithms, i,e., random forest (RF), and artificial neural network (ANN), this study developed multi-task (MT) models that can simultaneously predict Ptb values for various mammalian tissues, including liver, muscle, brain, lung, and adipose. Single-task (ST) models using partial least squares regression, RF, and ANN algorithms for each endpoint were established for comparison. Overall, the performances of MT models were superior to those of ST models. The MT model using ANN algorithms showed the highest prediction accuracy with determination coefficients ranging from 0.704 to 0.886, root mean square errors between 0.223 and 0.410, and mean absolute errors ranging from 0.178 to 0.285 log units. Results showed that lipophilicity and polarizability of molecules significantly influence their partition behavior in organisms. Applicability domains (ADs) of the models were characterized by weighted molecular similarity density, and weighted inconsistency in molecular activities of structure-activity landscapes. When constrained by ADs, the models displayed enhanced predictive accuracy, making them valuable tools for the risk assessment and management of chemicals.
Collapse
Affiliation(s)
- Shuying Zhang
- State Key Laboratory of Organic Geochemistry and Guangdong Key Laboratory of Environmental Resources Utilization and Protection, Guangzhou Institute of Geochemistry, Chinese Academy of Sciences, Guangzhou, 510640, China
| | - Xiaojun Luo
- State Key Laboratory of Organic Geochemistry and Guangdong Key Laboratory of Environmental Resources Utilization and Protection, Guangzhou Institute of Geochemistry, Chinese Academy of Sciences, Guangzhou, 510640, China.
| | - Bixian Mai
- State Key Laboratory of Organic Geochemistry and Guangdong Key Laboratory of Environmental Resources Utilization and Protection, Guangzhou Institute of Geochemistry, Chinese Academy of Sciences, Guangzhou, 510640, China
| |
Collapse
|
26
|
Li C, Jiang X, Zhang K. A transformer-based deep learning approach for fairly predicting post-liver transplant risk factors. J Biomed Inform 2024; 149:104545. [PMID: 37992791 DOI: 10.1016/j.jbi.2023.104545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 10/11/2023] [Accepted: 11/13/2023] [Indexed: 11/24/2023]
Abstract
Liver transplantation is a life-saving procedure for patients with end-stage liver disease. There are two main challenges in liver transplant: finding the best matching patient for a donor and ensuring transplant equity among different subpopulations. The current MELD scoring system evaluates a patient's mortality risk if not receiving an organ within 90 days. However, the donor-patient matching should also consider post-transplant risk factors, such as cardiovascular disease, chronic rejection, etc., which are all common complications after transplant. Accurate prediction of these risk scores remains a significant challenge. In this study, we used predictive models to solve the above challenges. Specifically, we proposed a deep learning model to predict multiple risk factors after a liver transplant. By formulating it as a multi-task learning problem, the proposed deep neural network was trained to simultaneously predict the five post-transplant risks and achieve equal good performance by exploiting task-balancing techniques. We also proposed a novel fairness-achieving algorithm to ensure prediction fairness across different subpopulations. We used electronic health records of 160,360 liver transplant patients, including demographic information, clinical variables, and laboratory values, collected from the liver transplant records of the United States from 1987 to 2018. The model's performance was evaluated using various performance metrics such as AUROC and AUPRC. Our experiment results highlighted the success of our multi-task model in achieving task balance while maintaining accuracy. The model significantly reduced the task discrepancy by 39 %. Further application of the fairness-achieving algorithm substantially reduced fairness disparity among all sensitive attributes (gender, age group, and race/ethnicity) in each risk factor. It underlined the potency of integrating fairness considerations into the task-balancing framework, ensuring robust and fair predictions across multiple tasks and diverse demographic groups.
Collapse
Affiliation(s)
- Can Li
- Department of Biostatistics and Data Science, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Xiaoqian Jiang
- Department of Health Data Science and Artificial Intelligence, McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA
| | - Kai Zhang
- Department of Health Data Science and Artificial Intelligence, McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA.
| |
Collapse
|
27
|
Tuan TA, Hoang LP, Le DD, Thang TN. A framework for controllable Pareto front learning with completed scalarization functions and its applications. Neural Netw 2024; 169:257-273. [PMID: 37913657 DOI: 10.1016/j.neunet.2023.10.029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 08/14/2023] [Accepted: 10/20/2023] [Indexed: 11/03/2023]
Abstract
Pareto Front Learning (PFL) was recently introduced as an efficient method for approximating the entire Pareto front, the set of all optimal solutions to a Multi-Objective Optimization (MOO) problem. In the previous work, the mapping between a preference vector and a Pareto optimal solution is still ambiguous, rendering its results. This study demonstrates the convergence and completion aspects of solving MOO with pseudoconvex scalarization functions and combines them into Hypernetwork in order to offer a comprehensive framework for PFL, called Controllable Pareto Front Learning. Extensive experiments demonstrate that our approach is highly accurate and significantly less computationally expensive than prior methods in term of inference time.
Collapse
Affiliation(s)
- Tran Anh Tuan
- School of Applied Mathematics and Informatics, Hanoi University of Science and Technology, Ha Noi, Viet Nam.
| | - Long P Hoang
- College of Engineering and Computer Science, VinUniversity, Ha Noi, Viet Nam.
| | - Dung D Le
- College of Engineering and Computer Science, VinUniversity, Ha Noi, Viet Nam.
| | - Tran Ngoc Thang
- School of Applied Mathematics and Informatics, Hanoi University of Science and Technology, Ha Noi, Viet Nam.
| |
Collapse
|
28
|
Wu X, Tan G, Luo H, Chen Z, Pu B, Li S, Li K. A knowledge-interpretable multi-task learning framework for automated thyroid nodule diagnosis in ultrasound videos. Med Image Anal 2024; 91:103039. [PMID: 37992495 DOI: 10.1016/j.media.2023.103039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Revised: 10/14/2023] [Accepted: 11/15/2023] [Indexed: 11/24/2023]
Abstract
Ultrasound has become the most widely used modality for thyroid nodule diagnosis, due to its portability, real-time feedback, lack of toxicity, and low cost. Recently, the computer-aided diagnosis (CAD) of thyroid nodules has attracted significant attention. However, most existing techniques can only be applied to either static images with prominent features (manually selected from scanning videos) or rely on 'black boxes' that cannot provide interpretable results. In this study, we develop a user-friendly framework for the automated diagnosis of thyroid nodules in ultrasound videos, by simulating the typical diagnostic workflow used by radiologists. This process consists of two orderly part-to-whole tasks. The first interprets the characteristics of each image using prior knowledge, to obtain corresponding frame-wise TI-RADS scores. Associated embedded representations not only provide diagnostic information for radiologists but also reduce computational costs. The second task models temporal contextual information in an embedding vector sequence and selectively enhances important information to distinguish benign and malignant thyroid nodules, thereby improving the efficiency and generalizability of the proposed framework. Experimental results demonstrated this approach outperformed other state-of-the-art video classification methods. In addition to assisting radiologists in understanding model predictions, these CAD results could further ease diagnostic workloads and improve patient care.
Collapse
Affiliation(s)
- Xiangqiong Wu
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China
| | - Guanghua Tan
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China.
| | - Hongxia Luo
- Department of Ultrasonic Diagnosis, The Second Affiliated Hospital and Yuying Children's Hospital, Wenzhou Medical University, Wenzhou, 325027, China
| | - Zhilun Chen
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China
| | - Bin Pu
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China
| | - Shengli Li
- Shenzhen Maternity and child Healthcare Hospital, Southern Medical University, Shenzhen, 518028, China
| | - Kenli Li
- College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China.
| |
Collapse
|
29
|
Dong H, Chen D, Chen Y, Tang Y, Yin D, Li X. A multi-task learning model with reinforcement optimization for ASD comorbidity discrimination. Comput Methods Programs Biomed 2024; 243:107865. [PMID: 37883824 DOI: 10.1016/j.cmpb.2023.107865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Revised: 10/11/2023] [Accepted: 10/11/2023] [Indexed: 10/28/2023]
Abstract
How to discriminate the comorbidities in autism spectrum disorder (ASD) population has long been an intriguing and challenging issue in neuroscience and neurology practices. Taking attention deficit hyperactivity disorder (ADHD) for example, electroencephalogram (EEG) analysis has alleviated the problem caused by the task of evaluation of similar behaviors of subjects with ASD, ADHD and ASD+ADHD, which requires a very high expertise to reach any concrete conclusions. However, the performance of ASD comorbidity discrimination is still limited by two major difficulties 1) crucial EEG features regarding ASD and ASD+ADHD largely overlap, and 2) reliable data for model training are routinely insufficient. This study proposes a multi-task learning method with "reinforcement optimization" (namely RO-MLT) working in a two-fold manner: 1)Modeling for Discrimination: a multi-task CNN model maintains the target discrimination task (ASD vs. ASD+ADHD) with the aid of the auxiliary task (ASD vs. Typically Developed (TD)), which is designed to mitigate the aforementioned difficulties on model training; and 2) Reinforcement Optimization: a reinforcement learning algorithm enhances the model's feature extraction and fusion capabilities by optimizing its shared structure. Experimental results based on resting-state EEG that collected from 150 ASD, ASD+ADHD or TD children with the RO-MLT method against the state-of-the-art counterparts indicate that RO-MLT is far superior in terms of all performance indicators (e.g., accuracy). Ablation experiments also show that introduction of multi-task learning and reinforcement optimization can achieve a performance boost-up by 11.07%, a gain even higher than the sums of introduction of two individual techniques to the model design.
Collapse
Affiliation(s)
- Heyou Dong
- School of Computer Science, Wuhan University, Wuhan, 430072, China
| | - Dan Chen
- School of Computer Science, Wuhan University, Wuhan, 430072, China.
| | - Yukang Chen
- School of Computer Science, Wuhan University, Wuhan, 430072, China
| | - Yunbo Tang
- School of Computer Science, Wuhan University, Wuhan, 430072, China
| | - Dingze Yin
- School of Computer Science, Wuhan University, Wuhan, 430072, China
| | - Xiaoli Li
- National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, 100875, China
| |
Collapse
|
30
|
Huang M, Jiang Z, Guo S. Phar-LSTM: a pharmacological representation-based LSTM network for drug-drug interaction extraction. PeerJ 2023; 11:e16606. [PMID: 38107590 PMCID: PMC10725669 DOI: 10.7717/peerj.16606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 11/15/2023] [Indexed: 12/19/2023] Open
Abstract
Pharmacological drug interactions are among the most common causes of medication errors. Many different methods have been proposed to extract drug-drug interactions from the literature to reduce medication errors over the last few years. However, the performance of these methods can be further improved. In this paper, we present a Pharmacological representation-based Long Short-Term Memory (LSTM) network named Phar-LSTM. In this method, a novel embedding strategy is proposed to extract pharmacological representations from the biomedical literature, and the information related to the target drug is considered. Then, an LSTM-based multi-task learning scheme is introduced to extract features from the different but related tasks according to their corresponding pharmacological representations. Finally, the extracted features are fed to the SoftMax classifier of the corresponding task. Experimental results on the DDIExtraction 2011 and DDIExtraction 2013 corpuses show that the performance of Phar-LSTM is competitive compared with other state-of-the-art methods. Our Python implementation and the corresponding data of Phar-LSTM are available by using the DOI 10.5281/zenodo.8249384.
Collapse
Affiliation(s)
- Mingqing Huang
- School of Software Engineering, Shenzhen Institute of Information Technology, Shenzhen, Guangdong, China
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China
| | - Zhenchao Jiang
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China
| | - Shun Guo
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, Guangdong, China
| |
Collapse
|
31
|
Lu X, Du J, Wang G, Li X, Sun L, Zheng L, Huang X. Identifying multiple soil pollutions of potentially contaminated sites based on multi-gate mixture-of-experts network. Sci Total Environ 2023; 903:166218. [PMID: 37572924 DOI: 10.1016/j.scitotenv.2023.166218] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 05/27/2023] [Accepted: 08/08/2023] [Indexed: 08/14/2023]
Abstract
With the rapid increase in the amount and sources of big data, using big data and machine learning methods to identify site soil pollution has become a research hotspot. However, previous studies that used basic information of sites as pollution identification indexes mainly have problems of low accuracy and efficiency when conducting complex model predictions for multiple soil pollution types. In this study, we collected the environmental data of 199 sites in 6 typical industries involving heavy metal and organic pollution. After feature fusion and selection, 10 indexes based on pollution sources and pathways were used to establish the soil pollution identification index system. The Multi-gate Mixture-of-Experts network (MMoE) were constructed to carry out the multi-tasks of soil heavy metals, VOCs and SVOCs pollution identification simultaneously. The SHAP framework was used to reveal the importance of pollution identification indexes on the multiple outputs of MMoE and obtain their driving factors. The results showed that the accuracies of MMoE model were 0.600, 0.783 and 0.850 for soil heavy metals, VOCs and SVOCs pollution identifications, respectively, which were 0-20 % higher than their accuracies of BP neural networks of single tasks. The indexes of raw material containing organic compounds, enterprise scale, soil pollution traces and industry types have the different significant importance on site soil pollutions. This study proposed a more efficient and accurate method to identify site soil pollutions and their driving factors, which offers a step towards realizing intelligent identification and risk control of site soil pollution globally.
Collapse
Affiliation(s)
- Xiaosong Lu
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China
| | - Junyang Du
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China
| | - Guoqing Wang
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China.
| | - Xuzhi Li
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China
| | - Li Sun
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China
| | - Liping Zheng
- State Environmental Protection Key Laboratory of Soil Environmental Management and Pollution Control, Nanjing Institute of Environmental Sciences, Ministry of Ecology and Environment, Nanjing 210042, China
| | - Xinghua Huang
- College of Environmental Science and Engineering, Yangzhou University, Yangzhou 225127, China
| |
Collapse
|
32
|
Xia J, Chen N, Qiu A. Multi-level and joint attention networks on brain functional connectivity for cross-cognitive prediction. Med Image Anal 2023; 90:102921. [PMID: 37666116 DOI: 10.1016/j.media.2023.102921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 05/15/2023] [Accepted: 07/31/2023] [Indexed: 09/06/2023]
Abstract
Deep learning on resting-state functional MRI (rs-fMRI) has shown great success in predicting a single cognition or mental disease. Nevertheless, cognitive functions or mental diseases may share neural mechanisms that can benefit their prediction/classification. We propose a multi-level and joint attention (ML-Joint-Att) network to learn high-order representations of brain functional connectivities that are specific and shared across multiple tasks. We design the ML-Joint-Att network with edge and node convolutional operators, an adaptive inception module, and three attention modules, including network-wise, region-wise, and region-wise joint attention modules. The adaptive inception learns brain functional connectivity at multiple spatial scales. The network-wise and region-wise attention modules take the multi-scale functional connectivities as input and learn features at the network and regional levels for individual tasks. Moreover, the joint attention module is designed as region-wise joint attention to learn shared brain features that contribute to and compensate for the prediction of multiple tasks. We employed the Adolescent Brain Cognitive Development (ABCD) dataset (n =9092) to evaluate the ML-Joint-Att network for the prediction of cognitive flexibility and inhibition. Our experiments demonstrated the usefulness of the three attention modules and identified brain functional connectivities and regions specific and common between cognitive flexibility and inhibition. In particular, the joint attention module can significantly improve the prediction of both cognitive functions. Moreover, leave-one-site cross-validation showed that the ML-Joint-Att network is robust to independent samples obtained from different sites of the ABCD study. Our network outperformed existing machine learning techniques, including Brain Bias Set (BBS), spatio-temporal graph convolution network (ST-GCN), and BrainNetCNN. We demonstrated the generalization of our method to other applications, such as the prediction of fluid intelligence and crystallized intelligence, which also outperformed the ST-GCN and BrainNetCNN.
Collapse
Affiliation(s)
- Jing Xia
- Department of Biomedical Engineering, National University of Singapore, Singapore
| | - Nanguang Chen
- Department of Biomedical Engineering, National University of Singapore, Singapore
| | - Anqi Qiu
- Department of Biomedical Engineering, National University of Singapore, Singapore; The N.1 Institute for Health, National University of Singapore, Singapore; NUS (Suzhou) Research Institute, National University of Singapore, China; Institute of Data Science, National University of Singapore, Singapore; Department of Health Technology and Informatics, Hong Kong Polytechnic University, Hong Kong; Department of Biomedical Engineering, the Johns Hopkins University, USA.
| |
Collapse
|
33
|
Farooq MU, Ullah Z, Khan A, Gwak J. DC-AAE: Dual channel adversarial autoencoder with multitask learning for KL-grade classification in knee radiographs. Comput Biol Med 2023; 167:107570. [PMID: 37897960 DOI: 10.1016/j.compbiomed.2023.107570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Revised: 08/25/2023] [Accepted: 10/10/2023] [Indexed: 10/30/2023]
Abstract
Knee osteoarthritis (OA) is a frequent musculoskeletal disorder that leads to physical disability in older adults. Manual OA assessment is performed via visual inspection, which is highly subjective as it suffers from moderate to high inter-observer variability. Many deep learning-based techniques have been proposed to address this issue. However, owing to the limited amount of labelled data, all existing solutions have limitations in terms of performance or the number of classes. This paper proposes a novel fully automatic Kellgren and Lawrence (KL) grade classification scheme in knee radiographs. We developed a semi-supervised multi-task learning-based approach that enables the exploitation of additional unlabelled data in an unsupervised as well as supervised manner. Specifically, we propose a dual-channel adversarial autoencoder, which is first trained in an unsupervised manner for reconstruction tasks only. To exploit the additional data in a supervised way, we propose a multi-task learning framework by introducing an auxiliary task. In particular, we use leg side identification as an auxiliary task, which allows the use of more datasets, e.g., CHECK dataset. The work demonstrates that the utilization of additional data can improve the primary task of KL-grade classification for which only limited labelled data is available. This semi-supervised learning essentially helps to improve the feature learning ability of our framework, which leads to improved performance for KL-grade classification. We rigorously evaluated our proposed model on the two largest publicly available datasets for various aspects, i.e., overall performance, the effect of additional unlabelled samples and auxiliary tasks, robustness analysis, and ablation study. The proposed model achieved the accuracy, precision, recall, and F1 score of 75.53%, 74.1%, 78.51%, and 75.34%, respectively. Furthermore, the experimental results show that the suggested model not only achieves state-of-the-art performance on two publicly available datasets but also exhibits remarkable robustness.
Collapse
Affiliation(s)
- Muhammad Umar Farooq
- Department of IT, Energy Convergence (BK21 FOUR), Korea National University of Transportation, Chungju 27469, South Korea
| | - Zahid Ullah
- Department of Software, Korea National University of Transportation, Chungju 27469, South Korea
| | - Asifullah Khan
- Pattern Recognition Lab, DCIS, PIEAS, Nilore, Islamabad 45650, Pakistan
| | - Jeonghwan Gwak
- Department of IT, Energy Convergence (BK21 FOUR), Korea National University of Transportation, Chungju 27469, South Korea; Department of Software, Korea National University of Transportation, Chungju 27469, South Korea; Department of Biomedical Engineering, Korea National University of Transportation, Chungju 27469, South Korea; Department of AI Robotics Engineering, Korea National University of Transportation, Chungju 27469, South Korea.
| |
Collapse
|
34
|
Men Q, Teng C, Drukker L, Papageorghiou AT, Noble JA. Gaze-probe joint guidance with multi-task learning in obstetric ultrasound scanning. Med Image Anal 2023; 90:102981. [PMID: 37863638 PMCID: PMC7615231 DOI: 10.1016/j.media.2023.102981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 08/31/2023] [Accepted: 09/26/2023] [Indexed: 10/22/2023]
Abstract
In this work, we exploit multi-task learning to jointly predict the two decision-making processes of gaze movement and probe manipulation that an experienced sonographer would perform in routine obstetric scanning. A multimodal guidance framework, Multimodal-GuideNet, is proposed to detect the causal relationship between a real-world ultrasound video signal, synchronized gaze, and probe motion. The association between the multi-modality inputs is learned and shared through a modality-aware spatial graph that leverages useful cross-modal dependencies. By estimating the probability distribution of probe and gaze movements in real scans, the predicted guidance signals also allow inter- and intra-sonographer variations and avoid a fixed scanning path. We validate the new multi-modality approach on three types of obstetric scanning examinations, and the result consistently outperforms single-task learning under various guidance policies. To simulate sonographer's attention on multi-structure images, we also explore multi-step estimation in gaze guidance, and its visual results show that the prediction allows multiple gaze centers that are substantially aligned with underlying anatomical structures.
Collapse
Affiliation(s)
- Qianhui Men
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom.
| | - Clare Teng
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom
| | - Lior Drukker
- Nuffield Department of Women's & Reproductive Health, University of Oxford, Oxford, OX3 9DU, United Kingdom; Department of Obstetrics and Gynecology, Tel-Aviv University, Tel Aviv, Ramat Aviv, 69978, Israel
| | - Aris T Papageorghiou
- Nuffield Department of Women's & Reproductive Health, University of Oxford, Oxford, OX3 9DU, United Kingdom
| | - J Alison Noble
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, OX3 7DQ, United Kingdom
| |
Collapse
|
35
|
Wang Y, Lam HK, Xu Y, Yin F, Qian K. Multi-task learning framework to predict the status of central venous catheter based on radiographs. Artif Intell Med 2023; 146:102721. [PMID: 38042594 DOI: 10.1016/j.artmed.2023.102721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Revised: 09/29/2023] [Accepted: 11/14/2023] [Indexed: 12/04/2023]
Abstract
Hospital patients can have catheters and lines inserted during the course of their admission to give medicines for the treatment of medical issues, especially the central venous catheter (CVC). However, malposition of CVC will lead to many complications, even death. Clinicians always detect the status of the catheter to avoid the above issues via X-ray images. To reduce the workload of clinicians and improve the efficiency of CVC status detection, a multi-task learning framework for catheter status classification based on the convolutional neural network (CNN) is proposed. The proposed framework contains three significant components which are modified HRNet, multi-task supervision including segmentation supervision and heatmap regression supervision as well as classification branch. The modified HRNet maintaining high-resolution features from the start to the end can ensure to generation of high-quality assisted information for classification. The multi-task supervision can assist in alleviating the presence of other line-like structures such as other tubes and anatomical structures shown in the X-ray image. Furthermore, during the inference, this module is also considered as an interpretation interface to show where the framework pays attention to. Eventually, the classification branch is proposed to predict the class of the status of the catheter. A public CVC dataset is utilized to evaluate the performance of the proposed method, which gains 0.823 AUC (Area under the ROC curve) and 82.6% accuracy in the test dataset. Compared with two state-of-the-art methods (ATCM method and EDMC method), the proposed method can perform best.
Collapse
Affiliation(s)
- Yuhan Wang
- Department of Engineering, King's College London, Strand, London, WC2R 2LS, United Kingdom
| | - Hak Keung Lam
- Department of Engineering, King's College London, Strand, London, WC2R 2LS, United Kingdom.
| | - Yujia Xu
- Department of Engineering, King's College London, Strand, London, WC2R 2LS, United Kingdom
| | - Faliang Yin
- Department of Engineering, King's College London, Strand, London, WC2R 2LS, United Kingdom
| | - Kun Qian
- Center for the Developing Brain, School of Biomedical Engineering and Imaging Sciences, King's College London, St Thomas' Campus, St Thomas' Hospital, Westminster Bridge Road, London, SE1 7EH, United Kingdom
| |
Collapse
|
36
|
Fan Z, Gong P, Tang S, Lee CU, Zhang X, Song P, Chen S, Li H. Joint localization and classification of breast masses on ultrasound images using an auxiliary attention-based framework. Med Image Anal 2023; 90:102960. [PMID: 37769552 PMCID: PMC10638858 DOI: 10.1016/j.media.2023.102960] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Revised: 09/01/2023] [Accepted: 09/05/2023] [Indexed: 10/03/2023]
Abstract
Multi-task learning (MTL) methods have been extensively employed for joint localization and classification of breast lesions on ultrasound images to assist in cancer diagnosis and personalized treatment. One typical paradigm in MTL is a shared trunk network architecture. However, such a model design may suffer information-sharing conflicts and only achieve suboptimal performance for individual tasks. Additionally, the model relies on fully-supervised learning methodologies, imposing heavy burdens on data annotation. In this study, we propose a novel joint localization and classification model based on attention mechanisms and a sequential semi-supervised learning strategy to address these challenges. Our proposed framework offers three primary advantages. First, a lesion-aware network with multiple attention modules is designed to improve model performance on lesion localization. An attention-based classifier explicitly establishes correlations between the two tasks, alleviating information-sharing conflicts while leveraging location information to assist in classification. Second, a two-stage sequential semi-supervised learning strategy is designed for model training to achieve optimal performance on both tasks and substantially reduces the need for data annotation. Third, the asymmetric and modular model architecture allows for the flexible interchangeability of individual components, rendering the model adaptable to various applications. Experimental results from two different breast ultrasound image datasets under varied conditions have demonstrated the effectiveness of the proposed method. Furthermore, we conduct comprehensive investigations into the impacts of various factors on model performance, gaining in-depth insights into the mechanism of our proposed framework. The code is available at https://github.com/comp-imaging-sci/lanet-bus.git.
Collapse
Affiliation(s)
- Zong Fan
- Department of Bioengineering, University of Illinois Urbana-Champaign, IL, USA
| | - Ping Gong
- Department of Radiology, Mayo Clinic College of Medicine and Science, Rochester, MN, USA
| | - Shanshan Tang
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, TX, USA
| | - Christine U Lee
- Department of Radiology, Mayo Clinic College of Medicine and Science, Rochester, MN, USA
| | - Xiaohui Zhang
- Department of Bioengineering, University of Illinois Urbana-Champaign, IL, USA
| | - Pengfei Song
- Department of Bioengineering, University of Illinois Urbana-Champaign, IL, USA; Department of Elect. & Computer Eng., University of Illinois Urbana-Champaign, IL, USA; Cancer Center at Illinois, Urbana, IL, USA; Beckman Institute, University of Illinois Urbana-Champaign, IL, USA
| | - Shigao Chen
- Department of Radiology, Mayo Clinic College of Medicine and Science, Rochester, MN, USA
| | - Hua Li
- Department of Bioengineering, University of Illinois Urbana-Champaign, IL, USA; Department of Radiation Oncology, Washington University in St. Louis, MO, USA; Cancer Center at Illinois, Urbana, IL, USA; Beckman Institute, University of Illinois Urbana-Champaign, IL, USA.
| |
Collapse
|
37
|
Chen D, Wang X, Zhu H, Jiang Y, Li Y, Liu Q, Liu Q. Predicting anticancer synergistic drug combinations based on multi-task learning. BMC Bioinformatics 2023; 24:448. [PMID: 38012551 PMCID: PMC10680313 DOI: 10.1186/s12859-023-05524-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 10/09/2023] [Indexed: 11/29/2023] Open
Abstract
BACKGROUND The discovery of anticancer drug combinations is a crucial work of anticancer treatment. In recent years, pre-screening drug combinations with synergistic effects in a large-scale search space adopting computational methods, especially deep learning methods, is increasingly popular with researchers. Although achievements have been made to predict anticancer synergistic drug combinations based on deep learning, the application of multi-task learning in this field is relatively rare. The successful practice of multi-task learning in various fields shows that it can effectively learn multiple tasks jointly and improve the performance of all the tasks. METHODS In this paper, we propose MTLSynergy which is based on multi-task learning and deep neural networks to predict synergistic anticancer drug combinations. It simultaneously learns two crucial prediction tasks in anticancer treatment, which are synergy prediction of drug combinations and sensitivity prediction of monotherapy. And MTLSynergy integrates the classification and regression of prediction tasks into the same model. Moreover, autoencoders are employed to reduce the dimensions of input features. RESULTS Compared with the previous methods listed in this paper, MTLSynergy achieves the lowest mean square error of 216.47 and the highest Pearson correlation coefficient of 0.76 on the drug synergy prediction task. On the corresponding classification task, the area under the receiver operator characteristics curve and the area under the precision-recall curve are 0.90 and 0.62, respectively, which are equivalent to the comparison methods. Through the ablation study, we verify that multi-task learning and autoencoder both have a positive effect on prediction performance. In addition, the prediction results of MTLSynergy in many cases are also consistent with previous studies. CONCLUSION Our study suggests that multi-task learning is significantly beneficial for both drug synergy prediction and monotherapy sensitivity prediction when combining these two tasks into one model. The ability of MTLSynergy to discover new anticancer synergistic drug combinations noteworthily outperforms other state-of-the-art methods. MTLSynergy promises to be a powerful tool to pre-screen anticancer synergistic drug combinations.
Collapse
Affiliation(s)
- Danyi Chen
- School of Software Engineering, Tongji University, Shanghai, 201804, China
| | - Xiaowen Wang
- School of Software Engineering, Tongji University, Shanghai, 201804, China
| | - Hongming Zhu
- School of Software Engineering, Tongji University, Shanghai, 201804, China
| | - Yizhi Jiang
- School of Software Engineering, Tongji University, Shanghai, 201804, China
| | - Yulong Li
- School of Software Engineering, Tongji University, Shanghai, 201804, China
| | - Qi Liu
- Translational Medical Center for Stem Cell Therapy and Institute for Regenerative Medicine, Shanghai East Hospital, Bioinformatics Department, School of Life Sciences and Technology, Tongji University, Shanghai, 200092, China.
| | - Qin Liu
- School of Software Engineering, Tongji University, Shanghai, 201804, China.
| |
Collapse
|
38
|
Zhang S, Liu X, Zhou L, Wang K, Shao J, Shi J, Wang X, Mu J, Gao T, Jiang Z, Chen K, Wang C, Wang G. Intelligent prognosis evaluation system for stage I-III resected non-small-cell lung cancer patients on CT images: a multi-center study. EClinicalMedicine 2023; 65:102270. [PMID: 38106558 PMCID: PMC10725055 DOI: 10.1016/j.eclinm.2023.102270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 09/27/2023] [Accepted: 09/27/2023] [Indexed: 12/19/2023] Open
Abstract
Background Prognosis is crucial for personalized treatment and surveillance suggestion of the resected non-small-cell lung cancer (NSCLC) patients in stage I-III. Although the tumor-node-metastasis (TNM) staging system is a powerful predictor, it is not perfect enough to accurately distinguish all the patients, especially within the same TNM stage. In this study, we developed an intelligent prognosis evaluation system (IPES) using pre-therapy CT images to assist the traditional TNM staging system for more accurate prognosis prediction of resected NSCLC patients. Methods 20,333 CT images of 6371 patients from June 12, 2009 to March 24, 2022 in West China Hospital of Sichuan University, Mianzhu People's Hospital, Peking University People's Hospital, Chengdu Shangjin Nanfu Hospital and Guangan Peoples' Hospital were included in this retrospective study. We developed the IPES based on self-supervised pre-training and multi-task learning, which aimed to predict an overall survival (OS) risk for each patient. We further evaluated the prognostic accuracy of the IPES and its ability to stratify NSCLC patients with the same TNM stage and with the same EGFR genotype. Findings The IPES was able to predict OS risk for stage I-III resected NSCLC patients in the training set (C-index 0.806; 95% CI: 0.744-0.846), internal validation set (0.783; 95% CI: 0.744-0.825) and external validation set (0.817; 95% CI: 0.786-0.849). In addition, IPES performed well in early-stage (stage I) and EGFR genotype prediction. Furthermore, by adopting IPES-based survival score (IPES-score), resected NSCLC patients in the same stage or with the same EGFR genotype could be divided into low- and high-risk subgroups with good and poor prognosis, respectively (p < 0.05 for all). Interpretation The IPES provided a non-invasive way to obtain prognosis-related information from patients. The identification of IPES for resected NSCLC patients with low and high prognostic risk in the same TNM stage or with the same EGFR genotype suggests that IPES have potential to offer more personalized treatment and surveillance suggestion for NSCLC patients. Funding This study was funded by the National Natural Science Foundation of China (grant 62272055, 92259303, 92059203), New Cornerstone Science Foundation through the XPLORER PRIZE, Young Elite Scientists Sponsorship Program by CAST (2021QNRC001), Clinical Medicine Plus X - Young Scholars Project, Peking University, the Fundamental Research Funds for the Central Universities (K.C.), Research Unit of Intelligence Diagnosis and Treatment in Early Non-small Cell Lung Cancer, Chinese Academy of Medical Sciences (2021RU002), BUPT Excellent Ph.D. Students Foundation (CX2022104).
Collapse
Affiliation(s)
- Siqi Zhang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Xiaohong Liu
- UCL Cancer Institute, University College London, London, WC1E 6DD, UK
| | - Lixin Zhou
- Thoracic Oncology Institute and Department of Thoracic Surgery, Peking University People's Hospital, Beijing, 100044, China
| | - Kai Wang
- College of Future Technology, Peking University and Peking-Tsinghua Center for Life Sciences, Beijing, 100871, China
| | - Jun Shao
- State Key Laboratory of Respiratory Health and Multimorbidity, Department of Pulmonary and Critical Care Medicine, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, 610041, China
| | - Jianyu Shi
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Xuan Wang
- Thoracic Oncology Institute and Department of Thoracic Surgery, Peking University People's Hospital, Beijing, 100044, China
| | - Jiaxing Mu
- Thoracic Oncology Institute and Department of Thoracic Surgery, Peking University People's Hospital, Beijing, 100044, China
| | - Tianrun Gao
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Zeyu Jiang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| | - Kezhong Chen
- Thoracic Oncology Institute and Department of Thoracic Surgery, Peking University People's Hospital, Beijing, 100044, China
| | - Chengdi Wang
- State Key Laboratory of Respiratory Health and Multimorbidity, Department of Pulmonary and Critical Care Medicine, Frontiers Science Center for Disease-related Molecular Network, West China Hospital, Sichuan University, Chengdu, 610041, China
| | - Guangyu Wang
- State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China
| |
Collapse
|
39
|
Fu Y, Jiang L, Pan S, Chen P, Wang X, Dai N, Chen X, Xu M. Deep multi-task learning for nephropathy diagnosis on immunofluorescence images. Comput Methods Programs Biomed 2023; 241:107747. [PMID: 37619430 DOI: 10.1016/j.cmpb.2023.107747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 06/14/2023] [Accepted: 08/03/2023] [Indexed: 08/26/2023]
Abstract
BACKGROUND AND OBJECTIVE As an advanced technique, immunofluorescence (IF) is one of the most widely-used medical image for nephropathy diagnosis, due to its ease of acquisition with low cost. In practice, the clinically collected IF images are commonly corrupted by blurs at different degrees, mainly because of the inaccurate focus at the acquisition stage. Although deep neural network (DNN) methods achieve the great success in nephropathy diagnosis, their performance dramatically drops over the blurred IF images. This significantly limits the potential of leveraging the advanced DNN techniques in real-world nephropathy diagnosis scenarios. METHODS This paper first establishes two IF databases with synthetic blurs (IFVB) and real-world blurs (Real-IF) for nephropathy diagnosis, respectively, including 1,659 patients and 6,521 IF images with various degrees of blurs. According to the analysis on these two databases, we propose a deep hierarchical multi-task learning based nephropathy diagnosis (DeepMT-ND) method to bridge the gap between the low-level vision and high-level medical tasks. Specifically, DeepMT-ND simultaneously handles the main task of automatic nephropathy diagnosis, as well as the auxiliary tasks of image quality assessment (IQA) and de-blurring. RESULTS Extensive experiments show the superiority of our DeepMT-ND in terms of diagnosis accuracy and generalization ability. For instance, our method performs better than nephrologists with at least 15.4% and 6.5% accuracy improvements in IFVB and Real-IF, respectively. Meanwhile, our method also achieves comparable performance in two auxiliary tasks of IQA and de-blurring on blurred IF images. CONCLUSIONS In this paper, we propose a new DeepMT-ND method for nephropathy diagnosis on blurred IF images. The proposed hierarchical multi-task learning framework provides the new scope to narrow the gap between the low-level vision and high-level medical tasks, and will contribute to nephropathy diagnosis in clinical scenarios. The diagnosis accuracy and generalization ability of DeepMT-ND are experimentally verified to be effective over both synthetic and real-world databases.
Collapse
Affiliation(s)
- Yibing Fu
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Lai Jiang
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Sai Pan
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China
| | - Pu Chen
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China
| | - Xiaofei Wang
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, United Kingdom
| | - Ning Dai
- School of Electronic and Information Engineering, Beihang University, Beijing, China
| | - Xiangmei Chen
- Department of Nephrology, Chinese People's Liberation Army General Hospital, Beijing, China.
| | - Mai Xu
- School of Electronic and Information Engineering, Beihang University, Beijing, China.
| |
Collapse
|
40
|
Xia W, Zou J, Qiu X, Chen F, Zhu B, Li C, Deng DL, Li X. Configured quantum reservoir computing for multi-task machine learning. Sci Bull (Beijing) 2023; 68:2321-2329. [PMID: 37679257 DOI: 10.1016/j.scib.2023.08.040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2023] [Revised: 07/22/2023] [Accepted: 08/16/2023] [Indexed: 09/09/2023]
Abstract
Amidst the rapid advancements in experimental technology, noise-intermediate-scale quantum (NISQ) devices have become increasingly programmable, offering versatile opportunities to leverage quantum computational advantage. Here we explore the intricate dynamics of programmable NISQ devices for quantum reservoir computing. Using a genetic algorithm to configure the quantum reservoir dynamics, we systematically enhance the learning performance. Remarkably, a single configured quantum reservoir can simultaneously learn multiple tasks, including a synthetic oscillatory network of transcriptional regulators, chaotic motifs in gene regulatory networks, and the fractional-order Chua's circuit. Our configured quantum reservoir computing yields highly precise predictions for these learning tasks, outperforming classical reservoir computing. We also test the configured quantum reservoir computing in foreign exchange (FX) market applications and demonstrate its capability to capture the stochastic evolution of the exchange rates with significantly greater accuracy than classical reservoir computing approaches. Through comparison with classical reservoir computing, we highlight the unique role of quantum coherence in the quantum reservoir, which underpins its exceptional learning performance. Our findings suggest the exciting potential of configured quantum reservoir computing for exploiting the quantum computation power of NISQ devices in developing artificial general intelligence.
Collapse
Affiliation(s)
- Wei Xia
- State Key Laboratory of Surface Physics, Key Laboratory of Micro and Nano Photonic Structures (MOE), and Department of Physics, Fudan University, Shanghai 200433, China
| | - Jie Zou
- State Key Laboratory of Surface Physics, Key Laboratory of Micro and Nano Photonic Structures (MOE), and Department of Physics, Fudan University, Shanghai 200433, China
| | - Xingze Qiu
- State Key Laboratory of Surface Physics, Key Laboratory of Micro and Nano Photonic Structures (MOE), and Department of Physics, Fudan University, Shanghai 200433, China; School of Physics Science and Engineering, Tongji University, Shanghai 200092, China
| | - Feng Chen
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
| | - Bing Zhu
- Hong Kong and Shang Hai Banking Corporation Laboratory, Hong Kong and Shang Hai Banking Corporation Holdings PLC, Guangzhou 511458, China
| | - Chunhe Li
- Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China; Shanghai Center for Mathematical Sciences and School of Mathematical Sciences, Fudan University, Shanghai 200433, China
| | - Dong-Ling Deng
- Center for Quantum Information, IIIS, Tsinghua University, Beijing 100084, China; Shanghai Qi Zhi Institute, AI Tower, Shanghai 200232, China
| | - Xiaopeng Li
- State Key Laboratory of Surface Physics, Key Laboratory of Micro and Nano Photonic Structures (MOE), and Department of Physics, Fudan University, Shanghai 200433, China; Shanghai Qi Zhi Institute, AI Tower, Shanghai 200232, China; Shanghai Research Center for Quantum Sciences, Shanghai 201315, China.
| |
Collapse
|
41
|
熊 亮, 秦 小, 刘 欣. [Non-local attention and multi-task learning based lung segmentation in chest X-ray]. Sheng Wu Yi Xue Gong Cheng Xue Za Zhi 2023; 40:912-919. [PMID: 37879920 PMCID: PMC10600435 DOI: 10.7507/1001-5515.202211079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 09/14/2023] [Indexed: 10/27/2023]
Abstract
Precise segmentation of lung field is a crucial step in chest radiographic computer-aided diagnosis system. With the development of deep learning, fully convolutional network based models for lung field segmentation have achieved great effect but are poor at accurate identification of the boundary and preserving lung field consistency. To solve this problem, this paper proposed a lung segmentation algorithm based on non-local attention and multi-task learning. Firstly, an encoder-decoder convolutional network based on residual connection was used to extract multi-scale context and predict the boundary of lung. Secondly, a non-local attention mechanism to capture the long-range dependencies between pixels in the boundary regions and global context was proposed to enrich feature of inconsistent region. Thirdly, a multi-task learning to predict lung field based on the enriched feature was conducted. Finally, experiments to evaluate this algorithm were performed on JSRT and Montgomery dataset. The maximum improvement of Dice coefficient and accuracy were 1.99% and 2.27%, respectively, comparing with other representative algorithms. Results show that by enhancing the attention of boundary, this algorithm can improve the accuracy and reduce false segmentation.
Collapse
Affiliation(s)
- 亮 熊
- 中国科学院成都计算机应用研究所(成都 610041)Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610041, P. R. China
- 中国科学院大学(北京 100049)University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - 小林 秦
- 中国科学院成都计算机应用研究所(成都 610041)Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610041, P. R. China
- 中国科学院大学(北京 100049)University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - 欣 刘
- 中国科学院成都计算机应用研究所(成都 610041)Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu 610041, P. R. China
| |
Collapse
|
42
|
Cornalba F, Disselkamp C, Scassola D, Helf C. Multi-objective reward generalization: improving performance of Deep Reinforcement Learning for applications in single-asset trading. Neural Comput Appl 2023; 36:619-637. [PMID: 38187995 PMCID: PMC10770001 DOI: 10.1007/s00521-023-09033-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 09/06/2023] [Indexed: 01/09/2024]
Abstract
We investigate the potential of Multi-Objective, Deep Reinforcement Learning for stock and cryptocurrency single-asset trading: in particular, we consider a Multi-Objective algorithm which generalizes the reward functions and discount factor (i.e., these components are not specified a priori, but incorporated in the learning process). Firstly, using several important assets (BTCUSD, ETHUSDT, XRPUSDT, AAPL, SPY, NIFTY50), we verify the reward generalization property of the proposed Multi-Objective algorithm, and provide preliminary statistical evidence showing increased predictive stability over the corresponding Single-Objective strategy. Secondly, we show that the Multi-Objective algorithm has a clear edge over the corresponding Single-Objective strategy when the reward mechanism is sparse (i.e., when non-null feedback is infrequent over time). Finally, we discuss the generalization properties with respect to the discount factor. The entirety of our code is provided in open-source format.
Collapse
Affiliation(s)
- Federico Cornalba
- Institute of Science and Technology Austria (ISTA), Am Campus 1, 3400 Klosterneuburg, Austria
- Department of Mathematical Sciences, University of Bath, Bath, UK
| | | | - Davide Scassola
- Department of Mathematics and Geosciences, University of Trieste, Trieste, Italy
| | | |
Collapse
|
43
|
Wang Y, Wu Z, Dai J, Morgan TN, Garbens A, Kominsky H, Gahan J, Larson EC. Evaluating robotic-assisted partial nephrectomy surgeons with fully convolutional segmentation and multi-task attention networks. J Robot Surg 2023; 17:2323-2330. [PMID: 37368225 PMCID: PMC10492672 DOI: 10.1007/s11701-023-01657-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 06/17/2023] [Indexed: 06/28/2023]
Abstract
We use machine learning to evaluate surgical skill from videos during the tumor resection and renography steps of a robotic assisted partial nephrectomy (RAPN). This expands previous work using synthetic tissue to include actual surgeries. We investigate cascaded neural networks for predicting surgical proficiency scores (OSATS and GEARS) from RAPN videos recorded from the DaVinci system. The semantic segmentation task generates a mask and tracks the various surgical instruments. The movements from the instruments found via semantic segmentation are processed by a scoring network that regresses (predicts) GEARS and OSATS scoring for each subcategory. Overall, the model performs well for many subcategories such as force sensitivity and knowledge of instruments of GEARS and OSATS scoring, but can suffer from false positives and negatives that would not be expected of human raters. This is mainly attributed to limited training data variability and sparsity.
Collapse
Affiliation(s)
- Yihao Wang
- Department of Computer Science, Southern Methodist University, Dallas, USA
| | - Zhongjie Wu
- Department of Computer Science, Southern Methodist University, Dallas, USA
| | - Jessica Dai
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, USA
| | - Tara N. Morgan
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, USA
| | - Alaina Garbens
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, USA
| | - Hal Kominsky
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, USA
| | - Jeffrey Gahan
- Department of Urology, University of Texas Southwestern Medical Center, Dallas, USA
| | - Eric C. Larson
- Department of Computer Science, Southern Methodist University, Dallas, USA
| |
Collapse
|
44
|
Lai PT, Wei CH, Luo L, Chen Q, Lu Z. BioREx: Improving biomedical relation extraction by leveraging heterogeneous datasets. J Biomed Inform 2023; 146:104487. [PMID: 37673376 DOI: 10.1016/j.jbi.2023.104487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 08/18/2023] [Accepted: 09/02/2023] [Indexed: 09/08/2023]
Abstract
Biomedical relation extraction (RE) is the task of automatically identifying and characterizing relations between biomedical concepts from free text. RE is a central task in biomedical natural language processing (NLP) research and plays a critical role in many downstream applications, such as literature-based discovery and knowledge graph construction. State-of-the-art methods were used primarily to train machine learning models on individual RE datasets, such as protein-protein interaction and chemical-induced disease relation. Manual dataset annotation, however, is highly expensive and time-consuming, as it requires domain knowledge. Existing RE datasets are usually domain-specific or small, which limits the development of generalized and high-performing RE models. In this work, we present a novel framework for systematically addressing the data heterogeneity of individual datasets and combining them into a large dataset. Based on the framework and dataset, we report on BioREx, a data-centric approach for extracting relations. Our evaluation shows that BioREx achieves significantly higher performance than the benchmark system trained on the individual dataset, setting a new SOTA from 74.4% to 79.6% in F-1 measure on the recently released BioRED corpus. We further demonstrate that the combined dataset can improve performance for five different RE tasks. In addition, we show that on average BioREx compares favorably to current best-performing methods such as transfer learning and multi-task learning. Finally, we demonstrate BioREx's robustness and generalizability in two independent RE tasks not previously seen in training data: drug-drug N-ary combination and document-level gene-disease RE. The integrated dataset and optimized method have been packaged as a stand-alone tool available at https://github.com/ncbi/BioREx.
Collapse
Affiliation(s)
- Po-Ting Lai
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), MD, 20894 Bethesda, USA
| | - Chih-Hsuan Wei
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), MD, 20894 Bethesda, USA
| | - Ling Luo
- School of Computer Science and Technology, Dalian University of Technology, 116024 Dalian, China
| | - Qingyu Chen
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), MD, 20894 Bethesda, USA
| | - Zhiyong Lu
- National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), MD, 20894 Bethesda, USA.
| |
Collapse
|
45
|
Sun L, Zhang Y, Liu T, Ge H, Tian J, Qi X, Sun J, Zhao Y. A collaborative multi-task learning method for BI-RADS category 4 breast lesion segmentation and classification of MRI images. Comput Methods Programs Biomed 2023; 240:107705. [PMID: 37454498 DOI: 10.1016/j.cmpb.2023.107705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/03/2022] [Revised: 06/15/2023] [Accepted: 07/01/2023] [Indexed: 07/18/2023]
Abstract
BACKGROUND AND OBJECTIVE The diagnosis of BI-RADS category 4 breast lesion is difficult because its probability of malignancy ranges from 2% to 95%. For BI-RADS category 4 breast lesions, MRI is one of the prominent noninvasive imaging techniques. In this paper, we research computer algorithms to segment lesions and classify the benign or malignant lesions in MRI images. However, this task is challenging because the BI-RADS category 4 lesions are characterized by irregular shape, imbalanced class, and low contrast. METHODS We fully utilize the intrinsic correlation between segmentation and classification tasks, where accurate segmentation will yield accurate classification results, and classification results will promote better segmentation. Therefore, we propose a collaborative multi-task algorithm (CMTL-SC). Specifically, a preliminary segmentation subnet is designed to identify the boundaries, locations and segmentation masks of lesions; a classification subnet, which combines the information provided by the preliminary segmentation, is designed to achieve benign or malignant classification; a repartition segmentation subnet which aggregates the benign or malignant results, is designed to refine the lesion segment. The three subnets work cooperatively so that the CMTL-SC can identify the lesions better which solves the three challenges. RESULTS AND CONCLUSION We collect MRI data from 248 patients in the Second Hospital of Dalian Medical University. The results show that the lesion boundaries delineated by the CMTL-SC are close to the boundaries delineated by the physicians. Moreover, the CMTL-SC yields better results than the single-task and multi-task state-of-the-art algorithms. Therefore, CMTL-SC can help doctors make precise diagnoses and refine treatments for patients.
Collapse
Affiliation(s)
- Liang Sun
- College of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Yunling Zhang
- College of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Tang Liu
- Department of Radiology, The Second Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Hongwei Ge
- College of Computer Science and Technology, Dalian University of Technology, Dalian, China
| | - Juan Tian
- Department of Radiology, The Second Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Xin Qi
- Department of Radiology, The Second Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Jian Sun
- Health Management Center, The Second Affiliated Hospital of Dalian Medical University, Dalian, China
| | - Yiping Zhao
- Department of Radiology, The Second Affiliated Hospital of Dalian Medical University, Dalian, China.
| |
Collapse
|
46
|
Ye S, Zhao W, Shen X, Jiang X, He T. An effective multi-task learning framework for drug repurposing based on graph representation learning. Methods 2023; 218:48-56. [PMID: 37516260 DOI: 10.1016/j.ymeth.2023.07.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Revised: 07/04/2023] [Accepted: 07/20/2023] [Indexed: 07/31/2023] Open
Abstract
Drug repurposing, which typically applies the procedure of drug-disease associations (DDAs) prediction, is a feasible solution to drug discovery. Compared with traditional methods, drug repurposing can reduce the cost and time for drug development and advance the success rate of drug discovery. Although many methods for drug repurposing have been proposed and the obtained results are relatively acceptable, there is still some room for improving the predictive performance, since those methods fail to consider fully the issue of sparseness in known drug-disease associations. In this paper, we propose a novel multi-task learning framework based on graph representation learning to identify DDAs for drug repurposing. In our proposed framework, a heterogeneous information network is first constructed by combining multiple biological datasets. Then, a module consisting of multiple layers of graph convolutional networks is utilized to learn low-dimensional representations of nodes in the constructed heterogeneous information network. Finally, two types of auxiliary tasks are designed to help to train the target task of DDAs prediction in the multi-task learning framework. Comprehensive experiments are conducted on real data and the results demonstrate the effectiveness of the proposed method for drug repurposing.
Collapse
Affiliation(s)
- Shengwei Ye
- Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei 430079, PR China; School of Computer, Central China Normal University, Wuhan, Hubei 430079, PR China; National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, Hubei 430079, PR China
| | - Weizhong Zhao
- Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei 430079, PR China; School of Computer, Central China Normal University, Wuhan, Hubei 430079, PR China; National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, Hubei 430079, PR China.
| | - Xianjun Shen
- Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei 430079, PR China; School of Computer, Central China Normal University, Wuhan, Hubei 430079, PR China; National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, Hubei 430079, PR China
| | - Xingpeng Jiang
- Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei 430079, PR China; School of Computer, Central China Normal University, Wuhan, Hubei 430079, PR China; National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, Hubei 430079, PR China
| | - Tingting He
- Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei 430079, PR China; School of Computer, Central China Normal University, Wuhan, Hubei 430079, PR China; National Language Resources Monitoring & Research Center for Network Media, Central China Normal University, Wuhan, Hubei 430079, PR China
| |
Collapse
|
47
|
Cho K, Kim J, Kim KD, Park S, Kim J, Yun J, Ahn Y, Oh SY, Lee SM, Seo JB, Kim N. MuSiC-ViT: A multi-task Siamese convolutional vision transformer for differentiating change from no-change in follow-up chest radiographs. Med Image Anal 2023; 89:102894. [PMID: 37562256 DOI: 10.1016/j.media.2023.102894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Revised: 06/29/2023] [Accepted: 07/07/2023] [Indexed: 08/12/2023]
Abstract
A major responsibility of radiologists in routine clinical practice is to read follow-up chest radiographs (CXRs) to identify changes in a patient's condition. Diagnosing meaningful changes in follow-up CXRs is challenging because radiologists must differentiate disease changes from natural or benign variations. Here, we suggest using a multi-task Siamese convolutional vision transformer (MuSiC-ViT) with an anatomy-matching module (AMM) to mimic the radiologist's cognitive process for differentiating baseline change from no-change. MuSiC-ViT uses the convolutional neural networks (CNNs) meet vision transformers model that combines CNN and transformer architecture. It has three major components: a Siamese network architecture, an AMM, and multi-task learning. Because the input is a pair of CXRs, a Siamese network was adopted for the encoder. The AMM is an attention module that focuses on related regions in the CXR pairs. To mimic a radiologist's cognitive process, MuSiC-ViT was trained using multi-task learning, normal/abnormal and change/no-change classification, and anatomy-matching. Among 406 K CXRs studied, 88 K change and 115 K no-change pairs were acquired for the training dataset. The internal validation dataset consisted of 1,620 pairs. To demonstrate the robustness of MuSiC-ViT, we verified the results with two other validation datasets. MuSiC-ViT respectively achieved accuracies and area under the receiver operating characteristic curves of 0.728 and 0.797 on the internal validation dataset, 0.614 and 0.784 on the first external validation dataset, and 0.745 and 0.858 on a second temporally separated validation dataset. All code is available at https://github.com/chokyungjin/MuSiC-ViT.
Collapse
Affiliation(s)
- Kyungjin Cho
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Jeeyoung Kim
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Ki Duk Kim
- Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Seungju Park
- Department of Biomedical Engineering, College of Health Sciences, Korea University, Seoul, Republic of Korea
| | - Junsik Kim
- Department of Biomedical Engineering, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, College of Medicine, University of Ulsan, Seoul, Republic of Korea
| | - Jihye Yun
- Department of Radiology, Asan Medical Center/University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Yura Ahn
- Department of Radiology, and Research of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
| | - Sang Young Oh
- Department of Radiology, Asan Medical Center/University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Sang Min Lee
- Department of Radiology, University of Ulsan College of Medicine and Asan Medical Center, Seoul, Republic of Korea
| | - Joon Beom Seo
- Department of Radiology, Asan Medical Center/University of Ulsan College of Medicine, Seoul, Republic of Korea.
| | - Namkug Kim
- Department of Convergence Medicine, Asan Medical Center/University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
48
|
Xu M, Zhu Z, Li Y, Zheng S, Li L, Wu H, Zhao Y. Cooperative dual medical ontology representation learning for clinical assisted decision-making. Comput Biol Med 2023; 163:107138. [PMID: 37329613 DOI: 10.1016/j.compbiomed.2023.107138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 05/28/2023] [Accepted: 06/04/2023] [Indexed: 06/19/2023]
Abstract
OBJECTIVE Predicting clinical events and providing assisted decision-making using Electronic Health Records (EHRs) play a central role in personalized healthcare. Despite the promising performance achieved for diagnosis and procedure predictions, most of the existing predictive models regard different medical codes as the same type and generally ignore the dependence between diagnoses and procedures in patients' admission history. To address these issues, we propose an end-to-end cooperative dual medical ontology representation learning framework for clinical assisted decision-making. MATERIALS AND METHODS The framework consists of two primary modules: (1) dual medical ontology representation learning to facilitate the learning of medical concepts and (2) task dependent multi-task prediction to capture the correlation between diagnoses and procedures in patients' admission history. We evaluate our method with EHRs from the MIMIC-III Clinical Database, covering 6321 patients and 16335 visits. RESULTS Experiments conducted on the MIMIC-III dataset show that the proposed model achieves the best performance, with a top-20 accuracy of 58.20% for diagnosis prediction and a top-20 accuracy of 75.85% for procedure prediction. In addition, a series of experimental analyses and case studies further illustrate the excellent performance of our model. CONCLUSION We propose an end-to-end cooperative dual medical ontology representation learning framework, which achieves superior performance on multi-task diagnosis and procedure predictions. The source code is available at https://github.com/mhxu1998/CoDMO.
Collapse
Affiliation(s)
- Muhao Xu
- Institute of Information Science, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, China
| | - Zhenfeng Zhu
- Institute of Information Science, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, China.
| | - Youru Li
- Institute of Information Science, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, China
| | - Shuai Zheng
- Institute of Information Science, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, China
| | - Linfeng Li
- Yidu Cloud Technology Inc., Beijing, China
| | - Haiyan Wu
- Department of Otorhinolaryngology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Beijing, China
| | - Yao Zhao
- Institute of Information Science, Beijing Jiaotong University, Beijing, China; Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, China
| |
Collapse
|
49
|
Wu Z, Loo CK, Obaidellah U, Pasupa K. A novel online multi-task learning for COVID-19 multi-output spatio-temporal prediction. Heliyon 2023; 9:e18771. [PMID: 37636411 PMCID: PMC10450863 DOI: 10.1016/j.heliyon.2023.e18771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 07/26/2023] [Accepted: 07/27/2023] [Indexed: 08/29/2023] Open
Abstract
In light of the ongoing COVID-19 pandemic, predicting its trend would significantly impact decision-making. However, this is not a straightforward task due to three main difficulties: temporal autocorrelation, spatial dependency, and concept drift caused by virus mutations and lockdown policies. Although machine learning has been extensively used in related work, no previous research has successfully addressed all three challenges simultaneously. To overcome this challenge, we developed a novel online multi-task regression algorithm that incorporates a chain structure to capture spatial dependency, the ADWIN drift detector to adapt to concept drift, and the lag time series feature to capture temporal autocorrelation. We conducted several comparative experiments based on the number of daily confirmed cases in 20 areas in California and affiliated cities. The results from our experiments demonstrate that our proposed model is superior in adapting to concept drift in COVID-19 data and capturing spatial dependencies across various regions. This leads to a significant improvement in prediction accuracy when compared to existing state-of-the-art batch machine learning methods, such as N-Beats, DeepAR, TCN, and LSTM.
Collapse
Affiliation(s)
- Zipeng Wu
- Faculty of Computer Science & Information Technology, University of Malaya,Kuala Lumpur, 50603, Malaysia
| | - Chu Kiong Loo
- Faculty of Computer Science & Information Technology, University of Malaya,Kuala Lumpur, 50603, Malaysia
| | - Unaizah Obaidellah
- Faculty of Computer Science & Information Technology, University of Malaya,Kuala Lumpur, 50603, Malaysia
| | - Kitsuchart Pasupa
- School of Information Technology, King Mongkut's Institute of Technology Ladkrabang,Bangkok, 10520, Thailand
| |
Collapse
|
50
|
Li C, Ding S, Zou N, Hu X, Jiang X, Zhang K. Multi-task learning with dynamic re-weighting to achieve fairness in healthcare predictive modeling. J Biomed Inform 2023; 143:104399. [PMID: 37211197 PMCID: PMC10665114 DOI: 10.1016/j.jbi.2023.104399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 04/14/2023] [Accepted: 05/15/2023] [Indexed: 05/23/2023]
Abstract
The emphasis on fairness in predictive healthcare modeling has increased in popularity as an approach for overcoming biases in automated decision-making systems. The aim is to guarantee that sensitive characteristics like gender, race, and ethnicity do not influence prediction outputs. Numerous algorithmic strategies have been proposed to reduce bias in prediction results, mitigate prejudice toward minority groups and promote prediction fairness. The goal of these strategies is to ensure that model prediction performance does not exhibit significant disparity among sensitive groups. In this study, we propose a novel fairness-achieving scheme based on multitask learning, which fundamentally differs from conventional fairness-achieving techniques, including altering data distributions and constraint optimization through regularizing fairness metrics or tampering with prediction outcomes. By dividing predictions on different sub-populations into separate tasks, we view the fairness problem as a task-balancing problem. To ensure fairness during the model-training process, we suggest a novel dynamic re-weighting approach. Fairness is achieved by dynamically modifying the gradients of various prediction tasks during neural network back-propagation, and this novel technique applies to a wide range of fairness criteria. We conduct tests on a real-world use case to predict sepsis patients' mortality risk. Our approach satisfies that it can reduce the disparity between subgroups by 98% while only losing less than 4% of prediction accuracy.
Collapse
Affiliation(s)
- Can Li
- School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX, United States
| | - Sirui Ding
- Department of Computer Science & Engineering, Texas A&M University, College Station, TX, United States
| | - Na Zou
- Department of Engineering Technology and Industrial Distribution, Texas A&M University, College Station, TX, United States
| | - Xia Hu
- Department of Computer Science, Rice University, Houston, TX, United States
| | - Xiaoqian Jiang
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States
| | - Kai Zhang
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States.
| |
Collapse
|