1
|
Qian Z, Wang Z, Zhang X, Wei B, Lai M, Shou J, Fan Y, Xu Y. MSNSegNet: attention-based multi-shape nuclei instance segmentation in histopathology images. Med Biol Eng Comput 2024; 62:1821-1836. [PMID: 38401007 DOI: 10.1007/s11517-024-03050-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Accepted: 02/13/2024] [Indexed: 02/26/2024]
Abstract
In clinical research, the segmentation of irregularly shaped nuclei, particularly in mesenchymal areas like fibroblasts, is crucial yet often neglected. These irregular nuclei are significant for assessing tissue repair in immunotherapy, a process involving neovascularization and fibroblast proliferation. Proper segmentation of these nuclei is vital for evaluating immunotherapy's efficacy, as it provides insights into pathological features. However, the challenge lies in the pronounced curvature variations of these non-convex nuclei, making their segmentation more difficult than that of regular nuclei. In this work, we introduce an undefined task to segment nuclei with both regular and irregular morphology, namely multi-shape nuclei segmentation. We propose a proposal-based method to perform multi-shape nuclei segmentation. By leveraging the two-stage structure of the proposal-based method, a powerful refinement module with high computational costs can be selectively deployed only in local regions, improving segmentation accuracy without compromising computational efficiency. We introduce a novel self-attention module to refine features in proposals for the sake of effectiveness and efficiency in the second stage. The self-attention module improves segmentation performance by capturing long-range dependencies to assist in distinguishing the foreground from the background. In this process, similar features get high attention weights while dissimilar ones get low attention weights. In the first stage, we introduce a residual attention module and a semantic-aware module to accurately predict candidate proposals. The two modules capture more interpretable features and introduce additional supervision through semantic-aware loss. In addition, we construct a dataset with a proportion of non-convex nuclei compared with existing nuclei datasets, namely the multi-shape nuclei (MsN) dataset. Our MSNSegNet method demonstrates notable improvements across various metrics compared to the second-highest-scoring methods. For all nuclei, the D i c e score improved by approximately 1.66 % , A J I by about 2.15 % , and D i c e obj by roughly 0.65 % . For non-convex nuclei, which are crucial in clinical applications, our method's A J I improved significantly by approximately 3.86 % and D i c e obj by around 2.54 % . These enhancements underscore the effectiveness of our approach on multi-shape nuclei segmentation, particularly in challenging scenarios involving irregularly shaped nuclei.
Collapse
Affiliation(s)
- Ziniu Qian
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Zihua Wang
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Xin Zhang
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Bingzheng Wei
- Xiaomi Corporation, Haidian District, Beijing, 100085, Beijing, China
| | - Maode Lai
- Department of Pathology, School of Medicine, Zhejiang Provincial Key Laboratory of Disease Proteomics and Alibaba-Zhejiang University Joint Research Center of Future Digital Healthcare, Zhejiang University, Hangzhou, 310027, Zhejiang, China
| | - Jianzhong Shou
- Department of Urology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Changyang District, Beijing, 100021, Beijing, China
| | - Yubo Fan
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China
| | - Yan Xu
- School of Biological Science and Medical Engineering, Beihang University, Haidian District, Beijing, 100191, Beijing, China.
| |
Collapse
|
2
|
Han S, Phasouk K, Zhu J, Fong Y. Optimizing deep learning-based segmentation of densely packed cells using cell surface markers. BMC Med Inform Decis Mak 2024; 24:124. [PMID: 38750526 PMCID: PMC11094866 DOI: 10.1186/s12911-024-02502-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 04/08/2024] [Indexed: 05/19/2024] Open
Abstract
BACKGROUND Spatial molecular profiling depends on accurate cell segmentation. Identification and quantitation of individual cells in dense tissues, e.g. highly inflamed tissue caused by viral infection or immune reaction, remains a challenge. METHODS We first assess the performance of 18 deep learning-based cell segmentation models, either pre-trained or trained by us using two public image sets, on a set of immunofluorescence images stained with immune cell surface markers in skin tissue obtained during human herpes simplex virus (HSV) infection. We then further train eight of these models using up to 10,000+ training instances from the current image set. Finally, we seek to improve performance by tuning parameters of the most successful method from the previous step. RESULTS The best model before fine-tuning achieves a mean Average Precision (mAP) of 0.516. Prediction performance improves substantially after training. The best model is the cyto model from Cellpose. After training, it achieves an mAP of 0.694; with further parameter tuning, the mAP reaches 0.711. CONCLUSION Selecting the best model among the existing approaches and further training the model with images of interest produce the most gain in prediction performance. The performance of the resulting model compares favorably to human performance. The imperfection of the final model performance can be attributed to the moderate signal-to-noise ratio in the imageset.
Collapse
Affiliation(s)
- Sunwoo Han
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA
| | - Khamsone Phasouk
- Department of Laboratory Medicine and Pathology, University of Washington School of Medicine, Seattle, United States
| | - Jia Zhu
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA.
- Department of Laboratory Medicine and Pathology, University of Washington School of Medicine, Seattle, United States.
| | - Youyi Fong
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA.
| |
Collapse
|
3
|
Han S, Phasouk K, Zhu J, Fong Y. Optimizing Deep Learning-Based Segmentation of Densely Packed Cells using Cell Surface Markers. RESEARCH SQUARE 2023:rs.3.rs-3307496. [PMID: 37841876 PMCID: PMC10571619 DOI: 10.21203/rs.3.rs-3307496/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/17/2023]
Abstract
Background Spatial molecular profiling depends on accurate cell segmentation. Identification and quantitation of individual cells in dense tissues, e.g. highly inflamed tissue caused by viral infection or immune reaction, remains a challenge. Methods We first assess the performance of 18 deep learning-based cell segmentation models, either pre-trained or trained by us using two public image sets, on a set of immunofluorescence images stained with immune cell surface markers in skin tissue obtained during human herpes simplex virus (HSV) infection. We then further train eight of these models using up to 10,000+ training instances from the current image set. Finally, we seek to improve performance by tuning parameters of the most successful method from the previous step. Results The best model before fine-tuning achieves a mean Average Precision (mAP) of 0.516. Prediction performance improves substantially after training. The best model is the cyto model from Cellpose. After training, it achieves an mAP of 0.694; with further parameter tuning, the mAP reaches 0.711. Conclusion Selecting the best model among the existing approaches and further training the model with images of interest produce the most gain in prediction performance. The performance of the resulting model compares favorably to human performance. The imperfection of the final model performance can be attributed to the moderate signal-to-noise ratio i the imageset.
Collapse
Affiliation(s)
- Sunwoo Han
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA
| | - Khamsone Phasouk
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA
| | - Jia Zhu
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA
| | - Youyi Fong
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, USA
| |
Collapse
|
4
|
Basu A, Senapati P, Deb M, Rai R, Dhal KG. A survey on recent trends in deep learning for nucleus segmentation from histopathology images. EVOLVING SYSTEMS 2023; 15:1-46. [PMID: 38625364 PMCID: PMC9987406 DOI: 10.1007/s12530-023-09491-3] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Accepted: 02/13/2023] [Indexed: 03/08/2023]
Abstract
Nucleus segmentation is an imperative step in the qualitative study of imaging datasets, considered as an intricate task in histopathology image analysis. Segmenting a nucleus is an important part of diagnosing, staging, and grading cancer, but overlapping regions make it hard to separate and tell apart independent nuclei. Deep Learning is swiftly paving its way in the arena of nucleus segmentation, attracting quite a few researchers with its numerous published research articles indicating its efficacy in the field. This paper presents a systematic survey on nucleus segmentation using deep learning in the last five years (2017-2021), highlighting various segmentation models (U-Net, SCPP-Net, Sharp U-Net, and LiverNet) and exploring their similarities, strengths, datasets utilized, and unfolding research areas.
Collapse
Affiliation(s)
- Anusua Basu
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| | - Pradip Senapati
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| | - Mainak Deb
- Wipro Technologies, Pune, Maharashtra India
| | - Rebika Rai
- Department of Computer Applications, Sikkim University, Sikkim, India
| | - Krishna Gopal Dhal
- Department of Computer Science and Application, Midnapore College (Autonomous), Paschim Medinipur, Midnapore, West Bengal India
| |
Collapse
|
5
|
Wang L, Shen J, Wang Y, Bi J. Identification of fatty acid metabolism-based molecular subtypes and prognostic signature to predict immune landscape and guide clinical drug treatment in renal clear cell carcinoma. Int Immunopharmacol 2023; 116:109735. [PMID: 36716517 DOI: 10.1016/j.intimp.2023.109735] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 01/04/2023] [Accepted: 01/11/2023] [Indexed: 01/29/2023]
Abstract
Three subtypes of samples were generated based on genes involved in fatty acid metabolism in The Cancer Genome Atlas (TCGA)-RCC patients using a non-negative matrix factorization (NMF) algorithm. 32 co-expressed modules were identified using WCGNA. We constructed a four-gene signature in our training set using least absolute shrinkage selection operator regression analysis and verified it in our testing and overall sets. A relevant study analysis in clinical trials was conducted, which showed the model had good stability and potential application value for predicting outcomes. We analyzed the immune microenvironment using MCPcounter, CIBERSORT, quanTIseq, TIMER and ESTIMATE algorithms, and the result indicated risk was positively related to T cells, B-lineage, and fibroblasts and negatively correlated with monocytic lineage, myeloid dendritic cells, neutrophils, and endothelial cells, and CPT1B was positively related to T cells, CD8 + T cells, Cytotoxic lymphocytes and NK cells, and negatively correlated with myeloid dendritic cells, fibroblasts, endothelial cells. Tumor mutation burden was positively related to risk score and the expression of CPT1B using the R packages corrplot, circlize. Through the R package pRRophetic, drug sensitivity tests showed that the low-risk score group would benefit more from sunitinib and less from pazopanib, sorafenib, temsirolimus, gemcitabine and doxorubicin than the high-risk score group. We performed the relevant basic assay validation for CPT1B, and the proliferation ability of RCC cells was inhibited after the knockdown of protein expression of CPT1B. In conclusion, we established a four-gene model that can predict outcomes of RCC with potential applications in diagnosis and treatment.
Collapse
Affiliation(s)
- Linhui Wang
- Department of Urology, The First Hospital of China Medical University, Shenyang, Liaoning, China
| | - Junlin Shen
- Department of Urology, The First Hospital of China Medical University, Shenyang, Liaoning, China
| | - Yutao Wang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jianbin Bi
- Department of Urology, The First Hospital of China Medical University, Shenyang, Liaoning, China.
| |
Collapse
|
6
|
Nan Y, Tang P, Zhang G, Zeng C, Liu Z, Gao Z, Zhang H, Yang G. Unsupervised Tissue Segmentation via Deep Constrained Gaussian Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3799-3811. [PMID: 35905069 DOI: 10.1109/tmi.2022.3195123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Tissue segmentation is the mainstay of pathological examination, whereas the manual delineation is unduly burdensome. To assist this time-consuming and subjective manual step, researchers have devised methods to automatically segment structures in pathological images. Recently, automated machine and deep learning based methods dominate tissue segmentation research studies. However, most machine and deep learning based approaches are supervised and developed using a large number of training samples, in which the pixel-wise annotations are expensive and sometimes can be impossible to obtain. This paper introduces a novel unsupervised learning paradigm by integrating an end-to-end deep mixture model with a constrained indicator to acquire accurate semantic tissue segmentation. This constraint aims to centralise the components of deep mixture models during the calculation of the optimisation function. In so doing, the redundant or empty class issues, which are common in current unsupervised learning methods, can be greatly reduced. By validation on both public and in-house datasets, the proposed deep constrained Gaussian network achieves significantly (Wilcoxon signed-rank test) better performance (with the average Dice scores of 0.737 and 0.735, respectively) on tissue segmentation with improved stability and robustness, compared to other existing unsupervised segmentation approaches. Furthermore, the proposed method presents a similar performance (p-value >0.05) compared to the fully supervised U-Net.
Collapse
|
7
|
Fu Z, Li J, Hua Z. DEAU-Net: Attention networks based on dual encoder for Medical Image Segmentation. Comput Biol Med 2022; 150:106197. [PMID: 37859289 DOI: 10.1016/j.compbiomed.2022.106197] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 09/25/2022] [Accepted: 10/09/2022] [Indexed: 11/15/2022]
Abstract
In recent years, variant networks derived from U-Net networks have achieved better results in the field of medical image segmentation. However, we found during our experiments that the current mainstream networks still have certain shortcomings in the learning and extraction of detailed features. Therefore, in this paper, we propose a feature attention network based on dual encoder. In the encoder stage, a dual encoder is used to implement macro feature extraction and micro feature extraction respectively. Feature attention fusion is then performed, resulting in the network that not only performs well in the recognition of macro features, but also in the processing of micro features, which is significantly improved. The network is divided into three stages: (1) learning and capture of macro features and detail features with dual encoders; (2) completing the mutual complementation of macro features and detail features through the residual attention module; (3) complete the fusion of the two features and output the final prediction result. We conducted experiments on two datasets on DEAU-Net and from the results of the comparison experiments, we showed better results in terms of edge detail features and macro features processing.
Collapse
Affiliation(s)
- Zhaojin Fu
- School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai 264005, China; School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
| | - Jinjiang Li
- School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai 264005, China; School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China; Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai 264005, China.
| | - Zhen Hua
- School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai 264005, China; School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China; Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Yantai 264005, China
| |
Collapse
|
8
|
Petmezas G, Stefanopoulos L, Kilintzis V, Tzavelis A, Rogers JA, Katsaggelos AK, Maglaveras N. State-of-the-art Deep Learning Methods on Electrocardiogram Data: A Systematic Review (Preprint). JMIR Med Inform 2022; 10:e38454. [PMID: 35969441 PMCID: PMC9425174 DOI: 10.2196/38454] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2022] [Revised: 06/03/2022] [Accepted: 07/03/2022] [Indexed: 11/13/2022] Open
Abstract
Background Electrocardiogram (ECG) is one of the most common noninvasive diagnostic tools that can provide useful information regarding a patient’s health status. Deep learning (DL) is an area of intense exploration that leads the way in most attempts to create powerful diagnostic models based on physiological signals. Objective This study aimed to provide a systematic review of DL methods applied to ECG data for various clinical applications. Methods The PubMed search engine was systematically searched by combining “deep learning” and keywords such as “ecg,” “ekg,” “electrocardiogram,” “electrocardiography,” and “electrocardiology.” Irrelevant articles were excluded from the study after screening titles and abstracts, and the remaining articles were further reviewed. The reasons for article exclusion were manuscripts written in any language other than English, absence of ECG data or DL methods involved in the study, and absence of a quantitative evaluation of the proposed approaches. Results We identified 230 relevant articles published between January 2020 and December 2021 and grouped them into 6 distinct medical applications, namely, blood pressure estimation, cardiovascular disease diagnosis, ECG analysis, biometric recognition, sleep analysis, and other clinical analyses. We provide a complete account of the state-of-the-art DL strategies per the field of application, as well as major ECG data sources. We also present open research problems, such as the lack of attempts to address the issue of blood pressure variability in training data sets, and point out potential gaps in the design and implementation of DL models. Conclusions We expect that this review will provide insights into state-of-the-art DL methods applied to ECG data and point to future directions for research on DL to create robust models that can assist medical experts in clinical decision-making.
Collapse
Affiliation(s)
- Georgios Petmezas
- Lab of Computing, Medical Informatics and Biomedical-Imaging Technologies, The Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Leandros Stefanopoulos
- Lab of Computing, Medical Informatics and Biomedical-Imaging Technologies, The Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Vassilis Kilintzis
- Lab of Computing, Medical Informatics and Biomedical-Imaging Technologies, The Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Andreas Tzavelis
- Department of Biomedical Engineering, Northwestern University, Evanston, IL, United States
| | - John A Rogers
- Department of Material Science, Northwestern University, Evanston, IL, United States
| | - Aggelos K Katsaggelos
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, United States
| | - Nicos Maglaveras
- Lab of Computing, Medical Informatics and Biomedical-Imaging Technologies, The Medical School, Aristotle University of Thessaloniki, Thessaloniki, Greece
| |
Collapse
|
9
|
Kiran I, Raza B, Ijaz A, Khan MA. DenseRes-Unet: Segmentation of overlapped/clustered nuclei from multi organ histopathology images. Comput Biol Med 2022; 143:105267. [PMID: 35114445 DOI: 10.1016/j.compbiomed.2022.105267] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 01/23/2022] [Accepted: 01/23/2022] [Indexed: 11/16/2022]
Abstract
Cancer is the second deadliest disease globally that can affect any human body organ. Early detection of cancer can increase the chances of survival in humans. Morphometric appearances of histopathology images make it difficult to segment nuclei effectively. We proposed a model to segment overlapped nuclei from H&E stained images. U-Net model achieved state-of-the-art performance in many medical image segmentation tasks; however, we modified the U-Net to learn a distinct set of consistent features. In this paper, we proposed the DenseRes-Unet model by integrating dense blocks in the last layers of the encoder block of U-Net, focused on relevant features from previous layers of the model. Moreover, we take advantage of residual connections with Atrous blocks instead of conventional skip connections, which helps to reduce the semantic gap between encoder and decoder paths. The distance map and binary threshold techniques intensify the nuclei interior and contour information in the images, respectively. The distance map is used to detect the center point of nuclei; moreover, it differentiates among nuclei interior boundary and core area. The distance map lacks a contour problem, which is resolved by using a binary threshold. Binary threshold helps to enhance the pixels around nuclei. Afterward, we fed images into the proposed DenseRes-Unet model, a deep, fully convolutional network to segment nuclei in the images. We have evaluated our model on four publicly available datasets for Nuclei segmentation to validate the model's performance. Our proposed model achieves 89.77% accuracy 90.36% F1-score, and 78.61% Aggregated Jaccard Index (AJI) on Multi organ Nucleus Segmentation (MoNuSeg).
Collapse
Affiliation(s)
- Iqra Kiran
- Department of Computer Science, COMSATS University Islamabad (CUI), Islamabad, Pakistan.
| | - Basit Raza
- Department of Computer Science, COMSATS University Islamabad (CUI), Islamabad, Pakistan.
| | - Areesha Ijaz
- Department of Computer Science, COMSATS University Islamabad (CUI), Islamabad, Pakistan.
| | - Muazzam A Khan
- Department of Computer Science, Quaid-i-Azam University, Islamabad, Pakistan.
| |
Collapse
|
10
|
Cao C, Liu Z, Liu G, Jin S, Xia S. Ability of weakly supervised learning to detect acute ischemic stroke and hemorrhagic infarction lesions with diffusion-weighted imaging. Quant Imaging Med Surg 2022; 12:321-332. [PMID: 34993081 DOI: 10.21037/qims-21-324] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 06/27/2021] [Indexed: 12/31/2022]
Abstract
BACKGROUND Gradient-recalled echo (GRE) sequence is time-consuming and not routinely performed. Herein, we aimed to investigate the ability of weakly supervised learning to identify acute ischemic stroke (AIS) and concurrent hemorrhagic infarction based on diffusion-weighted imaging (DWI). METHODS First, we proposed spatially locating small stroke lesions in different positions and hemorrhagic infarction lesions by residual neural and visual geometry group networks using weakly supervised learning. Next, we compared the sensitivity and specificity for identifying automatically concurrent hemorrhagic infarction in stroke patients with the sensitivity and specificity of human readings of diffusion and b0 images to evaluate the performance of the weakly supervised methods. Also, the labeling time of the weakly supervised approach was compared with that of the fully supervised approach. RESULTS Data from a total of 1,027 patients were analyzed. The residual neural network displayed a higher sensitivity than did the visual geometry group network in spatially locating the small stroke and hemorrhagic infarction lesions. The residual neural network had significantly greater patient-level sensitivity than did the human readers (98.4% versus 86.2%, P=0.008) in identifying concurrent hemorrhagic infarction with GRE as the reference standard; however, their specificities were comparable (95.4% versus 96.9%, P>0.99). Weak labeling of lesions required significantly less time than did full labeling of lesions (2.667 versus 10.115 minutes, P<0.001). CONCLUSIONS Weakly supervised learning was able to spatially locate small stroke lesions in different positions and showed more sensitivity than did human reading in identifying concurrent hemorrhagic infarction based on DWI. The proposed approach can reduce the labeling workload.
Collapse
Affiliation(s)
- Chen Cao
- Department of Radiology, First Central Clinical College, Tianjin Medical University, Tianjin, China.,Department of Radiology, Tianjin Huanhu Hospital, Tianjin, China
| | - Zhiyang Liu
- Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, College of Electronic Information and Optical Engineering, Nankai University, Tianjin, China
| | - Guohua Liu
- Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, College of Electronic Information and Optical Engineering, Nankai University, Tianjin, China
| | - Song Jin
- Department of Radiology, Tianjin Huanhu Hospital, Tianjin, China
| | - Shuang Xia
- Department of Radiology, Tianjin First Central Hospital, School of Medicine, Nankai University, Tianjin, China
| |
Collapse
|
11
|
Hajdowska K, Student S, Borys D. Graph based method for cell segmentation and detection in live-cell fluorescence microscope imaging. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103071] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
12
|
Liang H, Cheng Z, Zhong H, Qu A, Chen L. A region-based convolutional network for nuclei detection and segmentation in microscopy images. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103276] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
13
|
Pérez-Dones D, Ledesma-Terrón M, Míguez DG. Quantitative Approaches to Study Retinal Neurogenesis. Biomedicines 2021; 9:1222. [PMID: 34572408 PMCID: PMC8471905 DOI: 10.3390/biomedicines9091222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 09/07/2021] [Accepted: 09/11/2021] [Indexed: 11/16/2022] Open
Abstract
The study of the development of the vertebrate retina can be addressed from several perspectives: from a purely qualitative to a more quantitative approach that takes into account its spatio-temporal features, its three-dimensional structure and also the regulation and properties at the systems level. Here, we review the ongoing transition toward a full four-dimensional characterization of the developing vertebrate retina, focusing on the challenges at the experimental, image acquisition, image processing and quantification. Using the developing zebrafish retina, we illustrate how quantitative data extracted from these type of highly dense, three-dimensional tissues depend strongly on the image quality, image processing and algorithms used to segment and quantify. Therefore, we propose that the scientific community that focuses on developmental systems could strongly benefit from a more detailed disclosure of the tools and pipelines used to process and analyze images from biological samples.
Collapse
Affiliation(s)
- Diego Pérez-Dones
- Centro de Biología Molecular Severo Ochoa, Universidad Autónoma de Madrid, 28049 Madrid, Spain
- Física de la Materia Condensada (IFIMAC), Facultad de Ciencias, Universidad Autónoma de Madrid, 28049 Madrid, Spain
| | - Mario Ledesma-Terrón
- Centro de Biología Molecular Severo Ochoa, Universidad Autónoma de Madrid, 28049 Madrid, Spain
- Física de la Materia Condensada (IFIMAC), Facultad de Ciencias, Universidad Autónoma de Madrid, 28049 Madrid, Spain
| | - David G Míguez
- Centro de Biología Molecular Severo Ochoa, Universidad Autónoma de Madrid, 28049 Madrid, Spain
- Física de la Materia Condensada (IFIMAC), Facultad de Ciencias, Universidad Autónoma de Madrid, 28049 Madrid, Spain
| |
Collapse
|
14
|
Yao S, Chen Y, Tian X, Jiang R. Pneumonia Detection Using an Improved Algorithm Based on Faster R-CNN. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2021; 2021:8854892. [PMID: 33968160 PMCID: PMC8081632 DOI: 10.1155/2021/8854892] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2020] [Revised: 01/19/2021] [Accepted: 02/14/2021] [Indexed: 12/02/2022]
Abstract
Pneumonia remains a threat to human health; the coronavirus disease 2019 (COVID-19) that began at the end of 2019 had a major impact on the world. It is still raging in many countries and has caused great losses to people's lives and property. In this paper, we present a method based on DeepConv-DilatedNet of identifying and localizing pneumonia in chest X-ray (CXR) images. Two-stage detector Faster R-CNN is adopted as the structure of a network. Feature Pyramid Network (FPN) is integrated into the residual neural network of a dilated bottleneck so that the deep features are expanded to preserve the deep feature and position information of the object. In the case of DeepConv-DilatedNet, the deconvolution network is used to restore high-level feature maps into its original size, and the target information is further retained. On the other hand, DeepConv-DilatedNet uses a popular fully convolution architecture with computation shared on the entire image. Then, Soft-NMS is used to screen boxes and ensure sample quality. Also, K-Means++ is used to generate anchor boxes to improve the localization accuracy. The algorithm obtained 39.23% Mean Average Precision (mAP) on the X-ray image dataset from the Radiological Society of North America (RSNA) and got 38.02% Mean Average Precision (mAP) on the ChestX-ray14 dataset, surpassing other detection algorithms. So, in this paper, an improved algorithm that can provide doctors with location information of pneumonia lesions is proposed.
Collapse
Affiliation(s)
- Shangjie Yao
- Institute of Advanced Digital Technology and Instrumentation, Zhejiang University, Zhejiang 310027, China
| | - Yaowu Chen
- Zhejiang Provincial Key Laboratory for Network Multimedia Technologies, Zhejiang University, Zhejiang 310027, China
| | - Xiang Tian
- Institute of Advanced Digital Technology and Instrumentation, Zhejiang University and State Key Laboratory of Industrial Control Technology, Zhejiang University, Zhejiang 310027, China
| | - Rongxin Jiang
- Institute of Advanced Digital Technology and Instrumentation, Zhejiang University and State Key Laboratory of Industrial Control Technology, Zhejiang University, Zhejiang 310027, China
| |
Collapse
|
15
|
Efficient Stain-Aware Nuclei Segmentation Deep Learning Framework for Multi-Center Histopathological Images. ELECTRONICS 2021. [DOI: 10.3390/electronics10080954] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Existing nuclei segmentation methods have obtained limited results with multi-center and multi-organ whole-slide images (WSIs) due to the use of different stains, scanners, overlapping, clumped nuclei, and the ambiguous boundary between adjacent cell nuclei. In an attempt to address these problems, we propose an efficient stain-aware nuclei segmentation method based on deep learning for multi-center WSIs. Unlike all related works that exploit a single-stain template from the dataset to normalize WSIs, we propose an efficient algorithm to select a set of stain templates based on stain clustering. Individual deep learning models are trained based on each stain template, and then, an aggregation function based on the Choquet integral is employed to combine the segmentation masks of the individual models. With a challenging multi-center multi-organ WSIs dataset, the experimental results demonstrate that the proposed method outperforms the state-of-art nuclei segmentation methods with aggregated Jaccard index (AJI) and F1-scores of 73.23% and 89.32%, respectively, while achieving a lower number of parameters.
Collapse
|
16
|
Sohail A, Khan A, Wahab N, Zameer A, Khan S. A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images. Sci Rep 2021; 11:6215. [PMID: 33737632 PMCID: PMC7973714 DOI: 10.1038/s41598-021-85652-1] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2020] [Accepted: 02/16/2021] [Indexed: 12/24/2022] Open
Abstract
The mitotic activity index is a key prognostic measure in tumour grading. Microscopy based detection of mitotic nuclei is a significant overhead and necessitates automation. This work proposes deep CNN based multi-phase mitosis detection framework "MP-MitDet" for mitotic nuclei identification in breast cancer histopathological images. The workflow constitutes: (1) label-refiner, (2) tissue-level mitotic region selection, (3) blob analysis, and (4) cell-level refinement. We developed an automatic label-refiner to represent weak labels with semi-sematic information for training of deep CNNs. A deep instance-based detection and segmentation model is used to explore probable mitotic regions on tissue patches. More probable regions are screened based on blob area and then analysed at cell-level by developing a custom CNN classifier "MitosRes-CNN" to filter false mitoses. The performance of the proposed "MitosRes-CNN" is compared with the state-of-the-art CNNs that are adapted to cell-level discrimination through cross-domain transfer learning and by adding task-specific layers. The performance of the proposed framework shows good discrimination ability in terms of F-score (0.75), recall (0.76), precision (0.71) and area under the precision-recall curve (0.78) on challenging TUPAC16 dataset. Promising results suggest good generalization of the proposed framework that can learn characteristic features from heterogenous mitotic nuclei.
Collapse
Affiliation(s)
- Anabia Sohail
- Pattern Recognition Lab, DCIS, Pakistan Institute of Engineering and Applied Sciences (PIEAS), Nilore, Islamabad, 45650, Pakistan
| | - Asifullah Khan
- Pattern Recognition Lab, DCIS, Pakistan Institute of Engineering and Applied Sciences (PIEAS), Nilore, Islamabad, 45650, Pakistan.
- Deep Learning Lab, Centre for Mathematical Sciences, Pakistan Institute of Engineering and Applied Sciences, (PIEAS), Nilore, Islamabad, 45650, Pakistan.
| | - Noorul Wahab
- Department of Computer Science, Tissue Image Analytics (TIA) Lab, University of Warwick, Coventry, UK
| | - Aneela Zameer
- Pattern Recognition Lab, DCIS, Pakistan Institute of Engineering and Applied Sciences (PIEAS), Nilore, Islamabad, 45650, Pakistan
| | - Saranjam Khan
- Department of Physics, Islamia College Peshawar, Peshawar, Pakistan
| |
Collapse
|
17
|
RapidBrachyDL: Rapid Radiation Dose Calculations in Brachytherapy Via Deep Learning. Int J Radiat Oncol Biol Phys 2020; 108:802-812. [DOI: 10.1016/j.ijrobp.2020.04.045] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2020] [Revised: 04/23/2020] [Accepted: 04/30/2020] [Indexed: 12/20/2022]
|
18
|
Meijering E. A bird's-eye view of deep learning in bioimage analysis. Comput Struct Biotechnol J 2020; 18:2312-2325. [PMID: 32994890 PMCID: PMC7494605 DOI: 10.1016/j.csbj.2020.08.003] [Citation(s) in RCA: 60] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2020] [Revised: 07/26/2020] [Accepted: 08/01/2020] [Indexed: 02/07/2023] Open
Abstract
Deep learning of artificial neural networks has become the de facto standard approach to solving data analysis problems in virtually all fields of science and engineering. Also in biology and medicine, deep learning technologies are fundamentally transforming how we acquire, process, analyze, and interpret data, with potentially far-reaching consequences for healthcare. In this mini-review, we take a bird's-eye view at the past, present, and future developments of deep learning, starting from science at large, to biomedical imaging, and bioimage analysis in particular.
Collapse
Affiliation(s)
- Erik Meijering
- School of Computer Science and Engineering & Graduate School of Biomedical Engineering, University of New South Wales, Sydney, Australia
| |
Collapse
|
19
|
|
20
|
Heuristic Analysis for In-Plane Non-Contact Calibration of Rulers Using Mask R-CNN. INFORMATION 2020. [DOI: 10.3390/info11050259] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Determining an object measurement is a challenging task without having a well-defined reference. When a ruler is placed in the same plane of an object being measured it can serve as metric reference, thus a measurement system can be defined and calibrated to correlate actual dimensions with pixels contained in an image. This paper describes a system for non-contact object measurement by sensing and assessing the distinct spatial frequency of the graduations on a ruler. The approach presented leverages Deep Learning methods, specifically Mask Region proposal based Convolutional Neural Networks (R-CNN), for rulers’ recognition and segmentation, as well as several other computer vision (CV) methods such as adaptive thresholding and template matching. We developed a heuristic analytical method for calibrating an image by applying several filters to extract the spatial frequencies corresponding to the ticks on a given ruler. We propose an automated in-plane optical scaling calibration system for non-contact measurement.
Collapse
|