1
|
Wang H, Jin Q, Li S, Liu S, Wang M, Song Z. A comprehensive survey on deep active learning in medical image analysis. Med Image Anal 2024; 95:103201. [PMID: 38776841 DOI: 10.1016/j.media.2024.103201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 04/25/2024] [Accepted: 05/06/2024] [Indexed: 05/25/2024]
Abstract
Deep learning has achieved widespread success in medical image analysis, leading to an increasing demand for large-scale expert-annotated medical image datasets. Yet, the high cost of annotating medical images severely hampers the development of deep learning in this field. To reduce annotation costs, active learning aims to select the most informative samples for annotation and train high-performance models with as few labeled samples as possible. In this survey, we review the core methods of active learning, including the evaluation of informativeness and sampling strategy. For the first time, we provide a detailed summary of the integration of active learning with other label-efficient techniques, such as semi-supervised, self-supervised learning, and so on. We also summarize active learning works that are specifically tailored to medical image analysis. Additionally, we conduct a thorough comparative analysis of the performance of different AL methods in medical image analysis with experiments. In the end, we offer our perspectives on the future trends and challenges of active learning and its applications in medical image analysis. An accompanying paper list and code for the comparative analysis is available in https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysis.
Collapse
Affiliation(s)
- Haoran Wang
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China; Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China
| | - Qiuye Jin
- Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
| | - Shiman Li
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China; Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China
| | - Siyu Liu
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China; Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China
| | - Manning Wang
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China; Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China.
| | - Zhijian Song
- Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai 200032, China; Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention, Shanghai 200032, China.
| |
Collapse
|
2
|
Huang Z, Yang E, Shen J, Gratzinger D, Eyerer F, Liang B, Nirschl J, Bingham D, Dussaq AM, Kunder C, Rojansky R, Gilbert A, Chang-Graham AL, Howitt BE, Liu Y, Ryan EE, Tenney TB, Zhang X, Folkins A, Fox EJ, Montine KS, Montine TJ, Zou J. A pathologist-AI collaboration framework for enhancing diagnostic accuracies and efficiencies. Nat Biomed Eng 2024:10.1038/s41551-024-01223-5. [PMID: 38898173 DOI: 10.1038/s41551-024-01223-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 05/03/2024] [Indexed: 06/21/2024]
Abstract
In pathology, the deployment of artificial intelligence (AI) in clinical settings is constrained by limitations in data collection and in model transparency and interpretability. Here we describe a digital pathology framework, nuclei.io, that incorporates active learning and human-in-the-loop real-time feedback for the rapid creation of diverse datasets and models. We validate the effectiveness of the framework via two crossover user studies that leveraged collaboration between the AI and the pathologist, including the identification of plasma cells in endometrial biopsies and the detection of colorectal cancer metastasis in lymph nodes. In both studies, nuclei.io yielded considerable diagnostic performance improvements. Collaboration between clinicians and AI will aid digital pathology by enhancing accuracies and efficiencies.
Collapse
Affiliation(s)
- Zhi Huang
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA
| | - Eric Yang
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Jeanne Shen
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Dita Gratzinger
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Frederick Eyerer
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Brooke Liang
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Jeffrey Nirschl
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - David Bingham
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Alex M Dussaq
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Christian Kunder
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Rebecca Rojansky
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Aubre Gilbert
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | | | - Brooke E Howitt
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Ying Liu
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Emily E Ryan
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Troy B Tenney
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Xiaoming Zhang
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Ann Folkins
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Edward J Fox
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Kathleen S Montine
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | - Thomas J Montine
- Department of Pathology, Stanford University School of Medicine, Stanford, CA, USA.
| | - James Zou
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA.
| |
Collapse
|
3
|
Hasan E, Duhaime E, Trueblood JS. Boosting wisdom of the crowd for medical image annotation using training performance and task features. Cogn Res Princ Implic 2024; 9:31. [PMID: 38763994 PMCID: PMC11102897 DOI: 10.1186/s41235-024-00558-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 04/29/2024] [Indexed: 05/21/2024] Open
Abstract
A crucial bottleneck in medical artificial intelligence (AI) is high-quality labeled medical datasets. In this paper, we test a large variety of wisdom of the crowd algorithms to label medical images that were initially classified by individuals recruited through an app-based platform. Individuals classified skin lesions from the International Skin Lesion Challenge 2018 into 7 different categories. There was a large dispersion in the geographical location, experience, training, and performance of the recruited individuals. We tested several wisdom of the crowd algorithms of varying complexity from a simple unweighted average to more complex Bayesian models that account for individual patterns of errors. Using a switchboard analysis, we observe that the best-performing algorithms rely on selecting top performers, weighting decisions by training accuracy, and take into account the task environment. These algorithms far exceed expert performance. We conclude by discussing the implications of these approaches for the development of medical AI.
Collapse
Affiliation(s)
- Eeshan Hasan
- Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN, 47405-7007, USA.
- Cognitive Science Program, Indiana University, Bloomington, USA.
| | | | - Jennifer S Trueblood
- Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN, 47405-7007, USA.
- Cognitive Science Program, Indiana University, Bloomington, USA.
| |
Collapse
|
4
|
Chai B, Efstathiou C, Yue H, Draviam VM. Opportunities and challenges for deep learning in cell dynamics research. Trends Cell Biol 2023:S0962-8924(23)00228-3. [PMID: 38030542 DOI: 10.1016/j.tcb.2023.10.010] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 09/30/2023] [Accepted: 10/13/2023] [Indexed: 12/01/2023]
Abstract
The growth of artificial intelligence (AI) has led to an increase in the adoption of computer vision and deep learning (DL) techniques for the evaluation of microscopy images and movies. This adoption has not only addressed hurdles in quantitative analysis of dynamic cell biological processes but has also started to support advances in drug development, precision medicine, and genome-phenome mapping. We survey existing AI-based techniques and tools, as well as open-source datasets, with a specific focus on the computational tasks of segmentation, classification, and tracking of cellular and subcellular structures and dynamics. We summarise long-standing challenges in microscopy video analysis from a computational perspective and review emerging research frontiers and innovative applications for DL-guided automation in cell dynamics research.
Collapse
Affiliation(s)
- Binghao Chai
- School of Biological and Behavioural Sciences, Queen Mary University of London (QMUL), London E1 4NS, UK
| | - Christoforos Efstathiou
- School of Biological and Behavioural Sciences, Queen Mary University of London (QMUL), London E1 4NS, UK
| | - Haoran Yue
- School of Biological and Behavioural Sciences, Queen Mary University of London (QMUL), London E1 4NS, UK
| | - Viji M Draviam
- School of Biological and Behavioural Sciences, Queen Mary University of London (QMUL), London E1 4NS, UK; The Alan Turing Institute, London NW1 2DB, UK.
| |
Collapse
|
5
|
Nowroozilarki Z, Mortazavi BJ, Jafari R. Variational Autoencoders for Biomedical Signal Morphology Clustering and Noise Detection. IEEE J Biomed Health Inform 2023; PP:10.1109/JBHI.2023.3320585. [PMID: 37768790 PMCID: PMC10984704 DOI: 10.1109/jbhi.2023.3320585] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/30/2023]
Abstract
Accurate estimation of physiological biomarkers using raw waveform data from non-invasive wearable devices requires extensive data preprocessing. An automatic noise detection method in time-series data would offer significant utility for various domains. As data labeling is onerous, having a minimally supervised abnormality detection method for input data, as well as an estimation of the severity of the signal corruptness, is essential. We propose a model-free, time-series biomedical waveform noise detection framework using a Variational Autoencoder coupled with Gaussian Mixture Models, which can detect a range of waveform abnormalities without annotation, providing a confidence metric for each segment. Our technique operates on biomedical signals that exhibit periodicity of heart activities. This framework can be applied to any machine learning or deep learning model as an initial signal validator component. Moreover, the confidence score generated by the proposed framework can be incorporated into different models' optimization to construct confidence-aware modeling. We conduct experiments using dynamic time warping (DTW) distance of segments to validated cardiac cycle morphology. The result confirms that our approach removes noisy cardiac cycles and the remaining signals, classified as clean, exhibit a 59.92% reduction in the standard deviation of DTW distances. Using a dataset of bio-impedance data of 97885 cardiac cycles, we further demonstrate a significant improvement in the downstream task of cuffless blood pressure estimation, with an average reduction of 2.67 mmHg root mean square error (RMSE) of Diastolic Blood pressure and 2.13 mmHg RMSE of systolic blood pressure, with increases of average Pearson correlation of 0.28 and 0.08, with a statistically significant improvement of signal-to-noise ratio respectively in the presence of different synthetic noise sources. This enables burden-free validation of wearable sensor data for downstream biomedical applications.
Collapse
|
6
|
Long E, Wan P, Chen Q, Lu Z, Choi J. From function to translation: Decoding genetic susceptibility to human diseases via artificial intelligence. CELL GENOMICS 2023; 3:100320. [PMID: 37388909 PMCID: PMC10300605 DOI: 10.1016/j.xgen.2023.100320] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/01/2023]
Abstract
While genome-wide association studies (GWAS) have discovered thousands of disease-associated loci, molecular mechanisms for a considerable fraction of the loci remain to be explored. The logical next steps for post-GWAS are interpreting these genetic associations to understand disease etiology (GWAS functional studies) and translating this knowledge into clinical benefits for the patients (GWAS translational studies). Although various datasets and approaches using functional genomics have been developed to facilitate these studies, significant challenges remain due to data heterogeneity, multiplicity, and high dimensionality. To address these challenges, artificial intelligence (AI) technology has demonstrated considerable promise in decoding complex functional datasets and providing novel biological insights into GWAS findings. This perspective first describes the landmark progress driven by AI in interpreting and translating GWAS findings and then outlines specific challenges followed by actionable recommendations related to data availability, model optimization, and interpretation, as well as ethical concerns.
Collapse
Affiliation(s)
- Erping Long
- Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Peixing Wan
- Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Qingyu Chen
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Zhiyong Lu
- National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Jiyeon Choi
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| |
Collapse
|
7
|
Ameen YA, Badary DM, Abonnoor AEI, Hussain KF, Sewisy AA. Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images. BMC Bioinformatics 2023; 24:75. [PMID: 36869300 PMCID: PMC9983182 DOI: 10.1186/s12859-023-05199-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Accepted: 02/21/2023] [Indexed: 03/05/2023] Open
Abstract
BACKGROUND Applying deep learning to digital histopathology is hindered by the scarcity of manually annotated datasets. While data augmentation can ameliorate this obstacle, its methods are far from standardized. Our aim was to systematically explore the effects of skipping data augmentation; applying data augmentation to different subsets of the whole dataset (training set, validation set, test set, two of them, or all of them); and applying data augmentation at different time points (before, during, or after dividing the dataset into three subsets). Different combinations of the above possibilities resulted in 11 ways to apply augmentation. The literature contains no such comprehensive systematic comparison of these augmentation ways. RESULTS Non-overlapping photographs of all tissues on 90 hematoxylin-and-eosin-stained urinary bladder slides were obtained. Then, they were manually classified as either inflammation (5948 images), urothelial cell carcinoma (5811 images), or invalid (3132 images; excluded). If done, augmentation was eight-fold by flipping and rotation. Four convolutional neural networks (Inception-v3, ResNet-101, GoogLeNet, and SqueezeNet), pre-trained on the ImageNet dataset, were fine-tuned to binary classify images of our dataset. This task was the benchmark for our experiments. Model testing performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. Model validation accuracy was also estimated. The best testing performance was achieved when augmentation was done to the remaining data after test-set separation, but before division into training and validation sets. This leaked information between the training and the validation sets, as evidenced by the optimistic validation accuracy. However, this leakage did not cause the validation set to malfunction. Augmentation before test-set separation led to optimistic results. Test-set augmentation yielded more accurate evaluation metrics with less uncertainty. Inception-v3 had the best overall testing performance. CONCLUSIONS In digital histopathology, augmentation should include both the test set (after its allocation), and the remaining combined training/validation set (before being split into separate training and validation sets). Future research should try to generalize our results.
Collapse
Affiliation(s)
- Yusra A Ameen
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt.
| | - Dalia M Badary
- Department of Pathology, Faculty of Medicine, Assiut University, Asyut, Egypt
| | | | - Khaled F Hussain
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt
| | - Adel A Sewisy
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt
| |
Collapse
|
8
|
Hou W, Hong L, Xu H, Yin W. RoRED: Bootstrapping Labeling Rule Discovery for Robust Relation Extraction. Inf Sci (N Y) 2023. [DOI: 10.1016/j.ins.2023.01.132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
|
9
|
Lovis C, Weber J, Liopyris K, Braun RP, Marghoob AA, Quigley EA, Nelson K, Prentice K, Duhaime E, Halpern AC, Rotemberg V. Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study. JMIR Med Inform 2023; 11:e38412. [PMID: 36652282 PMCID: PMC9892985 DOI: 10.2196/38412] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Revised: 09/28/2022] [Accepted: 10/16/2022] [Indexed: 01/19/2023] Open
Abstract
BACKGROUND Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. OBJECTIVE The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. METHODS First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. RESULTS In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. CONCLUSIONS This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.
Collapse
Affiliation(s)
| | - Jochen Weber
- Dermatology Section, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Konstantinos Liopyris
- Department of Dermatology, Andreas Syggros Hospital of Cutaneous and Venereal Diseases, University of Athens, Athens, Greece
| | - Ralph P Braun
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland
| | - Ashfaq A Marghoob
- Dermatology Section, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Elizabeth A Quigley
- Dermatology Section, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Kelly Nelson
- Department of Dermatology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | | | | | - Allan C Halpern
- Dermatology Section, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Veronica Rotemberg
- Dermatology Section, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| |
Collapse
|
10
|
Wahab N, Miligy IM, Dodd K, Sahota H, Toss M, Lu W, Jahanifar M, Bilal M, Graham S, Park Y, Hadjigeorghiou G, Bhalerao A, Lashen AG, Ibrahim AY, Katayama A, Ebili HO, Parkin M, Sorell T, Raza SEA, Hero E, Eldaly H, Tsang YW, Gopalakrishnan K, Snead D, Rakha E, Rajpoot N, Minhas F. Semantic annotation for computational pathology: multidisciplinary experience and best practice recommendations. JOURNAL OF PATHOLOGY CLINICAL RESEARCH 2022; 8:116-128. [PMID: 35014198 PMCID: PMC8822374 DOI: 10.1002/cjp2.256] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 11/25/2021] [Accepted: 12/10/2021] [Indexed: 02/06/2023]
Abstract
Recent advances in whole‐slide imaging (WSI) technology have led to the development of a myriad of computer vision and artificial intelligence‐based diagnostic, prognostic, and predictive algorithms. Computational Pathology (CPath) offers an integrated solution to utilise information embedded in pathology WSIs beyond what can be obtained through visual assessment. For automated analysis of WSIs and validation of machine learning (ML) models, annotations at the slide, tissue, and cellular levels are required. The annotation of important visual constructs in pathology images is an important component of CPath projects. Improper annotations can result in algorithms that are hard to interpret and can potentially produce inaccurate and inconsistent results. Despite the crucial role of annotations in CPath projects, there are no well‐defined guidelines or best practices on how annotations should be carried out. In this paper, we address this shortcoming by presenting the experience and best practices acquired during the execution of a large‐scale annotation exercise involving a multidisciplinary team of pathologists, ML experts, and researchers as part of the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) consortium. We present a real‐world case study along with examples of different types of annotations, diagnostic algorithm, annotation data dictionary, and annotation constructs. The analyses reported in this work highlight best practice recommendations that can be used as annotation guidelines over the lifecycle of a CPath project.
Collapse
Affiliation(s)
- Noorul Wahab
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Islam M Miligy
- Pathology, University of Nottingham, Nottingham, UK.,Department of Pathology, Faculty of Medicine, Menoufia University, Shebin El-Kom, Egypt
| | - Katherine Dodd
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Harvir Sahota
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Michael Toss
- Pathology, University of Nottingham, Nottingham, UK
| | - Wenqi Lu
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | - Mohsin Bilal
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Simon Graham
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Young Park
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | - Abhir Bhalerao
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | | | | | - Ayaka Katayama
- Graduate School of Medicine, Gunma University, Maebashi, Japan
| | | | | | - Tom Sorell
- Department of Politics and International Studies, University of Warwick, Coventry, UK
| | | | - Emily Hero
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK.,Leicester Royal Infirmary, Histopathology, University Hospitals Leicester, Leicester, UK
| | - Hesham Eldaly
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Yee Wah Tsang
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | | | - David Snead
- Histopathology, University Hospital Coventry and Warwickshire, Coventry, UK
| | - Emad Rakha
- Pathology, University of Nottingham, Nottingham, UK
| | - Nasir Rajpoot
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| | - Fayyaz Minhas
- Tissue Image Analytics Centre, University of Warwick, Coventry, UK
| |
Collapse
|
11
|
Chen SB, Novoa RA. Artificial intelligence for dermatopathology: Current trends and the road ahead. Semin Diagn Pathol 2022; 39:298-304. [DOI: 10.1053/j.semdp.2022.01.003] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2021] [Revised: 01/06/2022] [Accepted: 01/12/2022] [Indexed: 02/07/2023]
|
12
|
Diao JA, Chen RJ, Kvedar JC. Efficient cellular annotation of histopathology slides with real-time AI augmentation. NPJ Digit Med 2021; 4:161. [PMID: 34811479 PMCID: PMC8608794 DOI: 10.1038/s41746-021-00534-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Accepted: 10/26/2021] [Indexed: 01/16/2023] Open
Abstract
In recent years, a steady swell of biological image data has driven rapid progress in healthcare applications of computer vision and machine learning. To make sense of this data, scientists often rely on detailed annotations from domain experts for training artificial intelligence (AI) algorithms. The time-consuming and costly process of collecting annotations presents a sizable bottleneck for AI research and development. HALS (Human-Augmenting Labeling System) is a collaborative human-AI labeling workflow that uses an iterative “review-and-revise” model to improve the efficiency of this critical process in computational pathology.
Collapse
Affiliation(s)
- James A Diao
- Program in Health Sciences and Technology, Harvard Medical School, Boston, MA, USA.
| | - Richard J Chen
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Joseph C Kvedar
- Department of Dermatology, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|