1
|
Lopes A, Ward AD, Cecchini M. Eye tracking in digital pathology: A comprehensive literature review. J Pathol Inform 2024; 15:100383. [PMID: 38868488 PMCID: PMC11168484 DOI: 10.1016/j.jpi.2024.100383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 04/28/2024] [Accepted: 05/14/2024] [Indexed: 06/14/2024] Open
Abstract
Eye tracking has been used for decades in attempt to understand the cognitive processes of individuals. From memory access to problem-solving to decision-making, such insight has the potential to improve workflows and the education of students to become experts in relevant fields. Until recently, the traditional use of microscopes in pathology made eye tracking exceptionally difficult. However, the digital revolution of pathology from conventional microscopes to digital whole slide images allows for new research to be conducted and information to be learned with regards to pathologist visual search patterns and learning experiences. This has the promise to make pathology education more efficient and engaging, ultimately creating stronger and more proficient generations of pathologists to come. The goal of this review on eye tracking in pathology is to characterize and compare the visual search patterns of pathologists. The PubMed and Web of Science databases were searched using 'pathology' AND 'eye tracking' synonyms. A total of 22 relevant full-text articles published up to and including 2023 were identified and included in this review. Thematic analysis was conducted to organize each study into one or more of the 10 themes identified to characterize the visual search patterns of pathologists: (1) effect of experience, (2) fixations, (3) zooming, (4) panning, (5) saccades, (6) pupil diameter, (7) interpretation time, (8) strategies, (9) machine learning, and (10) education. Expert pathologists were found to have higher diagnostic accuracy, fewer fixations, and shorter interpretation times than pathologists with less experience. Further, literature on eye tracking in pathology indicates that there are several visual strategies for diagnostic interpretation of digital pathology images, but no evidence of a superior strategy exists. The educational implications of eye tracking in pathology have also been explored but the effect of teaching novices how to search as an expert remains unclear. In this article, the main challenges and prospects of eye tracking in pathology are briefly discussed along with their implications to the field.
Collapse
Affiliation(s)
- Alana Lopes
- Department of Medical Biophysics, Western University, London, ON N6A 3K7, Canada
- Gerald C. Baines Centre, London Health Sciences Centre, London, ON N6A 5W9, Canada
| | - Aaron D. Ward
- Department of Medical Biophysics, Western University, London, ON N6A 3K7, Canada
- Gerald C. Baines Centre, London Health Sciences Centre, London, ON N6A 5W9, Canada
- Department of Oncology, Western University, London, ON N6A 3K7, Canada
| | - Matthew Cecchini
- Department of Pathology and Laboratory Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON N6A 3K7, Canada
| |
Collapse
|
2
|
Ghezloo F, Chang OH, Knezevich SR, Shaw KC, Thigpen KG, Reisch LM, Shapiro LG, Elmore JG. Robust ROI Detection in Whole Slide Images Guided by Pathologists' Viewing Patterns. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01202-x. [PMID: 39122892 DOI: 10.1007/s10278-024-01202-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Revised: 06/24/2024] [Accepted: 07/05/2024] [Indexed: 08/12/2024]
Abstract
Deep learning techniques offer improvements in computer-aided diagnosis systems. However, acquiring image domain annotations is challenging due to the knowledge and commitment required of expert pathologists. Pathologists often identify regions in whole slide images with diagnostic relevance rather than examining the entire slide, with a positive correlation between the time spent on these critical image regions and diagnostic accuracy. In this paper, a heatmap is generated to represent pathologists' viewing patterns during diagnosis and used to guide a deep learning architecture during training. The proposed system outperforms traditional approaches based on color and texture image characteristics, integrating pathologists' domain expertise to enhance region of interest detection without needing individual case annotations. Evaluating our best model, a U-Net model with a pre-trained ResNet-18 encoder, on a skin biopsy whole slide image dataset for melanoma diagnosis, shows its potential in detecting regions of interest, surpassing conventional methods with an increase of 20%, 11%, 22%, and 12% in precision, recall, F1-score, and Intersection over Union, respectively. In a clinical evaluation, three dermatopathologists agreed on the model's effectiveness in replicating pathologists' diagnostic viewing behavior and accurately identifying critical regions. Finally, our study demonstrates that incorporating heatmaps as supplementary signals can enhance the performance of computer-aided diagnosis systems. Without the availability of eye tracking data, identifying precise focus areas is challenging, but our approach shows promise in assisting pathologists in improving diagnostic accuracy and efficiency, streamlining annotation processes, and aiding the training of new pathologists.
Collapse
Affiliation(s)
- Fatemeh Ghezloo
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA.
| | - Oliver H Chang
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA
| | | | | | | | - Lisa M Reisch
- Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Linda G Shapiro
- Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA
| | - Joann G Elmore
- Department of Medicine, David Geffen School of Medicine, University of California, Los AngelesLos Angeles, CA, USA
| |
Collapse
|
3
|
Ibragimov B, Mello-Thoms C. The Use of Machine Learning in Eye Tracking Studies in Medical Imaging: A Review. IEEE J Biomed Health Inform 2024; 28:3597-3612. [PMID: 38421842 PMCID: PMC11262011 DOI: 10.1109/jbhi.2024.3371893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/02/2024]
Abstract
Machine learning (ML) has revolutionized medical image-based diagnostics. In this review, we cover a rapidly emerging field that can be potentially significantly impacted by ML - eye tracking in medical imaging. The review investigates the clinical, algorithmic, and hardware properties of the existing studies. In particular, it evaluates 1) the type of eye-tracking equipment used and how the equipment aligns with study aims; 2) the software required to record and process eye-tracking data, which often requires user interface development, and controller command and voice recording; 3) the ML methodology utilized depending on the anatomy of interest, gaze data representation, and target clinical application. The review concludes with a summary of recommendations for future studies, and confirms that the inclusion of gaze data broadens the ML applicability in Radiology from computer-aided diagnosis (CAD) to gaze-based image annotation, physicians' error detection, fatigue recognition, and other areas of potentially high research and clinical impact.
Collapse
|
4
|
Umer F, Adnan S, Lal A. Research and application of artificial intelligence in dentistry from lower-middle income countries - a scoping review. BMC Oral Health 2024; 24:220. [PMID: 38347508 PMCID: PMC10860267 DOI: 10.1186/s12903-024-03970-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 02/02/2024] [Indexed: 02/15/2024] Open
Abstract
Artificial intelligence (AI) has been integrated into dentistry for improvement of current dental practice. While many studies have explored the utilization of AI in various fields, the potential of AI in dentistry, particularly in low-middle income countries (LMICs) remains understudied. This scoping review aimed to study the existing literature on the applications of artificial intelligence in dentistry in low-middle income countries. A comprehensive search strategy was applied utilizing three major databases: PubMed, Scopus, and EBSCO Dentistry & Oral Sciences Source. The search strategy included keywords related to AI, Dentistry, and LMICs. The initial search yielded a total of 1587, out of which 25 articles were included in this review. Our findings demonstrated that limited studies have been carried out in LMICs in terms of AI and dentistry. Most of the studies were related to Orthodontics. In addition gaps in literature were noted such as cost utility and patient experience were not mentioned in the included studies.
Collapse
Affiliation(s)
- Fahad Umer
- Department of Surgery, Section of Dentistry, The Aga Khan University, Karachi, Pakistan
| | - Samira Adnan
- Department of Operative Dentistry, Sindh Institute of Oral Health Sciences, Jinnah Sindh Medical University, Karachi, Pakistan
| | - Abhishek Lal
- Department of Medicine, Section of Gastroenterology, The Aga Khan University, Karachi, Pakistan.
| |
Collapse
|
5
|
Lan K, Cheng J, Jiang J, Jiang X, Zhang Q. Modified UNet++ with atrous spatial pyramid pooling for blood cell image segmentation. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:1420-1433. [PMID: 36650817 DOI: 10.3934/mbe.2023064] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Blood cell image segmentation is an important part of the field of computer-aided diagnosis. However, due to the low contrast, large differences in cell morphology and the scarcity of labeled images, the segmentation performance of cells cannot meet the requirements of an actual diagnosis. To address the above limitations, we present a deep learning-based approach to study cell segmentation on pathological images. Specifically, the algorithm selects UNet++ as the backbone network to extract multi-scale features. Then, the skip connection is redesigned to improve the degradation problem and reduce the computational complexity. In addition, the atrous spatial pyramid pooling (ASSP) is introduced to obtain cell image information features from each layer through different receptive domains. Finally, the multi-sided output fusion (MSOF) strategy is utilized to fuse the features of different semantic levels, so as to improve the accuracy of target segmentation. Experimental results on blood cell images for segmentation and classification (BCISC) dataset show that the proposed method has significant improvement in Matthew's correlation coefficient (Mcc), Dice and Jaccard values, which are better than the classical semantic segmentation network.
Collapse
Affiliation(s)
- Kun Lan
- College of Mechanical Engineering, Quzhou University, Quzhou 324000, China
| | - Jianzhen Cheng
- Department of Rehabilitation, Quzhou Third Hospital, Quzhou 324000, China
| | - Jinyun Jiang
- College of Mechanical Engineering, Quzhou University, Quzhou 324000, China
| | - Xiaoliang Jiang
- College of Mechanical Engineering, Quzhou University, Quzhou 324000, China
| | - Qile Zhang
- Department of Rehabilitation, The Quzhou Affiliated Hospital of Wenzhou Medical University, Quzhou People's Hospital, Quzhou 324000, China
| |
Collapse
|