1
|
Huang Y, Asaria R, Stoyanov D, Sarunic M, Bano S. PseudoSegRT: efficient pseudo-labelling for intraoperative OCT segmentation. Int J Comput Assist Radiol Surg 2023:10.1007/s11548-023-02928-9. [PMID: 37233893 PMCID: PMC10329588 DOI: 10.1007/s11548-023-02928-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 04/19/2023] [Indexed: 05/27/2023]
Abstract
PURPOSE Robotic ophthalmic microsurgery has significant potential to help improve the success of challenging procedures and overcome the physical limitations of the surgeon. Intraoperative optical coherence tomography (iOCT) has been reported for the visualisation of ophthalmic surgical manoeuvres, where deep learning methods can be used for real-time tissue segmentation and surgical tool tracking. However, many of these methods rely heavily on labelled datasets, where producing annotated segmentation datasets is a time-consuming and tedious task. METHODS To address this challenge, we propose a robust and efficient semi-supervised method for boundary segmentation in retinal OCT to guide a robotic surgical system. The proposed method uses U-Net as the base model and implements a pseudo-labelling strategy which combines the labelled data with unlabelled OCT scans during training. After training, the model is optimised and accelerated with the use of TensorRT. RESULTS Compared with fully supervised learning, the pseudo-labelling method can improve the generalisability of the model and show better performance for unseen data from a different distribution using only 2% of labelled training samples. The accelerated GPU inference takes less than 1 millisecond per frame with FP16 precision. CONCLUSION Our approach demonstrates the potential of using pseudo-labelling strategies in real-time OCT segmentation tasks to guide robotic systems. Furthermore, the accelerated GPU inference of our network is highly promising for segmenting OCT images and guiding the position of a surgical tool (e.g. needle) for sub-retinal injections.
Collapse
Affiliation(s)
- Yu Huang
- Department of Computer Science, University College London, London, UK
| | - Riaz Asaria
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
- Ophthalmology, Royal Free Hospital, London, UK
| | - Danail Stoyanov
- Department of Computer Science, University College London, London, UK
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK
| | - Marinko Sarunic
- Institute of Ophthalmology, University College London, London, UK
- Department of Medical Physics and Biomedical Engineering, University College London, London, UK
| | - Sophia Bano
- Department of Computer Science, University College London, London, UK.
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London, London, UK.
| |
Collapse
|
2
|
Lindovsky J, Palkova M, Symkina V, Raishbrook MJ, Prochazka J, Sedlacek R. OCT and ERG Techniques in High-Throughput Phenotyping of Mouse Vision. Genes (Basel) 2023; 14:genes14020294. [PMID: 36833221 PMCID: PMC9956909 DOI: 10.3390/genes14020294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/17/2023] [Accepted: 01/20/2023] [Indexed: 01/26/2023] Open
Abstract
The purpose of the study is to demonstrate coherent optical tomography and electroretinography techniques adopted from the human clinical practice to assess the morphology and function of the mouse retina in a high-throughput phenotyping environment. We present the normal range of wild-type C57Bl/6NCrl retinal parameters in six age groups between 10 and 100 weeks as well as examples of mild and severe pathologies resulting from knocking out a single protein-coding gene. We also show example data obtained by more detailed analysis or additional methods useful in eye research, for example, the angiography of a superficial and deep vascular complex. We discuss the feasibility of these techniques in conditions demanding a high-throughput approach such as the systemic phenotyping carried out by the International Mouse Phenotyping Consortium.
Collapse
|
3
|
Kopp R, Joseph J, Ni X, Roy N, Wardle BL. Deep Learning Unlocks X-ray Microtomography Segmentation of Multiclass Microdamage in Heterogeneous Materials. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2022; 34:e2107817. [PMID: 34800056 DOI: 10.1002/adma.202107817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Revised: 11/16/2021] [Indexed: 06/13/2023]
Abstract
Four-dimensional quantitative characterization of heterogeneous materials using in situ synchrotron radiation computed tomography can reveal 3D sub-micrometer features, particularly damage, evolving under load, leading to improved materials. However, dataset size and complexity increasingly require time-intensive and subjective semi-automatic segmentations. Here, the first deep learning (DL) convolutional neural network (CNN) segmentation of multiclass microscale damage in heterogeneous bulk materials is presented, teaching on advanced aerospace-grade composite damage using ≈65 000 (trained) human-segmented tomograms. The trained CNN machine segments complex and sparse (<<1% of volume) composite damage classes to ≈99.99% agreement, unlocking both objectivity and efficiency, with nearly 100% of the human time eliminated, which traditional rule-based algorithms do not approach. The trained machine is found to perform as well or better than the human due to "machine-discovered" human segmentation error, with machine improvements manifesting primarily as new damage discovery and segmentation augmentation/extension in artifact-rich tomograms. Interrogating a high-level network hyperparametric space on two material configurations, DL is found to be a disruptive approach to quantitative structure-property characterization, enabling high-throughput knowledge creation (accelerated by two orders of magnitude) via generalizable, ultrahigh-resolution feature segmentation.
Collapse
Affiliation(s)
- Reed Kopp
- Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
| | - Joshua Joseph
- MIT Quest for Intelligence, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
| | - Xinchen Ni
- Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
| | - Nicholas Roy
- Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
- MIT Quest for Intelligence, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
| | - Brian L Wardle
- Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
- Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA
| |
Collapse
|
4
|
Morales S, Colomer A, Mossi JM, Del Amor R, Woldbye D, Klemp K, Larsen M, Naranjo V. Retinal layer segmentation in rodent OCT images: Local intensity profiles & fully convolutional neural networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 198:105788. [PMID: 33130492 DOI: 10.1016/j.cmpb.2020.105788] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2019] [Accepted: 09/30/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Optical coherence tomography (OCT) is a useful technique to monitor retinal layer state both in humans and animal models. Automated OCT analysis in rats is of great relevance to study possible toxic effect of drugs and other treatments before human trials. In this paper, two different approaches to detect the most significant retinal layers in a rat OCT image are presented. METHODS One approach is based on a combination of local horizontal intensity profiles along with a new proposed variant of watershed transformation and the other is built upon an encoder-decoder convolutional network architecture. RESULTS After a wide validation, an averaged absolute distance error of 3.77 ± 2.59 and 1.90 ± 0.91 µm is achieved by both approaches, respectively, on a batch of the rat OCT database. After a second test of the deep-learning-based method using an unseen batch of the database, an averaged absolute distance error of 2.67 ± 1.25 µm is obtained. The rat OCT database used in this paper is made publicly available to facilitate further comparisons. CONCLUSIONS Based on the obtained results, it was demonstrated the competitiveness of the first approach since outperforms the commercial Insight image segmentation software (Phoenix Research Labs) as well as its utility to generate labelled images for validation purposes speeding significantly up the ground truth generation process. Regarding the second approach, the deep-learning-based method improves the results achieved by the more conventional method and also by other state-of-the-art techniques. In addition, it was verified that the results of the proposed network can be generalized to new rat OCT images.
Collapse
Affiliation(s)
- Sandra Morales
- Instituto de Investigación e Innovación en Bioingeniería, I3B, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain.
| | - Adrián Colomer
- Instituto de Investigación e Innovación en Bioingeniería, I3B, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
| | - José M Mossi
- ITEAM Research Institute, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
| | - Rocío Del Amor
- Instituto de Investigación e Innovación en Bioingeniería, I3B, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
| | - David Woldbye
- Laboratory of Neural Plasticity, Department of Neuroscience, University of Copenhagen, Denmark
| | - Kristian Klemp
- Dept. of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Copenhagen, Denmark; Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Michael Larsen
- Dept. of Ophthalmology, Rigshospitalet-Glostrup, Glostrup, Copenhagen, Denmark; Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Valery Naranjo
- Instituto de Investigación e Innovación en Bioingeniería, I3B, Universitat Politècnica de València, Camino de Vera s/n, 46022 Valencia, Spain
| |
Collapse
|
5
|
Carass A, Roy S, Gherman A, Reinhold JC, Jesson A, Arbel T, Maier O, Handels H, Ghafoorian M, Platel B, Birenbaum A, Greenspan H, Pham DL, Crainiceanu CM, Calabresi PA, Prince JL, Roncal WRG, Shinohara RT, Oguz I. Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis. Sci Rep 2020; 10:8242. [PMID: 32427874 PMCID: PMC7237671 DOI: 10.1038/s41598-020-64803-w] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Accepted: 04/20/2020] [Indexed: 11/09/2022] Open
Abstract
The Sørensen-Dice index (SDI) is a widely used measure for evaluating medical image segmentation algorithms. It offers a standardized measure of segmentation accuracy which has proven useful. However, it offers diminishing insight when the number of objects is unknown, such as in white matter lesion segmentation of multiple sclerosis (MS) patients. We present a refinement for finer grained parsing of SDI results in situations where the number of objects is unknown. We explore these ideas with two case studies showing what can be learned from our two presented studies. Our first study explores an inter-rater comparison, showing that smaller lesions cannot be reliably identified. In our second case study, we demonstrate fusing multiple MS lesion segmentation algorithms based on the insights into the algorithms provided by our analysis to generate a segmentation that exhibits improved performance. This work demonstrates the wealth of information that can be learned from refined analysis of medical image segmentations.
Collapse
Affiliation(s)
- Aaron Carass
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA.
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA.
| | - Snehashis Roy
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Adrian Gherman
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Jacob C Reinhold
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Andrew Jesson
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Tal Arbel
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Oskar Maier
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Heinz Handels
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Mohsen Ghafoorian
- Institute for Computing and Information Sciences, Radboud University, 6525, HP, Nijmegen, Netherlands
| | - Bram Platel
- Diagnostic Image Analysis Group, Radboud University Medical Center, 6525, GA, Nijmegen, Netherlands
| | - Ariel Birenbaum
- Department of Electrical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Hayit Greenspan
- Department of Biomedical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Dzung L Pham
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Ciprian M Crainiceanu
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Peter A Calabresi
- Department of Neurology, The Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - William R Gray Roncal
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics & Epidemiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Center for Biomedical Image Computing and Analytics, Department of Radiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Ipek Oguz
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, 37203, USA
| |
Collapse
|
6
|
Antony BJ, Carass A, Lang A, Kim BJ, Zack DJ, Prince JL. Longitudinal Analysis of Mouse SDOCT Volumes. ACTA ACUST UNITED AC 2017; 10137. [PMID: 29138527 DOI: 10.1117/12.2257432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Spectral-domain optical coherence tomography (SDOCT), in addition to its routine clinical use in the diagnosis of ocular diseases, has begun to find increasing use in animal studies. Animal models are frequently used to study disease mechanisms as well as to test drug efficacy. In particular, SDOCT provides the ability to study animals longitudinally and non-invasively over long periods of time. However, the lack of anatomical landmarks makes the longitudinal scan acquisition prone to inconsistencies in orientation. Here, we propose a method for the automated registration of mouse SDOCT volumes. The method begins by accurately segmenting the blood vessels and the optic nerve head region in the scans using a pixel classification approach. The segmented vessel maps from follow-up scans were registered using an iterative closest point (ICP) algorithm to the baseline scan to allow for the accurate longitudinal tracking of thickness changes. Eighteen SDOCT volumes from a light damage model study were used to train a random forest utilized in the pixel classification step. The area under the curve (AUC) in a leave-one-out study for the retinal blood vessels and the optic nerve head (ONH) was found to be 0.93 and 0.98, respectively. The complete proposed framework, the retinal vasculature segmentation and the ICP registration, was applied to a secondary set of scans obtained from a light damage model. A qualitative assessment of the registration showed no registration failures.
Collapse
Affiliation(s)
- Bhavna J Antony
- Department of Electrical and Computer Engineering, Johns Hopkins University
| | - Aaron Carass
- Department of Electrical and Computer Engineering, Johns Hopkins University
| | - Andrew Lang
- Department of Electrical and Computer Engineering, Johns Hopkins University
| | - Byung-Jin Kim
- Wilmer Eye Institute, Johns Hopkins University School of Medicine
| | - Donald J Zack
- Wilmer Eye Institute, Johns Hopkins University School of Medicine
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, Johns Hopkins University
| |
Collapse
|